|By Michael Bushong||
|May 28, 2014 06:00 AM EDT||
The rapid growth in compute demand is well understood. To keep up with accelerating requirements, CPUs have gone through a massive transformation over the years. Starting with relatively low-capacity CPUs, the expansion of capability to what is available today has certainly been remarkable – enough to satisfy even Gordon Moore. But keeping up with demand was not a matter of simply making bigger and faster chips. To get more capacity, we actually went smaller.
As it turns out, there are practical limitations to just scaling things larger. To get more capacity out of individual CPUs, we went from large single cores to multi-core processors. This obviously required a change in applications to take advantage of multiple cores. The result is a distributed architecture and the proliferation of “scale out” as a buzzword in our industry.
From an application perspective, the trend continues. Applications that require performance continue to move to multi-tiered applications that are distributed across a number of VMs. This is true for massive web-scale applications like Facebook, but also for other applications like MapReduce.
To get bigger, we get smaller
The technology trend is clear: to get more output, move to smaller blocks of capacity, and coordinate workloads across that capacity.
If this is true, then the future will be lots of small pools of resources that rely on the network for interconnectivity. As applications become more distributed, then performance between these pools becomes even more critical. Even small amounts of pool-to-pool latency can aggregate up into significant impacts, either because of interesting failure conditions with asynchronous operations or because of the cumulative performance impact.
As interconnectivity takes a larger role, we should expect the discussion of commoditization of network resources to expand. Today, there is a strong argument around commoditizing the switch hardware (largely via merchant silicon) and the switch operating system (through players like Cumulus, Big Switch, and Pica8). But massive distribution will require both a commoditized interconnect and a commoditized orchestration platform.
On the latter, it would seem that OpenDaylight is poised to lead the charge. With an industry-backed open source solution, it will be difficult to justify premium control products, which should be sufficient in driving that aspect of the solution towards commodity. But that still leaves the interconnect piece unaccounted for.
Getting to a cheaper interconnect
There is probably a case to be made for leaf-spine architectures here, but if the number of servers continues to expand, there are some ugly economics at play. Scaling out in a leaf-spine architecture requires scaling up at the same time. As the interconnect demands increase, the number of spine switches increases. You eventually get into spines of spines, which starts to look an awful like like traditional three-tier architectures.
The sheer number of devices and cables drive the cost unfavorably. And when you consider the long-term operational costs tied to power, cooling, space, and management, it’s unclear where the budgetary breaking point is. Beyond just the costs, the other issue here is that every time a new layer is added, you add a couple of more fabric switch hops. If application performance is based on both capacity and latency, then every time you add switch hops, you incur a potentially heavy performance penalty.
At some point, you need to move away from multi-hop connectivity through the fabric.
Moving away from multi-hop fabrics
Instinctively, we already know this. There is already a tendency to rack gear up in close proximity to other gear to which it is tied. You might, for example, balance Hadoop loads across a number of servers that are in the same rack. Essentially, what we are doing in these cases is acknowledging that proximity matters, and we are statically designing for it.
But what happens when things aren’t static?
In a datacenter where applications are portable across servers, the network capacity cannot be statically planned. And as application requirements change (often dynamically as load changes), then the network capacity demands will also change. This requires an interconnect that is both high in capacity and dynamic.
This problem is slightly different than the compute problem. On the compute side, it was enough to free up resources (or create additional ones) and then move the application to the resource. In this case, the application is fixed, which means the capacity has to move to the application. When capacity is statically allocated, this poses a problem.
The bottom line
The only solutions here are to either over provision everything, or move towards a dynamic interconnect. The first is counter to the trends we learn from compute – make things smaller and more distributed. In this case you get out of the problem by paying for it. The question is whether this flies in the face of all the commoditization trends. What good is commoditizing something if the end solution requires buying a ton more? You would have to see cost declines match capacity increases, but this seems unlikely as there is no upper limit for capacity whereas cost will asymptotically approach some profit threshold.
If the trends in compute and storage hold true for networking, then the current trajectory of some networking solutions will need to change. Learning from the past is a great way to shape the future.
[Today’s fun fact: Lobster was one of the main entrees at the first Thanksgiving dinner. They also had Cheddar Bay Biscuits I think.]
Adobe is changing the world though digital experiences. Adobe helps customers develop and deliver high-impact experiences that differentiate brands, build loyalty, and drive revenue across every screen, including smartphones, computers, tablets and TVs. Adobe content solutions are used daily by millions of companies worldwide-from publishers and broadcasters, to enterprises, marketing agencies and household-name brands. Building on its established design leadership, Adobe enables customers not o...
Sep. 30, 2016 07:30 AM EDT
Everyone knows that truly innovative companies learn as they go along, pushing boundaries in response to market changes and demands. What's more of a mystery is how to balance innovation on a fresh platform built from scratch with the legacy tech stack, product suite and customers that continue to serve as the business' foundation. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, will discuss why and how ReadyTalk diverted from healthy revenue an...
Sep. 30, 2016 07:15 AM EDT Reads: 2,239
Cognitive Computing is becoming the foundation for a new generation of solutions that have the potential to transform business. Unlike traditional approaches to building solutions, a cognitive computing approach allows the data to help determine the way applications are designed. This contrasts with conventional software development that begins with defining logic based on the current way a business operates. In her session at 18th Cloud Expo, Judith S. Hurwitz, President and CEO of Hurwitz & ...
Sep. 30, 2016 07:15 AM EDT Reads: 3,266
The Internet of Things will challenge the status quo of how IT and development organizations operate. Or will it? Certainly the fog layer of IoT requires special insights about data ontology, security and transactional integrity. But the developmental challenges are the same: People, Process and Platform and how we integrate our thinking to solve complicated problems. In his session at 19th Cloud Expo, Craig Sproule, CEO of Metavine, will demonstrate how to move beyond today's coding paradigm ...
Sep. 30, 2016 07:00 AM EDT Reads: 3,464
The Transparent Cloud-computing Consortium (abbreviation: T-Cloud Consortium) will conduct research activities into changes in the computing model as a result of collaboration between "device" and "cloud" and the creation of new value and markets through organic data processing High speed and high quality networks, and dramatic improvements in computer processing capabilities, have greatly changed the nature of applications and made the storing and processing of data on the network commonplace.
Sep. 30, 2016 06:00 AM EDT Reads: 1,260
Digital transformation is too big and important for our future success to not understand the rules that apply to it. The first three rules for winning in this age of hyper-digital transformation are: Advantages in speed, analytics and operational tempos must be captured by implementing an optimized information logistics system (OILS) Real-time operational tempos (IT, people and business processes) must be achieved Businesses that can "analyze data and act and with speed" will dominate those t...
Sep. 30, 2016 05:30 AM EDT Reads: 1,281
Almost two-thirds of companies either have or soon will have IoT as the backbone of their business in 2016. However, IoT is far more complex than most firms expected. How can you not get trapped in the pitfalls? In his session at @ThingsExpo, Tony Shan, a renowned visionary and thought leader, will introduce a holistic method of IoTification, which is the process of IoTifying the existing technology and business models to adopt and leverage IoT. He will drill down to the components in this fra...
Sep. 30, 2016 05:00 AM EDT Reads: 1,884
As ridesharing competitors and enhanced services increase, notable changes are occurring in the transportation model. Despite the cost-effective means and flexibility of ridesharing, both drivers and users will need to be aware of the connected environment and how it will impact the ridesharing experience. In his session at @ThingsExpo, Timothy Evavold, Executive Director Automotive at Covisint, will discuss key challenges and solutions to powering a ride sharing and/or multimodal model in the a...
Sep. 30, 2016 04:45 AM EDT Reads: 673
Internet of @ThingsExpo, taking place November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with the 19th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world and ThingsExpo Silicon Valley Call for Papers is now open.
Sep. 30, 2016 04:45 AM EDT Reads: 4,719
I'm a lonely sensor. I spend all day telling the world how I'm feeling, but none of the other sensors seem to care. I want to be connected. I want to build relationships with other sensors to be more useful for my human. I want my human to understand that when my friends next door are too hot for a while, I'll soon be flaming. And when all my friends go outside without me, I may be left behind. Don't just log my data; use the relationship graph. In his session at @ThingsExpo, Ryan Boyd, Engi...
Sep. 30, 2016 04:45 AM EDT Reads: 1,411
SYS-CON Events announced today that Pulzze Systems will exhibit at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Pulzze Systems, Inc. provides infrastructure products for the Internet of Things to enable any connected device and system to carry out matched operations without programming. For more information, visit http://www.pulzzesystems.com.
Sep. 30, 2016 04:30 AM EDT Reads: 1,925
Why do your mobile transformations need to happen today? Mobile is the strategy that enterprise transformation centers on to drive customer engagement. In his general session at @ThingsExpo, Roger Woods, Director, Mobile Product & Strategy – Adobe Marketing Cloud, covered key IoT and mobile trends that are forcing mobile transformation, key components of a solid mobile strategy and explored how brands are effectively driving mobile change throughout the enterprise.
Sep. 30, 2016 03:45 AM EDT Reads: 2,243
SYS-CON Events announced today that ReadyTalk, a leading provider of online conferencing and webinar services, has been named Vendor Presentation Sponsor at the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. ReadyTalk delivers audio and web conferencing services that inspire collaboration and enable the Future of Work for today’s increasingly digital and mobile workforce. By combining intuitive, innovative tec...
Sep. 30, 2016 03:45 AM EDT Reads: 3,091
If you’re responsible for an application that depends on the data or functionality of various IoT endpoints – either sensors or devices – your brand reputation depends on the security, reliability, and compliance of its many integrated parts. If your application fails to deliver the expected business results, your customers and partners won't care if that failure stems from the code you developed or from a component that you integrated. What can you do to ensure that the endpoints work as expect...
Sep. 30, 2016 03:15 AM EDT Reads: 1,757
WebRTC adoption has generated a wave of creative uses of communications and collaboration through websites, sales apps, customer care and business applications. As WebRTC has become more mainstream it has evolved to use cases beyond the original peer-to-peer case, which has led to a repeating requirement for interoperability with existing infrastructures. In his session at @ThingsExpo, Graham Holt, Executive Vice President of Daitan Group, will cover implementation examples that have enabled ea...
Sep. 30, 2016 03:00 AM EDT Reads: 1,632
There is growing need for data-driven applications and the need for digital platforms to build these apps. In his session at 19th Cloud Expo, Muddu Sudhakar, VP and GM of Security & IoT at Splunk, will cover different PaaS solutions and Big Data platforms that are available to build applications. In addition, AI and machine learning are creating new requirements that developers need in the building of next-gen apps. The next-generation digital platforms have some of the past platform needs a...
Sep. 30, 2016 03:00 AM EDT Reads: 1,945
So, you bought into the current machine learning craze and went on to collect millions/billions of records from this promising new data source. Now, what do you do with them? Too often, the abundance of data quickly turns into an abundance of problems. How do you extract that "magic essence" from your data without falling into the common pitfalls? In her session at @ThingsExpo, Natalia Ponomareva, Software Engineer at Google, provided tips on how to be successful in large scale machine learning...
Sep. 30, 2016 02:45 AM EDT Reads: 2,460
In his session at @ThingsExpo, Kausik Sridharabalan, founder and CTO of Pulzze Systems, Inc., will focus on key challenges in building an Internet of Things solution infrastructure. He will shed light on efficient ways of defining interactions within IoT solutions, leading to cost and time reduction. He will also introduce ways to handle data and how one can develop IoT solutions that are lean, flexible and configurable, thus making IoT infrastructure agile and scalable.
Sep. 30, 2016 02:15 AM EDT Reads: 1,641
SYS-CON Events announced today that Numerex Corp, a leading provider of managed enterprise solutions enabling the Internet of Things (IoT), will exhibit at the 19th International Cloud Expo | @ThingsExpo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Numerex Corp. (NASDAQ:NMRX) is a leading provider of managed enterprise solutions enabling the Internet of Things (IoT). The Company's solutions produce new revenue streams or create operating...
Sep. 30, 2016 02:00 AM EDT Reads: 2,088
In his general session at 18th Cloud Expo, Lee Atchison, Principal Cloud Architect and Advocate at New Relic, discussed cloud as a ‘better data center’ and how it adds new capacity (faster) and improves application availability (redundancy). The cloud is a ‘Dynamic Tool for Dynamic Apps’ and resource allocation is an integral part of your application architecture, so use only the resources you need and allocate /de-allocate resources on the fly.
Sep. 30, 2016 12:45 AM EDT Reads: 2,991