Click here to close now.


Apache Authors: Jim Scott, Liz McMillan, Craig Lowell, AppDynamics Blog, Dana Gardner

Blog Feed Post

Building a Plexxi Network

[This post is penned by Plexxi executive Bill Koss]

Two months ago I read an interesting blog post by Greg Ferro titled “Cheap Network Equipment Makes a Better Data Centre.”  At Plexxi, I am the person who leads our sales team and I found the post interesting because I think there is a lot of disinformation in the market regarding the cost of procuring networks.  Every week it seems I read a report or article on networking margins, port prices, market share, SDN, bare metal, leaf/spine, open source, linux, etc.  When I speak to customers, there is an equal amount of confusion and in my view it perpetuates the current state in favor of the incumbents.

As a level set, when a potential client asks: what does Plexxi do?  I tell them we build ethernet switches based on merchant silicon; we use Linux for an operating system, we use a photonic inter-connect fabric and distributed controller architecture.  Our switches speak ethernet and IP, just like all ethernet switches, but it is the controller and photonic fabric that make our switches different.  Together we believe that our technology results in a system that has transformative scale, performance, and efficiency advantages compared to legacy network architectures.


Recently we provided a proposal to a potential client regarding their network.  As with Greg Ferro’s project, we provided an all in proposal that included cables, software, transceivers, switches, controllers, accessories, support costs and the fabric interconnect ports.  There were two design options based around the size of the interconnect fabric, which most people refer to as the spine or core.   I have condensed some of the details for reading ease, but I think this is a transparent description of the proposal regarding (i) the costs of building a Plexxi network and (ii) how it is different with the choice of two different fabric options, much like the blog post from Greg.  Here is the summary table of the network design options:

# 10G Client Ports $ Per 10G Client Port Fabric Size


Total 1 Year Cost


$1,223 2.88 Tbps




$1,223 11.52 Tbps




$1,223 23.04 Tbps




$517 1.44 Tbps




$517 5.76 Tbps




$517 11.52 Tbps



In a Plexxi network, we use a controller architecture.  Our controller computes efficient photonic forwarding topologies.  This type of architecture, often referred to as SDN, provides a number of benefits, but it begins with a fluid pool of capacity within the fabric.  The capacity in our photonic fabric can be allocated, reassigned and reconfigured.  Without providing a long technical description of how our controller operates, an important concept to understand is we use 100% of the fabric and we do not implement spanning tree or block links.  We compute forwarding topologies using multi-flow commodity, graph theory math; that is one of the jobs of Plexxi Control.  A Plexxi fabric can be used inside your data center and the use of a photonic fabric means the fabric can be extended to campus and metro area designs.  Here are a few points regarding our controller and fabric architecture:

  • The photonic fabric is managed as pool of capacity
  • Applications and workloads that are important can be assigned fabric capacity
  • Capacities can change, evolve and forwarding topologies can be diurnal
  • Controller based fabrics are deterministic as opposed to distributed state fabrics that take time to compute state, which may or may not result in an optimal design.
  • Controller based fabrics can centrally compute optimal paths and provide fast convergence by pre-computing failure recovery states

The use of a controller with a photonic fabric provides a number of scaling benefits.  The most obvious benefit is the linear cost curves in terms of price per client port.  The following chart shows the 10G client port cost in scaling from 400 to 3400 ports.


The linearity of the Plexxi architecture in terms of client port cost can also be seen in terms of power and cooling.  In a Plexxi architecture, the performance of the network benefits when the controller tries to keep packets in the photonic portion of the network as much as possible, thus limiting silicon switch hops and incurring latency.  Uniformed latency and uniformed power consumption per client power are benefits of the Plexxi architecture:


A question we often get is whether a Plexxi network requires a greenfield or can it be deployed in a brownfield.  The answer is there are no greenfields.  Plexxi networks have been deployed in a number of variations.  We have had clients deploy Plexxi in the spine, while leaving the legacy ToR and server connections in place.  We have had clients deploy Plexxi as a replacement strategy for the their leaf/spine network, thus collapsing a two tier or three tier network to a single tier.  We have had clients deploy Plexxi between data centers providing a single hop, load-balanced fabric between data centers.  Traditionally, most Plexxi customers connect our switches to legacy routers and switches using 10G or 40G ports.  We have a handful of customers who have extended the Plexxi fabric via DWDM connections to legacy optical transport platforms.

Another question I have been asked is whether or not our photonic fabric is proprietary.  The way to think about our photonic fabric is to compare it to the fabric modules found in traditional core and spine switches.  What we have done at Plexxi is to take the backplane capacity in spine/core switches found in multi-tier networks and distribute that capacity into each switch.  When you add port capacity, you add fabric capacity.  Five years ago this type of network design was not possible, only with the advent of the modern controller architecture coupled with low cost, multi-path photonic interconnects has it become possible.   The design objective of a Plexxi network is to manage the network as a resource pool that can be correlated with the needs of compute and storage.   We believe that networking is entering the era of plenty and that networks built with rich amounts of path diversity are building blocks of the new networks.  We believe this because that has been direction of compute and storage.  Compute and storage have entered the era of plenty and it is time for the network to enter the era of plenty as well.  The era of plenty for networking will be built using a controller architecture because a controller architecture combined with photonics, merchant silicon and Linux, is the best means to deliver the following benefits:

  • Simplicity: Single tier photonic network
  • High Utilization: Load balanced L2 fabric
  • Controller Architecture: Unified view of network fabric
  • Uniformed Low Latency: Direct connectivity
  • Faster Failure Handling: Pre-computed forward topologies that converge rapidly to target optimum
  • Elastic Network Capacity: Large-scale computation and path optimization through Controller enables fluidity of network capacity
  • Reduced Cabling: Simplified network element deployment and insertion


[Today's fun fact: The average American/Canadian drinks about 600 sodas a year. Of course, the American version is 72 ounces compared to Canada's 12 ounces.]

The post Building a Plexxi Network appeared first on Plexxi.

Read the original blog entry...

More Stories By Bill Koss

Bill Koss is VP of Sales at Plexxi. Sales is about having a keen understanding of customer needs and matching them to appropriate solutions so that everyone benefits. The best salespeople are fluent in both business and technology, translating product into value. Vice President of Sales he has acquired these skills over a 20-year career in networking and information technology.

Plexxi is Bill’s fifth venture-backed startup, having successfully exited CrossComm, SDL, Sonoma Systems and Internet Photonics. Notably, Ciena acquired Internet Photonics in 2004 to include the CN4200 platform, which became a $1B+ product line. While with Ciena, Bill held a variety of roles including VP of North American Sales and VP of Global Partners. During Bill’s tenure at Ciena, annual revenue grew from $290M to $700M.

@ThingsExpo Stories
Mobile messaging has been a popular communication channel for more than 20 years. Finnish engineer Matti Makkonen invented the idea for SMS (Short Message Service) in 1984, making his vision a reality on December 3, 1992 by sending the first message ("Happy Christmas") from a PC to a cell phone. Since then, the technology has evolved immensely, from both a technology standpoint, and in our everyday uses for it. Originally used for person-to-person (P2P) communication, i.e., Sally sends a text message to Betty – mobile messaging now offers tremendous value to businesses for customer and empl...
SYS-CON Events announced today that Sandy Carter, IBM General Manager Cloud Ecosystem and Developers, and a Social Business Evangelist, will keynote at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
The IoT market is on track to hit $7.1 trillion in 2020. The reality is that only a handful of companies are ready for this massive demand. There are a lot of barriers, paint points, traps, and hidden roadblocks. How can we deal with these issues and challenges? The paradigm has changed. Old-style ad-hoc trial-and-error ways will certainly lead you to the dead end. What is mandatory is an overarching and adaptive approach to effectively handle the rapid changes and exponential growth.
WebRTC converts the entire network into a ubiquitous communications cloud thereby connecting anytime, anywhere through any point. In his session at WebRTC Summit,, Mark Castleman, EIR at Bell Labs and Head of Future X Labs, will discuss how the transformational nature of communications is achieved through the democratizing force of WebRTC. WebRTC is doing for voice what HTML did for web content.
Today air travel is a minefield of delays, hassles and customer disappointment. Airlines struggle to revitalize the experience. GE and M2Mi will demonstrate practical examples of how IoT solutions are helping airlines bring back personalization, reduce trip time and improve reliability. In their session at @ThingsExpo, Shyam Varan Nath, Principal Architect with GE, and Dr. Sarah Cooper, M2Mi's VP Business Development and Engineering, will explore the IoT cloud-based platform technologies driving this change including privacy controls, data transparency and integration of real time context w...
The IoT is upon us, but today’s databases, built on 30-year-old math, require multiple platforms to create a single solution. Data demands of the IoT require Big Data systems that can handle ingest, transactions and analytics concurrently adapting to varied situations as they occur, with speed at scale. In his session at @ThingsExpo, Chad Jones, chief strategy officer at Deep Information Sciences, will look differently at IoT data so enterprises can fully leverage their IoT potential. He’ll share tips on how to speed up business initiatives, harness Big Data and remain one step ahead by apply...
"Matrix is an ambitious open standard and implementation that's set up to break down the fragmentation problems that exist in IP messaging and VoIP communication," explained John Woolf, Technical Evangelist at Matrix, in this interview at @ThingsExpo, held Nov 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
WebRTC has had a real tough three or four years, and so have those working with it. Only a few short years ago, the development world were excited about WebRTC and proclaiming how awesome it was. You might have played with the technology a couple of years ago, only to find the extra infrastructure requirements were painful to implement and poorly documented. This probably left a bitter taste in your mouth, especially when things went wrong.
Nowadays, a large number of sensors and devices are connected to the network. Leading-edge IoT technologies integrate various types of sensor data to create a new value for several business decision scenarios. The transparent cloud is a model of a new IoT emergence service platform. Many service providers store and access various types of sensor data in order to create and find out new business values by integrating such data.
The broad selection of hardware, the rapid evolution of operating systems and the time-to-market for mobile apps has been so rapid that new challenges for developers and engineers arise every day. Security, testing, hosting, and other metrics have to be considered through the process. In his session at Big Data Expo, Walter Maguire, Chief Field Technologist, HP Big Data Group, at Hewlett-Packard, will discuss the challenges faced by developers and a composite Big Data applications builder, focusing on how to help solve the problems that developers are continuously battling.
There are so many tools and techniques for data analytics that even for a data scientist the choices, possible systems, and even the types of data can be daunting. In his session at @ThingsExpo, Chris Harrold, Global CTO for Big Data Solutions for EMC Corporation, will show how to perform a simple, but meaningful analysis of social sentiment data using freely available tools that take only minutes to download and install. Participants will get the download information, scripts, and complete end-to-end walkthrough of the analysis from start to finish. Participants will also be given the pract...
WebRTC: together these advances have created a perfect storm of technologies that are disrupting and transforming classic communications models and ecosystems. In his session at WebRTC Summit, Cary Bran, VP of Innovation and New Ventures at Plantronics and PLT Labs, will provide an overview of this technological shift, including associated business and consumer communications impacts, and opportunities it may enable, complement or entirely transform.
SYS-CON Events announced today that Dyn, the worldwide leader in Internet Performance, will exhibit at SYS-CON's 17th International Cloud Expo®, which will take place on November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. Dyn is a cloud-based Internet Performance company. Dyn helps companies monitor, control, and optimize online infrastructure for an exceptional end-user experience. Through a world-class network and unrivaled, objective intelligence into Internet conditions, Dyn ensures traffic gets delivered faster, safer, and more reliably than ever.
WebRTC services have already permeated corporate communications in the form of videoconferencing solutions. However, WebRTC has the potential of going beyond and catalyzing a new class of services providing more than calls with capabilities such as mass-scale real-time media broadcasting, enriched and augmented video, person-to-machine and machine-to-machine communications. In his session at @ThingsExpo, Luis Lopez, CEO of Kurento, will introduce the technologies required for implementing these ideas and some early experiments performed in the Kurento open source software community in areas ...
Too often with compelling new technologies market participants become overly enamored with that attractiveness of the technology and neglect underlying business drivers. This tendency, what some call the “newest shiny object syndrome,” is understandable given that virtually all of us are heavily engaged in technology. But it is also mistaken. Without concrete business cases driving its deployment, IoT, like many other technologies before it, will fade into obscurity.
Who are you? How do you introduce yourself? Do you use a name, or do you greet a friend by the last four digits of his social security number? Assuming you don’t, why are we content to associate our identity with 10 random digits assigned by our phone company? Identity is an issue that affects everyone, but as individuals we don’t spend a lot of time thinking about it. In his session at @ThingsExpo, Ben Klang, Founder & President of Mojo Lingo, will discuss the impact of technology on identity. Should we federate, or not? How should identity be secured? Who owns the identity? How is identity ...
The buzz continues for cloud, data analytics and the Internet of Things (IoT) and their collective impact across all industries. But a new conversation is emerging - how do companies use industry disruption and technology enablers to lead in markets undergoing change, uncertainty and ambiguity? Organizations of all sizes need to evolve and transform, often under massive pressure, as industry lines blur and merge and traditional business models are assaulted and turned upside down. In this new data-driven world, marketplaces reign supreme while interoperability, APIs and applications deliver un...
Electric power utilities face relentless pressure on their financial performance, and reducing distribution grid losses is one of the last untapped opportunities to meet their business goals. Combining IoT-enabled sensors and cloud-based data analytics, utilities now are able to find, quantify and reduce losses faster – and with a smaller IT footprint. Solutions exist using Internet-enabled sensors deployed temporarily at strategic locations within the distribution grid to measure actual line loads.
The Internet of Everything is re-shaping technology trends–moving away from “request/response” architecture to an “always-on” Streaming Web where data is in constant motion and secure, reliable communication is an absolute necessity. As more and more THINGS go online, the challenges that developers will need to address will only increase exponentially. In his session at @ThingsExpo, Todd Greene, Founder & CEO of PubNub, will explore the current state of IoT connectivity and review key trends and technology requirements that will drive the Internet of Things from hype to reality.
The Internet of Things (IoT) is growing rapidly by extending current technologies, products and networks. By 2020, Cisco estimates there will be 50 billion connected devices. Gartner has forecast revenues of over $300 billion, just to IoT suppliers. Now is the time to figure out how you’ll make money – not just create innovative products. With hundreds of new products and companies jumping into the IoT fray every month, there’s no shortage of innovation. Despite this, McKinsey/VisionMobile data shows "less than 10 percent of IoT developers are making enough to support a reasonably sized team....