Towards the end of 2005, there was a large and conscious effort by the research community to understand the implications of Internet design. The Internet was designed almost 30 years ago primarily for best-effort data-services. However, contemporary revenue bearing services indicate that non-data-services such as video, leased circuits, virtualization, voice, and enterprise connectivity, etc. dominate earnings. The NSF in the US, the NICT in Japan, and the EC in Europe began programs aimed at wiping off the non-service oriented architecture and replacing it with a clean-slate Internet design. The fundamental business problem with this approach was that there already existed several hundred billion dollars of telecommunication and networking equipment in the Internet. From a technical perspective, the Internet was wired to support TCP/IP, Ethernet, SONET/SDH and WDM as a suite of protocols. Any new design had to be cognizant of the existing infrastructure, and be backward compatible with the history of protocols that existed.
In this regard, we proposed the framework of Omnipresent Ethernet (OE for short), in 2009. The conceptual idea behind OE was to collapse multiple networking layers into a single layer, be backward compatible with existing technologies, make a strong impact on CAPEX and OPEX, and be able to support carrier-class services that generate revenue (essentially be deterministic). In OE, we took advantage of existing patterns in the Internet connection graph and manipulated such interconnection to meet our goal of end-to-end carrier-class services on a single layer. We observed that irrespective of the physical topology, a network within the Internet could be abstracted to a tree shaped structure. Using this interconnection hierarchy, along with the well-known concept of binary routing and source routing, we were able to convert any network to a binary tree or graph by adding “dummy” nodes. This conversion to a graph that supported all nodes to be binary nodes was done by a centralized network management system that then accorded routes to services based on binary routing. Further, we showed that we could assign addresses to nodes in a way such that if the destination address (in a binary format) was available, then the source node could compute the journey to the destination. This kind of addressing and subsequent routing is called source routing. The advantages of source and binary routing are immense – preventing large lookup tables leading to low-latency – this meant low energy consumption thereby reducing the overall total cost of ownership. The concept was simple: by making the packet wait at a node for a shorter duration, we would be more deterministic in terms of end-to-end delay and reduce the power consumption at the node.
While source routing and binary routing have been around for a while, the next question was how to adapt these to the present Internet hierarchy. Here, we proposed the use of Carrier Ethernet advances. Ethernet, which has been around for over 4 decades and is ubiquitously used for communication by PCs, handhelds, and servers, is widely accepted as a popular LAN technology. However, since 2000, there has been a move to make Ethernet work as a service in Wide Area Networks or WAN (with the adoption of the Gigabit Ethernet, GE, and the 10 Gigabit Ethernet 10GE standards by the IEEE). In WAN, Ethernet is made service oriented as opposed to its earlier role of being Ethernet as an infrastructure. In this migration of Ethernet to Carrier Ethernet, the frame-format has changed to accommodate a series of user defined Virtual LAN or VLAN Tags sometimes also denoted as labels in the Multi-Protocol Label Switching (MPLS) nomenclature. Traditional features in Ethernet like spanning tree protocol and MAC learning are switched OFF. Taking advantage of this new frame-format, we insert the binary addresses and routes that are calculated using binary routing and source routing over a binary tree or binary graph by defining new tags/labels. This means that when a packet enters a node, there is no need for an exhaustive look up, and all that the node examines are 2log2N bits relevant to the NxN node interfaces. This speeds up the routing process through the node leading to energy efficiency. With this innovation, we are able to perform switching, routing, and transmission in a fiber, all at the Ethernet layer. In this manner, the concept of Omnipresent Ethernet was able to produce all the different functions of the Internet stack at layer 2. We also published an article in OFC 2010 on Optical Bit-Switching (OBiS) that facilitated implementing OE like functions at layer 1 – a more futuristic paradigm.
To prove that OE worked, we published a post-deadline paper at OFC 2009, showcasing an OE experiment in the lab. This paper was very well accepted by academia and industry. The technology was first displayed in parts to the international audience through journal and conference articles that were used to vet the theoretical framework. Subsequently, prototypes were built using off-the-shelf equipment to demonstrate the working of the technology. These were primarily coded by engineers at IIT Bombay. On successful demonstration of the prototypes, an effort was made to build our own PCB that could lead to a series of commercial products. Three teams were formed that focused on the hardware, software, and the PCB. The hardware team was responsible for the RTL and ASICs, the software team for the network management system, and the PCB team for the PCBs, the testing, and mechanicals of the board. 5 versions of the hardware code exist – the final version, which has now become the backbone of the commercial router manufactured by ECIL, is close to 100,000 lines of RTL code. The network management software was given as a challenge contest between two groups in the lab – one focusing on a Java variant and another on a C# variant. The Java variant (64,000 lines of code) formed the commercial control system, while the C# variant would be the backbone of training programs on the router. PCBs between 14 and 22 layers were designed and included high-speed traces for up to 11.1 Gbps line-rates. Signal integrity at such speeds was a challenge, and we were lucky to get it right in the first go itself! In addition to the code, our boxes had between 800 and 1500 components. Inventory management and ensuring that when the technology became a product we had to create a sustainable supply chain were some non-technical challenges. Independent testing and quality assurance was conducted by a team from BARC.
We built three products – a small box for your home/office environment that has 8 Ethernet copper ports a 2 Gigabit Ethernet Fiber/Copper ports with scalability built in, and a metropolitan network aggregator with 10×1 Gigabit Ethernet ports and 2×10-Gigabit Ethernet ports. The aggregators’ cornerstone was the 1-microsecond port-to-port interconnection latency and for its size of being a 60Gbps duplex cross-connect, consuming only 28Watts of power. The third box was a core router which in its basic configuration supported 4x10Gbps core ports and 8x1Gbps edge ports. The ports in this box supported a technology called OTN or optical transport network that facilitated wavelength division multiplexing of channels and reach up to 1000 km without signal regeneration. This box also had a low latency of 3-5 microseconds and an energy consumption of about 65 Watts. The network management system was designed in compliance with the concepts of software defined networks or SDNs, whereby a user could softly configure network parameters to his taste and set up services that were configured to meet his requirement. Services could be set up based on IPv4, IPv6, port, TAG, VLAN, MAC or any other customizable identifiers. The routers being compliant with Carrier Ethernet v2.0, could set up services for ELINE, ELAN and ETREE.
User Acceptance by a Tier-1 Service Provider: In 2010, the incumbent service provider in Mumbai, MTNL (www.mtnlmumbai.in) approached IIT Bombay and my group and asked for the creation of a data-center. MTNL through a series of meetings and evaluations decided to use our CESR. This was perhaps the first example of a product developed in the academia being deployed in a tier-1 service provider network. The highlight of this data-center using our CESR was its clocking of a mere 1 microsecond port-to-port latency across 3 layers of the networking stack. The two data-centers in Worli and Belapur work on 56 of our CESRs since May 2011.
Technology Sell-out to ECIL: As a result of the success in deploying the CESR product and our continued interaction with BARC, we were approached by DAE’s Electronic Corporation of India Ltd (ECIL), who were interested in purchasing the technology. After intense negotiations, we sold the entire CESR technology as a nation-wide manufacturing licence to ECIL. ECIL would now manufacture and sell routers (CESRs) to the domestic market. (http://www.ecil.co.in/news/MOU_Press_Release.pdf and http://www.thehindubusinessline.com/industry-and-economy/info-tech/article2373134.ece)
The products were launched in April 2012 and are now being deployed in various networks across the country. For example, recently MTNL announced that they will deploy the CESRs in their network.
The highlight of this technology sale was that this was the largest ever technology sale (from a monetary perspective) between IIT Bombay and the industry.
ECR Router
Epilogue: From the onset we knew that our work is going to be different from the regular bread-and-butter research that happens in academia. There was tremendous support from our current Director, Prof. Khakhar who made things work for us. Key support came from Prof. Rangan Banerjee, Prof. Mujumdar and Prof. Kaliappan – successive Deans and Associate Dean R&D. Product development can be a very frustrating exercise; there are days when the team and I were down, that is when a pat on the back went a long way! Prof. Subhasis Chaudhuri and Prof. Juzer Vasi have been instrumental in keeping us motivated. Successive department heads helped us with space and other requests – some of which were perhaps beyond my fair share of departmental produce! The faith that the administration reposed in us was and will also in the future be crucial for this productization to succeed.
We did face criticism from various quarters as well for such product development – there were stray comments about the novelty of this work, about the correctness of some of our published work, and even about the risk IITB is taking in selling to ECIL. Whenever there was criticism, there was support, whenever there were problems, we found solutions, innovation happened everyday across the product cycle. Most of us in the lab worked 15-18 hour days, 6-7 days a week for more than two years. But with a small team, a key idea and a good plan, we knew we would succeed it was just a question of time.
Today, we feel good that a technology developed in India is passed on to an Indian PSU – who may have its own limitations in terms of marketing, but whose integrity is Everest like! We have deployed our boxes for NKN, BARC, and MTNL and several strategic networks are undergoing tests. The road ahead to selling this concept – which is a paradigmatic shift from what exists out there –, is a difficult one. We have a new roadmap of products, primarily moving back into our core business of being in the optical networking/very high-speed telecom space – a new reconfigurable optical add drop multiplexer with coherent optics is under development. This should expand our capacity to about 30Tbps per box – more than the entire installed capacity of bandwidth in the country! I strongly believe that product development is not an industry forte – it is something we have to undertake – products are the culmination of theoretical research, and products are always backed up by key papers and patents that are peer reviewed! Accepting new technology has always been a tough challenge. We have to persevere, and we have to convince others through results. I am sure that this product development cycle will be part of many more success stories from the IIT system in the years to come! ECIL has its hands full with selling this product, and we have our next target in our minds and heart!
Launch of MTNL’s Carrier Ethernet Network: PSA Dr. R. Chidambaram with CMD MTNL, AK Garg, CMD ECIL P. Sudhakar and Prof. Ashwin Gumaste
