Comparing SDN Controllers: Open Daylight and ONOS


Over the last few months, we have been doing testing of the SDN controllers Open Daylight Helium SR3 (mostly via the Brocade Vyatta Controller v1.2) and ONOS Cardinal v1.2.  In this initial article we will start to compare the controllers, focusing on scale, specifically the number of switches that can be handled, by running both OpenFlow 1.0 and 1.3 switches emulated via IXIA and using physical via Pica8 switches.

Note: In the latest version of ONOS (Cardinal) v1.2, there is an issue ONOS handling emulated OpenFlow v1.3 from IXIA, so all scale testing in ONOS was done using OpenFlow 1.0.  Also, in ONOS the term “node” references a copy of ONOS (we run two nodes in our tests) while in Open Daylight, the term “node” refers to a OpenFlow switch.

User Interface

One of the main differences between the ONOS and Open Daylight/BVC is in the controls and information available directly from the Graphical User Interface (GUI).


The ONOS GUI has multiple panes including Summary, Node(s) and Controls.

ONOS 1.2

ONOS GUI With 300 Switches

The ONOS GUI displays end hosts in a well defined fashion, you can see them spanning out from the switches.


ONOS 96 Hosts Visible

Open Daylight

The default Open Daylight GUI is defined by the features installed and can include features such as a pane to display Nodes, a Yang UI and Yang Visualizer.

Open Daylight

Open Daylight GUI

When attempting to display end hosts, the Open Daylight GUI is not as clean as ONOS as the hosts are interlaced with switches.

ODL Nodes

Open Daylight with 400 Nodes

In the above screen capture of Open Daylight you can see both Nodes (Switches) and Hosts.

Brocade Vyatta Controller

The GUI for the Brocade Vyatta Controller (BVC) is cleaner than the default Open Daylight GUI and in this screen shot has extra modules for their Vyatta vRouter 5600 EMS and their “Path Explorer” application.

BVC 1.2 With EMS 5600

Brocade Vyatta Controller GUI

The current way of displaying Hosts and Switches in Open Daylight/BVC is not easy to work with nor does it scale well.


In scale testing, we started with 100 Switches and scaled up to 400 Switches, each switch holding 12 hosts.  While Open Daylight (via BVC) was able to scale to 400 switches, ONOS stopped functioning before 400.

Here is BVC with 400 Switches, 800 links and multiple (96 out of 4800) hosts sending traffic to each other.

BVC with 400 Switches

BVC with 400 Switches and Multiple Hosts

BVC 400 Switches Building

BVC 400 Switches Installing Hosts

Here is ONOS when it has reached capacity and is no longer able to handle the number of switches/links/hosts that are being sent to it.

ONOS With 400 Switches

The above screen shot shows two ONOS nodes with 400 switches, 800 links and 0 hosts (we are attempting to send traffic between 48 hosts).  While the devices (switches) are in the database, the hosts are not in the database and the GUI has become unstable and no longer shows any information.


Both ONOS and Open Daylight are solid products when acting as SDN controllers with multiple southbound and northbound interfaces.  The testing done here only focuses on OpenFlow and specifically on scale.  The Brocade version of Open Daylight is well packaged and has some nice extras such as the EMS application which ties in the Brocade Vyatta vRouter 5600.  ONOS continues to focus on providing tools and information in their GUI and 300 switches is a perfectly reasonable amount and certainly more than anyone should put on one or two controllers.

Using The Brocade Vyatta Controller – Part 1

As part of NetDEF, I’ve been working with different SDN controllers, including; the Brocade Vyatta Controller v1.1.1 (BVC), the OpenDaylight Controller (Helium Release) and the ONOS v1.0 Controller.  Of the three, the Brocade Controller has been the most user-friendly and straightforward.

To install the Brocade Vyatta Controller, simply sign up, download, read the quick guide and follow the instructions.  As Lisa Caywood points out in her blog post, there is even a nice video “Install Brocade Vyatta Controller” with links to the files needed to install BVC.

The Setup


For my testing, I used a Ubuntu 14.0.4 Server VM with 6G RAM and 32G Disk to run BVC.  For OpenFlow switches I used a pair of Pica8 3290’s running PicaOS v2.5 in crossflow mode.  For end hosts I used four VMs Linux VMs and eight IXIA ports.  The BVC was connected to the switches via the management network.


My first test was to ping between VM1-VM4. Which showed the correct information in the BVC topology screen:

BVC 4 Hosts

Next, I installed the BVC Path Explorer (the installation was simple and went as shown in the documentation).  I added a few paths, including one that crossed switches and everything worked as expected.

BVC Path Explorer

Once I had everything working as expected with four hosts, I added a few more (about 84).

BVC Zoom Out 80 Hosts

The BVC had no issue adding all of the hosts and allowing them to be interacted with.

BVC Close up 80 Nodes

I also did some testing using postman (a chrome REST API plugin).  Thanks to Keith Burns, who pointed this tool out to me.

Screenshot 2015-01-31 17.21.29

Above is the output of the GET topology command, neatly formatted in JSON.

Screenshot 2015-01-31 17.24.50


Above is the output of the OpenDaylight inventory API call, showing some of my hosts.

While I am just starting my testing and plan to do more extensive work utilizing the Vyatta vRouter connector, IXIA OpenFlow tester and other tools/add-ons, I am impressed with the release of BVC 1.1.1.  The software and tools appear to be reasonably stable while the documentation is clear and professional.

The Vyatta Cloud Router Story

Vyatta and their approach to Cloud Routers

A few weeks ago I had the pleasure of speaking with Scott Sneddon, Cloud Solutions Architect at Vyatta Inc.  I’ve known Scott since the late 1990’s when he and I both worked for Exodus Communications.

Vyatta is one of the few full featured software based routing vendors in the market today.  Their product is a mix of OpenSource and proprietary software combined together creating a router that can not only live in the cloud but will in the future be able to utilize some of the hardware such as Intel’s Sandy Bridge (and later generation Ivy Bridge) processors as Network Processors.

Network Processors are key to hardware forwarding routers such as the Juniper T series and the Cisco Carrier Routing System allowing them to perform forwarding and features at line rate, something that routers using software based forwarding struggle with.  To get a better picture of software vs hardware forwarding you can read Router Analysis’ Enterprise Edge Router Upgrade Guide where I discuss the Cisco 7200 which uses a software forwarding engine and compare it with higher performance routers with hardware forwarding capabilities.

Vyatta offers a full featured router solution by including VPN, Firewall and other features normally found in hardware locked solutions in their software product.  I feel that Vyatta has a jump on other vendors in the True Virtual Data Center space.  One of the most important parts of the Virtual Data Center is the router and it’s ability to perform equal to or greater than the hardware based router it is replacing.  Using software forwarding alone Vyatta claims to be able to handle up to 2Mpps, which depending on packet size can easily be multiple gigabits of traffic.  In testing Vyatta is seeing up to 11Mpps using an Intel Sandy Bridge processor as a network processor.

A quick note about integrated firewalls: While software firewalls contained within the same hardware as the routers, switches and/or hosts are very useful, they are not a replacement for hardware firewalls.  In security (which I do not claim to be an expert at) the separation of networks using physical links is key.  There is some great information available in this thread on the Cisco support forums where they are discussing the ASA 1000V.

Vyatta keeps a tight relationship with the OpenSource community by hosting where you can find free versions of Vyatta’s Core Software along with community support, documentation and forums.

SDN Testing, the software defined side of Router Analysis plans to put the Vyatta product through rigorous testing in the coming weeks.

The Virtual Data Center Reality

Virtual Data Centers start to become reality.

Previously posted on Router Analysis

With the recent announcement of the CSR 1000v from Cisco, there are now two commercial Virtual Data Center stories (three if we look at the VMWare vCNS products and use one of the other vendors products for a router) Cisco and Vyatta.

What is a Virtual Data Center?  There will be a lot of different answers but in my view it consists of the following:

  • A pair of redundant Routers with multiple provider uplinks
  • A pair of redundant Firewalls
  • A pair of Load Balancers
  • Front and Backend Servers

In my previous life designing and building Internet Data Centers we would have build this entire setup out of separate parts taking up an entire rack or two.  Now it could be done in a single blade server with multiple redundant power supplies or a pair of highly spec’d servers.

Now, I want to be clear here: I don’t think that the software based Firewalls are up to the task of the hardware based ones.  I think most security companies/consultants would agree that there is a danger when you host both your servers and your firewall on the same shared hardware.  You could design the setup in a way that the ASA is only hosted on it’s own blade(s) but there is still the inherent risk of a misconfiguration or privilege escalation hack allowing someone to bypass the firewall.

Sadly the way around the security issue is to put a physical firewall in the line.  This can be easily done, so it’s mainly just a CapEx issue.

For routers at this time, we only have Cisco and Vyatta commercially.  They both are offering strong products but Cisco’s CSR 1000v is more feature rich supporting many protocols and features that come from using the previously designed and coded Cisco code base.

In the coming weeks and months I am going to be writing about the products available in the space and what limitations still need to be overcome.  I will be working with Cisco, Vyatta, VMWare and others to try and compile as much information as possible.

Summary: The Virtual Datacenter is here.  It’s not perfect, but all of the parts exist from multiple vendors.  The world of Virtualization just got a lot more interesting.

What are your thoughts?

What is SDN and What are we Testing?

SDN Testing

SDN stands for “Software Defined Network”, a simple name with thousands of different meanings.  As defined by WikiPedia “SDN separates the control plane from the data plane in network switches and routers. Under SDN, the control plane is implemented in software in servers separate from the network equipment and the data plane is implemented in commodity network equipment.”

The most important aspect of SDN is the separation of the control and data planes.  This decoupling allows end users (Service Providers, Enterprises) to use commodity hardware to build and expand their networks.

Some of the major players in the SDN space are Vyatta, Cisco, Big Switch and Nicira.  Nicira was recently purchased by VMWare and is being merged into VMWares core product offerings.

Vyatta offers a Quagga based software router with firewall and VPN support.  I recently talked with Vyatta and found their vision and commitment to the Open Source community great.  Vyatta is currently the top player when it comes to software defined routers.

Cisco offers many of the parts needed to create SDNs but some parts have not been released yet.  Cisco has released the Nexus 1000v software switch, the ASA 1000v software firewall and has announced the CSR 1000v IOS XE based software router.  Once Cisco gets the full solution out, they have a chance to leapfrog over the competition due to their history and ability to re-use their current software features.

Big Switch offers an SDN Controller.  Currently they are offering Floodlight, a Open Source version of their product with promises of a commercial version coming soon.

The last company I will cover is Nicira.  Nicira provides what they call NVP, or the Nicira Virtualization Platform.  They combine their software with the Open Source OpenVSwitch to provide a fully software controlled and forwarded network.