In our work with SDN controllers we concentrate on a few including On.Lab’s ONOS, The OpenDaylight Project and RYU. In this post we will discuss setting up a distributed ONOS cluster.
First things first, the ONOS wiki pages give a good overview of the requirements:
- Ubuntu Server 14.04 LTS 64-bit
- 2GB or more RAM
- 2 or more processors
- Oracle Java 8 JRE and Maven installed
For our setup, we used Ubuntu 14.0.4.2 (the latest at the time) installed as an OpenSSH server and we installed Java and Maven from the instructions from the ONOS wiki:
|
1 2 3 4 |
sudo apt-get install software-properties-common -y sudo add-apt-repository ppa:webupd8team/java -y sudo apt-get update sudo apt-get install oracle-java8-installer oracle-java8-set-default maven -y |
We then installed the required packages on a OSX VM
- Java 8 JDK
- Apache Maven (3.0 and later)
- git
- Apache Karaf (3.0.3 and later)
Once the dependencies were installed we were able to grab the code from ONOS github page and build it using the “mvn clean install” command using the directions shown on the ONOS wiki
Once built, the process to install the software and get it working properly is not perfectly documented, we ended up suffering issues with websocket failures which were related to the ONOS systems not being setup properly, all of the information is on the ONOS wiki but on different pages. Here is what we found worked:
|
1 2 3 4 5 6 7 8 |
export ONOS_NIC=192.168.21.* onos-package onos-patch-vm 192.168.21.129 onos-patch-vm 192.168.21.130 onos-config 192.168.21.129 onos-config 192.168.21.129 onos-install -f 192.168.21.129 onos-install -f 192.168.21.130 |
Once the install finished, we were able to access the ONOS installations at http://192.168.21.129:8181/onos/ui/index.html and http://192.168.21.130:8181/onos/ui/index.html
Once up, we needed to modify a few files on each server and restart the servers:
/opt/onos/config/cluster.json
|
1 2 3 4 5 |
{ "ipPrefix": "192.168.21.*", "nodes":[ { "id": "192.168.21.129", "ip": "192.168.21.129", "tcpPort": 9876 }, { "id": "192.168.21.130", "ip": "192.168.21.130", "tcpPort": 9876 } ]} |
and /opt/onos/config/tablets.json
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
{ "nodes": [ { "ip": "192.168.21.129", "id": "192.168.21.129", "tcpPort": 7238 }, { "ip": "192.168.21.130", "id": "192.168.21.130", "tcpPort": 7238 } ], "partitions": { "p1": [ { "ip": "192.168.21.129", "id": "192.168.21.129", "tcpPort": 7238 }, { "ip": "192.168.21.130", "id": "192.168.21.130", "tcpPort": 7238 } ], "p2": [ { "ip": "192.168.21.130", "id": "192.168.21.130", "tcpPort": 7238 }, { "ip": "192.168.21.129", "id": "192.168.21.129", "tcpPort": 7238 } ] } } |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
sdn@onos-1:$ /opt/onos/bin/onos Logging in as karaf Welcome to Open Network Operating System (ONOS)! ____ _ ______ ____ / __ \/ |/ / __ \/ __/ / /_/ / / /_/ /\ \ \____/_/|_/\____/___/ Hit '<tab>' for a list of available commands and '[cmd] --help' for help on a specific command. Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to shutdown ONOS. onos> system:shutdown Confirm: halt instance root (yes/no): |
Since ONOS will restart automatically, you simply need to do a system:shutdown.
By default ONOS uses 192.168.56.* as the IP address blocks, so we had to modify them to match our setup.
On the switches we set the controllers
|
1 |
ovs-vsctl set-controller switch1 tcp:192.168.21.129:6633 tcp:192.168.21.130:6633 |
And we were then able to ping between our physical hosts connected to ports 1 and 2 on each switch.
