Building a virtual Cisco Nexus lab on VMware vSphere

Today a blog post on something I hope you will find useful: How to build a virtual Cisco Nexus lab on VMware vSphere. The goal is to create a lab containing two or more virtual Cisco Nexus switches, which can be used to learn/test NX-OS. You can use the Titanium emulator for this setup, which has been leaked to the internet. Just use Google to find it.

The lab which is discussed in this article runs on vSphere/ESXi and is built according to the following design:

The configuration consists of:

• Two Nexus 7000 switches (bases on a simulator);
• For management of the switches you have a serial port and an SSH connection available. The serial port is connected to the management LAN using the serial-port-to-network feature of ESXi. For the SSH connection Nexus port “mgmt 0” is configured and connected to the management LAN.
• On the first switch (N7K-1) port e2/1 is connected to the 192.168.192.0/24 network, and port e2/2 is connected to the 172.30.96.0/24 network;
• On the second switch (N2K-2) port e2/1 is connected to the 172.30.96.0/24 network, and port e2/2 is connected to a third network with subnet 172.30.97.0/24.

Building the Nexus lab

The Titanium emulator comes as a set of files which can be uploaded to a VMFS/NFS datastore on an ESXi host. After registering the virtual machine it becomes available in your vSphere inventory. The default NICs in the emulator won’t work with ESXi, so remove them and add E1000 adapters. Also remove the serial port (if available) and add a new serial port to your virtual Nexus switch. This article explains how you can access the Nexus serial port through a telnet connection.

The configuration of the Titanium virtual machine should look like this:

For the N7K-1 switch (displayed in the image) add (for example) 7 adapters to the switch and a serial port:

• The serial port can be connected through Telnet on port 6001;
• Network Adapter 1 is the mgmt 0  interface on the Nexus;
• Network Adapter 2-7 are devices e2/1-6 on the Nexus;
• The nxos-mgmt = the management network;
• LAN network is not connected or used at this point.

Network Adapter 3 (e2/2) is connected to the 172.30.96.0 segment, which is actually a Virtual Machine portgroup name on the ESXi host. Nexus N7K-2 is also connected to this segment:

vSwitch1 is deployed to interconnect N7K-1 and N7K2.

After booting the switches you first have to access the switch through the serial port. After this you can set an IP address on the ‘mgmt 0’ interface to access the switch through SSH.

n7k-1# conf t
Enter configuration commands, one per line.  End with CNTL/Z.
n7k-1(config)# int mgmt 0
n7k-1(config-if)# ip add 192.168.192.164 255.255.255.0
n7k-1(config-if)# no shutdown
n7k-1(config-if)# sh int mgmt 0
mgmt0 is up
  Hardware: Ethernet, address: 000c.2965.d46d (bia 000c.2965.d46d)
  Internet Address is 192.168.192.164/24
  MTU 1500 bytes, BW 1000000 Kbit, DLY 10 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA
  full-duplex, 1000 Mb/s
  Auto-Negotiation is turned on
  EtherType is 0x0000
  1 minute input rate 5456 bits/sec, 3 packets/sec
  1 minute output rate 24 bits/sec, 0 packets/sec
  Rx
    35644 input packets 336 unicast packets 23179 multicast packets
    12129 broadcast packets 6339779 bytes
  Tx
    360 output packets 219 unicast packets 139 multicast packets
    2 broadcast packets 50551 bytes

n7k-1(config-if)#

Notice that the MAC address (000c.2965.d46d) of mgmt0 is (of course) the same as the MAC address on ESXi:

After adding IP addresses to the e2/1 and e2/2 interfaces and configuring a routing protocol (OSPF for example), you can even test routes between the Nexus routers.

Routing table on switch N7K-1:

n7k-1(config-if-range)# sh ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]

172.30.96.0/24, ubest/mbest: 1/0, attached
    *via 172.30.96.254, Eth2/2, [0/0], 00:13:54, direct
172.30.96.254/32, ubest/mbest: 1/0, attached
    *via 172.30.96.254, Eth2/2, [0/0], 00:13:54, local
172.30.97.0/24, ubest/mbest: 1/0
    *via 172.30.96.253, Eth2/2, [110/80], 00:05:25, ospf-router1, intra
192.168.192.0/24, ubest/mbest: 1/0, attached
    *via 192.168.192.154, Eth2/1, [0/0], 00:13:54, direct
192.168.192.154/32, ubest/mbest: 1/0, attached
    *via 192.168.192.154, Eth2/1, [0/0], 00:13:54, local

Good luck building your own Cisco lab!