Jason Nash’s Blog

This isn’t a complicated topic, but it’s something I’ve been asked a few times and thought it would be worthy of a quick post to help others out there Googling for an answer.  I won’t get in to the pros and cons of jumbo frames in a vSphere environment as others already have here.  This is merely to show you how to enable large MTUs on the 1000v, that can sometimes do things different than other Cisco switches.

It’s very simple.  When configuring your uplink profiles (the “type ethernet” ones) just use the mtu command.  For example:

port-profile type ethernet Primary-Uplink  
vmware port-group  
switchport mode trunk  
switchport trunk native vlan 500  
switchport trunk allowed vlan 1,500-501  
mtu 9000  
channel-group auto mode active  
no shutdown  
system vlan 1,501  
state enabled

You don’t need to set the MTU anywhere else in the 1000v…not on the vethernet profiles or interfaces…not on the port-channel interfaces.  Just in the uplink.  Now, if you go look around you may see references to the older system mtu command.  This command is no longer required in sv1(4).  So you can just ignore that.

If you plan to use jumbo frames with your NAS storage you’ll need to create your vmkernel interfaces via the CLI so you can manually set the MTU.  If you already have your vmkernel interfaces created you’ll need to recreate them as we still can’t change the MTU on an existing vmkernel interface.  If you have your existing ones on the 1000v you can recreate them following these steps:

~ # esxcfg-vswitch -l

DVS Name         Num Ports   Used Ports  Configured Ports  MTU     Uplinks
Nexus1Kv         256         45          256               1500    vmnic0,vmnic1

DVPort ID           In Use      Client
224                 1           vmnic1
225                 1           vmnic0

…other IDs omitted…

65                  1           vmk1

66                  1           Nexus 1000v VSM-2.eth0

32                  1           Nexus 1000v VSM-2.eth1

67                  1           Nexus 1000v VSM-2.eth2

37                  1           vmk0

You want to delete your existing vmkernel interface and recreate it.  Make sure you know the IP that it currently uses and then match it to the list you get from the esxcfg-vswitch -l command.  You want to note the DVPort ID as well.  In this example I’ll be using vmk1 on DVPort ID 65.  You will want to recreate it using the same port ID so it matches the existing vethernet interface in the 1Kv.

~ # esxcfg-vmknic -d -s Nexus1Kv -v 65

Then recreate it:

~ # esxcfg-vmknic -a -i 1.1.1.2 -n 255.255.255.0 -m 9000 -s Nexus1Kv -v 65

Note that I set the MTU to 9000 here.  Now, assuming that you have enabled jumbo frames on the full path from the vSphere host to the destination you can test it:

~ # vmkping -d -s 8000 1.1.1.1

PING 1.1.1.1 (1.1.1.1): 8000 data bytes

8008 bytes from 1.1.1.1: icmp_seq=0 ttl=64 time=0.778 ms

8008 bytes from 1.1.1.1: icmp_seq=1 ttl=64 time=1.540 ms

8008 bytes from 1.1.1.1: icmp_seq=2 ttl=64 time=0.697 ms

— 1.1.1.1 ping statistics —

3 packets transmitted, 3 packets received, 0% packet loss

round-trip min/avg/max = 0.697/1.005/1.540 ms

Success!  Make sure to use the -d option so it doesn’t try to fragment the packet and give you a false success.  You only need to do this for vmkernel interfaces.  For Virtual Machine networking you just need to change the MTU on the guest operating system and you’re good to go.