PRO TIP: how to fix uneven IP load balancing over a pair of ECMP links on Cisco 7600

I've recently encountered a few instances of Cisco 7600s achieving a rather uneven load sharing over a pair of ECMP links. How to fix?

Here I'm looking in my 7600 where 1.3.3.7 is the loopback0 interface of a directly connected neighbor

7600#sh ip route 1.3.3.7
Routing entry for 1.3.3.7/32
  Known via "isis", distance 115, metric 10012, type level-2
  Redistributing via isis MEKK
  Last update from 10.10.10.10 on TenGigabitEthernet2/3, 4w5d ago
  Routing Descriptor Blocks:
    10.10.10.10, from 1.3.3.7, 4w5d ago, via TenGigabitEthernet2/3
      Route metric is 12, traffic share count is 1
  * 11.11.11.11, from 1.3.3.7, 4w5d ago, via TenGigabitEthernet2/1
      Route metric is 12, traffic share count is 1
7600#

One is easily led to believe that this would be evenly split over Te2/3 and Te2/1, but due implementation specifics it's a little more complicated than that...

7600#sh mls cef lookup 1.3.3.7 detail

Codes: M - mask entry, V - value entry, A - adjacency index, P - priority bit
       D - full don't switch, m - load balancing modnumber, B - BGP Bucket sel
       V0 - Vlan 0,C0 - don't comp bit 0,V1 - Vlan 1,C1 - don't comp bit 1
       RVTEN - RPF Vlan table enable, RVTSEL - RPF Vlan table select
Format: IPV4_DA - (8 | xtag vpn pi cr recirc tos prefix)
Format: IPV4_SA - (9 | xtag vpn pi cr recirc prefix)
M(82 ): E | 1 FFF 0 0 0 0 255.255.255.255
V(82 ): 8 | 1 0 0 0 0 0 1.3.3.7 (A:32914 ,P:1,D:0,m:14,B:0 )

Here we see that our Adjacency index is 32914 and our load balancing modnumber is 14. I'm interpreting this (I believe correctly ;) as that I have 32914 plus 14 adjacencies, ie 15 adjacencies all in all. 15 is not evenly dividable by two (as in my two 10GE links). Looking at the adjacencies I can get some more information:

7600#sh mls cef adjacency entry 32914 detail

Index: 32914 smac: 0025.8429.76c0, dmac: 001f.9daf.4100
               mtu: 4488, vlan: 1019, dindex: 0x0, l3rw_vld: 1
               format: MAC_TCP, flags: 0x2000208408
               delta_seq: 0, delta_ack: 0
               packets: 17, bytes: 1834

7600#sh mls cef adjacency entry 32915 detail | i vlan
               mtu: 4488, vlan: 1019, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32916 detail | i vlan
               mtu: 4488, vlan: 1019, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32917 detail | i vlan
               mtu: 4488, vlan: 1019, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32918 detail | i vlan
               mtu: 4488, vlan: 1019, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32919 detail | i vlan
               mtu: 4488, vlan: 1019, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32920 detail | i vlan
               mtu: 4488, vlan: 1019, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32921 detail | i vlan
               mtu: 4488, vlan: 1023, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32922 detail | i vlan
               mtu: 4488, vlan: 1023, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32923 detail | i vlan
               mtu: 4488, vlan: 1023, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32924 detail | i vlan
               mtu: 4488, vlan: 1023, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32925 detail | i vlan
               mtu: 4488, vlan: 1023, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32926 detail | i vlan
               mtu: 4488, vlan: 1023, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32927 detail | i vlan
               mtu: 4488, vlan: 1023, dindex: 0x0, l3rw_vld: 1
7600#sh mls cef adjacency entry 32928 detail | i vlan
               mtu: 4488, vlan: 1023, dindex: 0x0, l3rw_vld: 1
7600#sh vlan intern usage | i 1019|1023
1019 TenGigabitEthernet2/1
1023 TenGigabitEthernet2/3

So as can be seen, we have a 7/8 adjacency split in favour of vlan 1023. Also verified that 1019 and 1023 indeed map to the TenGig interfaces in question. So why the uneven split? IOS pecularities! ;)
How to fix?

In my case, this simple command does the magic:

mls ip cef load-sharing simple

This event was sponsored by Saku Ytti, the reigning Nordic Champion of the Cisco 7600 platform!

Comments