Talking SNMP(v3) with Nokia Siemens Network (NSN) Surpass hiT 7300

During a recent 40G trial with NSN equipment I found myself wanting to have a (close to) realtime view of performance data of the box, unfortunately this is not provided by the crippled NSN GUI...

If you have used DWDM equipment from some of the larger vendors you know the management systems are often rather cumbersome to deal with and crippled in many ways. In addition, since the management system is sold by the same party as the DWDM system itself, there is a strong incentive from the vendor to hinder you from using other standard management systems with their DWDM systems as they can make more money selling you both.

In the NSN hiT7300 you will find just such an example. I've been participating in some 40G tests with coherent transponders and while tuning the underlying DWDM system one wants to look at the performance data in close to real time to see how changes affects the overall performance. Via the Element Manager built into the hit7300 you will only find a 15 minute counter. Not only is this a fairly large timespan but it is also built in a rather strange fashion. If you look in a Cisco router on the input / output rate of an interface, you will get the average of a sliding window covering approximately the last X seconds. For example;

ROUTER#sh inte te 2/3 | i put rate
  30 second input rate 3576021000 bits/sec, 474293 packets/sec
  30 second output rate 554643000 bits/sec, 161213 packets/sec

will show you an average of the last 30 seconds (plus a few seconds as the counters are cached for a few seconds by default). The hiT7300 on the other hand starts a new window at an even quarter of an hour, ie minute 00, 15, 30 or 45 and calculates / presents an average from that time till the current. This means that if you look at the counter at xx:00:01 you will get the average of the last second, whereas if you look at xx:05:00 you will get a five minute average and up to xx:14:59 where you will be presented with almost a quarter of an hours average. Imagine doing a change and having to wait for more than fifteen minutes to really see the effect of your change.

Not good enough I say and started looking for alternative methods. It hit me when logging on to the Element Manage of the hiT7300 that you actually specify both remote host and a port. That port is set to 161 by default... SNMP!

Given the username and password it was obvious this thing ran SNMPv3 and never having touched said version of SNMP, this proved somewhat of a challenge. SNMPv3 really feels like a giant, you need both authentication and privacy (encryption) settings, the Net-SNMP documentation is pretty bad and it's all just rather frustrating. After a few hours and no results, I went to the site where the hiT7300 was located, pulled the CF card and performed a dd to a local copy. Analyzing the image, I can tell you the control board runs Linux 2.6.12 on a e500 CPU (as can also be witnessed just by the diagnostic data). File system is cramfs, given the e500 CPU, it's big endian and my laptop is of course little endian. I tried using cramfsswap which supposedly should be able to swap endianess of a cramfs filesystem. To my dismay, it was able to display files but most were unreadable. Instead I turned to fuse-cram which, using the latest version and mounted in read-only mode, was able to decode the files just fine. I found /etc/passwd, a few ssh keys (mostly public) and some other interesting stuff. Above all though, a file containing some SNMP configuration, I found the user I had previously configured and could see the password and privacy key, although in hex format. What was now obvious was that these were equal. I did try using my password as cypher key in my initial testing, but having the wrong context name, it still didn't work.

Anyway, now that I had all the cryptographic parameters needed, I could input these into Wireshark and watch the actual conversation between the element manager GUI on my computer and the hiT7300 chassis. The mentioned context name that is mandatory (and will result in no responses if incorrect) is 'tnms', ie the name of NSNs management system.

To find the actual performance counters I was interested in I fired up the performance view in the element manager and took a snapshot in Wireshark while pressing the Update button. Got a few SNMP OIDs and tried polling.. tried a lot actually, two hours later I give up and get a fresh copy of the diagnostics data from the element manager GUI (right click on controller -> something -> something -> diagnostics data, and upload it to a FTP). There's a log/security_snmp.txt in the diagnostics data file you get and in there I found that I was missing a session, it seems the EM GUI sets a OID which registers a session and after that is able to poll these values. The OID set incorporates the source IP address of the client as well as port number. Since snmpget uses a random source port for every query, I couldn't first set a session and then send a get request as I would surely be allocated different ports between my runs of the snmp tools. It was time for python!

... and more on that in a later post! At least now you know how to query the NSN hiT7300;

kll@machine ~ $ snmpget -v3 -a MD5 -X AES -l authPriv -n tnms \
SNMPv2-MIB::sysDescr.0 = STRING: <censored> hiT7300 4.30.01  3.0EB0050101101020 \ 
   Copyright 2010 Nokia Siemens Networks. All rights reserved.