Webinar Recap: Top 5 Utilities for Network Engineers

In a recent NetBeez webinar, I presented on “Top 5 Utilities for Network Engineers”, which ended up being one of the most popular webinars we have ever hosted! I talked about the command-line Linux utilities that you can use for troubleshooting, but the time constraints of a live webinar only allowed for a quick demonstration. In saying that, I would like to further explain the utility commands for you to follow along at your own pace.

How can you get access to a Linux console?

If you are new to Linux, there are many options to gain access to a Linux box for free (or at a very low cost):

Mac OS: if you are a Mac user all you have to do is a open a terminal window. The operating system itself is based on Unix, which is a close cousin to Linux.

Windows 10: you can use Linux-like environments such as Cygwin. In addition, in 2016, Windows 10 started supporting Linux as a native application.

Cloud Linux: most major cloud providers like AWS, Azure, GCP, have free-tier offerings that you can use to spin up a Linux box in the cloud.

Linux box: if you want to have your own Linux box, considerinvesting $50-$100 on a single board computers such as the Raspberry Pi, Odroid, or Beaglebone. Also, if you have an old laptop you can install Linux on it.

All of the utility examples below were performed on Debian-based Linux. Note: If you are using some other distribution, things might be slightly different.

The first thing that you need to do for each command is to install it on the OS (if it’s not installed already). Here are the commands to install the five utilities:

1

2

3

4

5

sudo apt-get install nmap

sudo apt-get install tcpdump

sudo apt-get install netcat

sudo apt-get install iperf

sudo apt-get install python-pip;sudo pip install speedtest-cli

The protocol relies on a software agent that runs on each monitored device and replies to queries from a network management server (NMS). The NMS, also called SNMP poller, periodically requests each device utilization values of its resources to get a status update and verify that it’s working properly. If the value of one or more resources reported by the agent exceed a threshold set by the administrator, the server will generate an alert for the network administrator. An SNMP agent uses port UDP 161 to receive requests from a poller. SNMP can also be used to apply configuration changes to devices and, if needed, to send notifications, called traps, to an SNMP trap receiver when an event that requires administrative attention happens on the device itself. An SNMP trap could be generated if, for example, the network interface of a router goes down or if a BGP neighbor becomes unreachable. By default, SNMP traps are sent via UDP to port 162.

The nmap ping sweep scans a subnet for any available hosts; it’s one of the most basic commands you can run. Here is what it found in my local subnet:

1

2

3

4

5

6

7

8

9

10

11

172.31.0.25@netbeez.net$nmap-sP172.31.0.0/24

Starting Nmap6.40(http://nmap.org ) at 2017-11-06 10:59 PST

Nmap scan report for172.31.0.167

Host isup(-0.100slatency).

MAC Address:B8:27:EB:96:CF:1F(Raspberry Pi Foundation)

Nmap scan report for172.31.0.202

Host isup(0.00098slatency).

MAC Address:B8:27:EB:AA:1C:E9(Raspberry Pi Foundation)

Nmap scan report for172.31.0.25

Host isup.

Nmap done:256IP addresses(11hosts up)scanned in5.35seconds

It took nmap 5.35 seconds to get information on IPs, MACs, OUI lookups, and latencies for each host.
If I scan a specific host I get the following details which include open port information:

1

2

3

4

5

6

7

8

9

10

11

172.31.0.25@netbeez.net$nmap172.31.0.1

Starting Nmap6.40(http://nmap.org ) at 2017-11-06 12:06 PST

Nmap scan report for172.31.0.1

Host isup(0.026slatency).

Notshown:916closed ports,81filtered ports

PORT STATE SERVICE

22/tcp open ssh

25/tcp open smtp

111/tcp open rpcbind

MAC Address:00:01:C0:15:A3:32(CompuLab)

Nmap done:1IP address(1host up)scanned in1.67seconds

And if I want more details on the operating system I can use the “-O” option as follows:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

172.31.0.25@netbeez.net$nmap-O172.31.0.1

Starting Nmap6.40(http://nmap.org ) at 2017-11-06 12:04 PST

Nmap scan report for172.31.0.1

Host isup(0.019slatency).

Notshown:916closed ports,81filtered ports

PORT STATE SERVICE

22/tcp open ssh

25/tcp open smtp

111/tcp open rpcbind

MAC Address:00:01:C0:15:A3:32(CompuLab)

Device type:general purpose

Running:Linux2.6.X|3.X

OS CPE:cpe:/o:linux:linux_kernel:2.6cpe:/o:linux:linux_kernel:3

OS details:Linux2.6.32-3.2

Network Distance:1hop

OS detection performed.Please report any incorrect results at

http://nmap.org/submit/ .

Nmap done:1IP address(1host up)scanned in3.85seconds

The operating system scan took more than double the time of the previous scan. Keep this in mind that when you try to do operating-system scans fors subnets with many hosts, since it might take awhile to get the results back.

A common use case for nmap is to scan a network before and after a change to make sure that all hosts are connected back to the network after the change. Here is a one-line bash loop that runs the ping sweep command every 5 seconds.

1

while[`clear`];donmap-sP172.31.0.0/24;sleep5;done

You run this command until you verify all hosts are back, and terminate it by hitting Ctlr+C on your console.For each one of these utilities you can get detailed information about their capabilities and options by reading their manual. On the console you can just type “man nmap” or “nmap -help”.

The manual can be difficult to read and might be complicated for novice users, in which case, you can find tons of tutorials and guides online. For example, here is an excellent nmap tutorial with many more details: https://hackertarget.com/nmap-tutorial/

The operating system scan took more than double the time of the previous scan. Keep this in mind that when you try to do operating-system scans fors subnets with many hosts, since it might take awhile to get the results back.

A common use case for nmap is to scan a network before and after a change to make sure that all hosts are connected back to the network after the change. Here is a one-line bash loop that runs the ping sweep command every 5 seconds.

As the name suggests, tcpdump dumps network traffic onto your terminal window. The simplest command you can use is “tcpdump -i eth0”, which dumps all packets going in and out of interface eth0. Here is a short snippet of the output:

1

2

3

4

5

6

7

8

9

172.31.0.25@netbeez.net$tcpdump-ieth0

tcpdump:verbose output suppressed,use-vor-vv forfull protocol decode

listening on eth0,link-type EN10MB(Ethernet),capture size262144bytes

12:22:55.983450IP172.31.0.25.ssh>172.30.10.202.56410:Flags[P.],seq

1483392643:1483392831,ack3843673783,win168,length188

Sorting through all packets going in and out of an interface can be overwhelming. To be productive with tcpdump you need to use the right filters to display only the important packets and traffic. As an example, I can filter packets by host 172.30.10.202 with
tcpdmp-ieth0 host172.30.10.202 or
tcpdmp-ieth0 port53

A common tcpdump use case is to connect your Linux box to a switch’s span port and capture all traffic. In that case, using the right filters is the key to looking at the packets relevant to your issue.

Netcat allows you to create connections between two hosts with TCP and UDP traffic. To run this example, you can use two different hosts, or open two console windows on the same host.

We can create a server-client communication by having the server listen on port 20000 for connections with the commandnetcat-l-p20000.On the client console we can connect to this server with
netcat172.31.0.2520000. Once the connection is established, anything that we type on client window will appear on the server window and vice versa.

Server window:

1

2

172.31.0.25@netbeez.net$netcat-l-p20000

Hello World!

Client Window:

1

2

172.31.0.142@netbeez.net$netcat172.31.0.2520000

Hello World!

You can use this server-client communication to test if a firewall successfully blocks traffic by setting up the server behind the firewall and trying to connect to it from the outside world.

Netcat can also be used as a “quick-and-dirty” way to move files between hosts. Let’s assume we want to move log_file.txt. Here are the the commands you need to run:

iPerf is a performance testing tool that sends TCP or UDP traffic between two hosts and measures the bandwidth it can generate. There are two versions of iPerf. iPerf 2 is the most prolific one since it’s older and more widely used. iPerf3 is more recent, and although it’s very similar in functionality with iPerf2, they are incompatible to each other. At NetBeez, we use iPerf 2.

iPerf needs a source and a destination host to send and receive the traffic. The first step is to start the iPerf server on the receiving side which will wait for iPerf traffic to be sent. The default option is to send TCP traffic, in which case it tries to push as much bandwidth as possible between the source and the destination.

1

2

3

4

5

6

7

8

172.31.0.25@netbeez.net$iperf-s

------------------------------------------------------------

Server listening on TCP port5001

TCP window size:85.3KByte(default)

------------------------------------------------------------

[4]local172.31.0.25port5001connected with172.31.0.142port48180

[ID]Interval Transfer Bandwidth

[4]0.0-10.0sec112MBytes94.1Mbits/sec

On the sending side we start the iPerf client with the command:

1

2

3

4

5

6

7

8

172.31.0.142@netbeez.net$iperf-c172.31.0.25

------------------------------------------------------------

Client connecting to172.31.0.25,TCP port5001

TCP window size:43.8KByte(default)

------------------------------------------------------------

[3]local172.31.0.142port48180connected with172.31.0.25port5001

[ID]Interval Transfer Bandwidth

[3]0.0-10.0sec112MBytes94.2Mbits/sec

After 10 seconds the test is over, and we see that the client and server were able to achieve 94.2 Mbps.

For UDP traffic we have to use the “-u” option as follows on the server side:

1

2

3

4

5

6

7

8

9

172.31.0.25@netbeez.net$iperf-s-u

------------------------------------------------------------

Server listening on UDP port5001

Receiving1470bytedatagrams

UDP buffer size:208KByte(default)

------------------------------------------------------------

[3]local172.31.0.25port5001connected with172.31.0.142port48295

[ID]Interval Transfer Bandwidth Jitter Lost/Total Datagrams

[3]0.0-10.0sec3.58MBytes3.00Mbits/sec0.026ms0/2552(0%)

On the sending side, we also have to use the “-u” option. In addition, we can determine the amount of UDP traffic by using the “-b” option as follows:

1

2

3

4

5

6

7

8

9

10

11

12

172.31.0.142@netbeez.net$iperf-c172.31.0.25-u-b3M

------------------------------------------------------------

Client connecting to172.31.0.25,UDP port5001

Sending1470bytedatagrams

UDP buffer size:160KByte(default)

------------------------------------------------------------

[3]local172.31.0.142port48295connected with172.31.0.25port5001

[ID]Interval Transfer Bandwidth

[3]0.0-10.0sec3.58MBytes3.00Mbits/sec

[3]Sent2552datagrams

[3]Server Report:

[3]0.0-10.0sec3.58MBytes3.00Mbits/sec0.025ms0/2552(0%)

As requested the client was able to send 3 Mbps of UDP traffic to the server. In addition, UDP iPerf gives the jitter (0.025 ms) and packet loss (0/2552 0%) at the end.
A common use case of iPerf is to prove that “It’s not the Network!”, by verifying that the network can pass a certain amount of traffic.

This is the console version of the Ookla speedtest.net that you can run on your browser. Ookla has close to 5000 servers around the world that can be used to measure how much upload and download bandwidth can be achieved. The difference with iPerf is that, with the speedtest, you don’t have control of the server that is used to test your bandwidth performance. For example, the speedtest measurements can be affected by the number of concurrent tests a server is running.

As you can see, it informs me that the server it selected is maintained by Softlayer Technologies, Inc in San Jose (my current location) with latency 24.371 ms. The download and upload speeds achieved are 57.64 Mbps and 6.22 Mbps.

All of these tie with the NetBeez dashboard through the GUI console. In a nutshell, NetBeez captures the user experience by using wired and wireless hardware sensors deployed in a WAN or WLAN network. Each sensor can run automatic tests such as ping, dns, http, iperf, and speedtest. Since each sensor is a Linux box, NetBeez gives console access to them through the GUI.

Having NetBeez agents deployed allows you to conduct a quick troubleshooting session without having to get remote access to a local machine, go through firewalls, or connect to a NAT’ed host.