LFCE Preparation Guide: Network Administration – Monitoring Network Performance

Introduction

The scope of this competency has been difficult for me to determine.  The only other seemingly-reputable LFCE guide on the Internet (TecMint) hasn’t yet published its post on this subject, and I’m very interested to see how they handle it.

In my experience, monitoring network performance is typically done either at a high level with specialized security software focused largely on detecting anomalous behavior (i.e., if a system is typically idle at 2:00 AM but suddenly shows a large amount of network traffic, someone might want to know about it) or at a very on-demand level focused on determining the current cause of a networking/communication problem.  I’m very fond of generating usage reports and optimizing system performance, myself, but I don’t often concern myself too much with network usage.  I have been fortunate, I suspect, to most often operate in environments with no real bandwidth concerns.

The former will certainly not be included on the LFCE examination, but the latter might need some expanding.  I can imagine the examination asking for a report of network traffic during the completion of a certain task or over the course of a certain period of time.  It could reasonably ask for the administrator to identify which processes are communicating over the network and at what rate.  Whatever the case, given the automated scoring of the examination, we should be well prepared to produce reports and record data.

Perhaps the go-to solution for CentOS on the LFCE examination should be iptraf.  It is available in the base repository, and it includes logging features that allow for relatively detailed packet information to be captured over defined periods of time.  It doesn’t identify PIDs responsible for traffic, but for just about any other monitoring needs, it’s pretty decent.  It’s also based on ncurses, so its interface is pretty easy to navigate.

For observing process-based network traffic, nethogs can’t be beat.  It’s in the EPEL repository for CentOS 6.  I’m not sure if that’ll be available for the LFCE examination, but we can probably safely expect that it will be made available if necessary for the examination goal.  I recommend installing and testing nethogs – it’s very basic, but very useful when trying to isolate the problematic process eating up your bandwidth.

Again, from EPEL, vnstat is excellent for monitoring network usage over long periods of time (it runs as a daemon) and generating reports.  It is geared towards quantity of traffic and not any further details, so it’s primary use is in determining whether or not a system is routinely reaching the peak of its network performance, or if there are extended periods of disuse, etc.

Resources

  • Manual pages
    • iptraf(8)
    • nethogs(8)
    • vnstat(8)
    • vnstatd(1)

Techniques

  • Commands
    • iptraf
    • nethogs
    • vnstat
  • Logs
    • /var/log/iptraf
    • /var/lib/vnstat

Procedural Examples

  • Determine all hosts with whom the monitored system is communicating
  • Determine the process on a system currently consuming the most bandwidth
  • Configure a system so that its network usage is monitored over the appropriate period at appropriate intervals
  • Generate a report of historical bandwidth usage for a system
  • Generate a report of historically anomalous communications involving the monitored system

Tactical Exercises

As the Red Hat Enterprise Linux 6 Performance Tuning Guide states, most network issues are actually the result of faulty hardware or infrastructure.  This seems accurate to me, as most of my networking work involves troubleshooting (that’s another competency) and not monitoring.  As far as monitoring goes, configuring systems with the appropriate degree of network usage monitoring and generating reports at request is the next most likely area of effort expenditure, but it takes a back seat to troubleshooting networking issues.  The area of monitoring most often helpful in day-to-day operations is the determination of network usage per PID, so nethogs is a good (and simple) tool to understand.

Because network monitoring requires meaningful communication in order to serve as valuable exercise, the most convenient way to practice network monitoring is likely on a Fedora workstation.  I, for example, have a Fedora 21 desktop (with which you are familiar if you read the rest of the blog) which serves as my professional system when working from home (through which I establish a VPN connection with my organization and interact with various Linux and Windows systems) and my gaming platform when engaged in the occasional bout of Left 4 Dead 2 with friends on Steam.  These real-world system uses provide excellent exposure to the results gathered from network monitoring.

So…

1)  Implement vnstat network monitoring on your desktop.  After a day, week, or even month of use, query the vnstat databases for the network usage seen during that period of time.

2)  Investigate any anomalous activity.  If you were asleep at 2:00 AM while your computer ran and your network usage spiked, find out why!

3)  Determine if you are capable of maxing out your NIC given its limitations, your home routing infrastructure, and the bandwidth afforded by your Internet Service Provider.

4)  Configure iptraf to log hosts with whom you communicate over a given period of time (pick one during significant activity, of course).  Look for interesting host interactions – try enabling reverse DNS lookups by iptraf to assist in identifying questionable or otherwise interesting system interactions.

Learn to use common utilities such as grep and sort to parse iptraf log data.  It is very verbose, and even a five minute capture of low network activity can yield tens of thousands of lines of log data.  Knowing how to find all unique hostnames and IP addresses in the log, for example, can serve you very well; log analysis in general is a highly valuable and transferrable skill.

Advertisements
This entry was posted in Academics, Information Technology and tagged , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s