Traffic Logging with Asus RT-N56U

UPDATES! 21/6/14: Use a different interface that does just WAN traffic and made the cron thing actually work. Scroll down for the green bits (padavans firmware only though).

I’ve spent the last few days fiddling around with network traffic monitoring on my modem so I can track downloads for each different computer on my network. This article is a how to guide and a set of notes about getting it working.
If you want to follow this guide, you will need an Asus RT-N56U or one of the similar models. You can also mostly follow this guide using any OpenWRT routers (but you will need to adjust as you go to suit your router). I initially set it up using the stock firmware on the RT-N56U, however I ran into issues where it was reporting the wrong amount of traffic. After I installed the Padavan’s Firmware, I still had the same problem (I later solved it by disabling hardware NAT below). In theory this technique should work with the stock firmware and I’ll detail it below as well as how to get Padavan’s Firmware up and running too.

The final result of this is to be able to see a screen like this:

Method 1: Using Stock Firmware

Sorry this method isn’t super complete as I never got it working (though you can get it working in theory by following this guide).
1. Update to the latest firmware.
⁃ Always a good idea, you can download the firmware off the Asus Website
2. Enable Telnet
⁃ You can enable telnet by going to the Administration -> System section. You can log in via telnet using the username/password you’ve got set for the web interface login and either Putty on Windows or Terminal on Mac.
⁃ Note that the Padavans Firmware allows SSH access too.
3. Install wrtbwmon (http://code.google.com/p/wrtbwmon/)
⁃ Because you can’t write to the system in the stock firmware (that I know of), this app will exist in the /tmp directory and disappear after a reboot. The br0 part here is probably wrong, See the padavan version of it!
[code]wget http://wrtbwmon.googlecode.com/files/wrtbwmon -O /tmp/wrtbwmon && chmod +x /tmp/wrtbwmon
/tmp/wrtbwmon setup br0[/code]
– This Code will download wrtbwmon from the googlecode servers, save it into /tmp/ and make it executable. It’ll then do the initial setup of wrtbwmon.
4. Disable Hardware NAT. See Below to see the issues I had with Hardware NAT and the method this uses to track the usage. According to some forums (http://forums.smallnetbuilder.com/showthread.php?t=4317&page=9), if you enable some options like VPN passthrough or QoS, it will disable the hardware NAT and leave you on the software based one that allows the wrtbwmon to work. I didn’t get to test this because I ended up using Padavans Firmware that gives you a simple option to disable it.
5. Make a file to match MAC addresses to Names. (Optional)
⁃ This one is simple, just substitute the following with your own MAC address/Device names (The device listing in the Router web page will help you with this). Note: Using the stock firmware you need to use Upper case MAC Addresses.
⁃ Code (repeat this for each device.)
[code]echo "MA:CA:DD:RE:SS:00,DeviceName" >> /www/user/users.txt[/code]
6. Run wrtbwmon every x minutes to update the usage.
⁃ To update the usage, you need to run:
[code]/tmp/wrtbwmon update /tmp/usage.db peak[/code]
– This will update the usage. You should set this to run say every 15 minutes during peak hours, and replace the word peak with offpeak to do the off-peak hours. It doesn’t matter how often it is run, as it will zero the counters every time it is run (so you can run it every hour, or every minute if you like).
– After this is run, You’ll need to then write the usage data to the webpage form, which is done by the command:
[code]/tmp/wrtbwmon publish /tmp/usage.db /www/user/usage.htm /www/user/users.txt[/code]
– This tells wrtbwmon to publish a html file to /www/user/usage.htm, using the /www/user/users.txt file which has the MAC address -> User mapping
– The /www/user/ folder is actually a symlink to /tmp/www/user (correct me if I’m wrong, I’m writing this part from memory), so it gets wiped out when you reboot the router too.
7. Setup cron to automate the above step
⁃ I didn’t get to this step, the Hardware NAT issue in step 4 had me stumped until I was on Padavan’s firmware, but the above should work *in theory* on stock firmware.
8. Work out how to make it persist through reboots.
⁃ Again, another problem I didn’t get around to sorting out, as I had moved to Padavan’s firmware by this time. I believe there’s an option somehow to use a thing called optware, which will allow you to run the program off a USB stick and store the data there. Padavan’s firmware has an easier way of doing this 😛
If you work a better way of doing this (and instructions to step 7/8, please leave a comment!)
You can then access the traffic page at http://192.168.1.1/user/usage.htm

Method 2: Using Padavan’s Firmware

This is the method I ultimately ended up using after having the trouble with Hardware based NAT (Padavan’s has an easy option to turn it off).
1. Download the latest Padavan’s Firmware for your model of router
⁃ http://code.google.com/p/rt-n56u/
⁃ MAKE SURE YOU GET THE EXACT RIGHT VERSION FOR YOUR ROUTER. I don’t think it matters which one you get in terms of the aria/base/dlna version (they have slightly different additional features, I used the dlna version). There’s an N56U and N65U version, don’t get confused!
2. Install Padavan’s Firmware!
⁃ The version I grabbed was RT-N56U_3.4.3.7-072_dlna.zip
⁃ Unzip the Firmware, this will give you a .trx file like so: RT-N56U_3.4.3.7-072_dlna.trx
⁃ Go to the router home page -> System -> Firmware Update and upload the .trx file.
⁃ This will take 3-5 minutes or so to complete. Don’t interrupt it, don’t touch it, don’t touch your computer, don’t even breathe (this is the scariest step)
⁃ After the update is complete, your router will be reset to default settings. YOU CAN NOT RESTORE YOUR SETTINGS FROM THE OLD FIRMWARE USING THE .CFG FILE BACKUP THING. Just go through and redo all your settings manually.
3. Enable SSH support
⁃ This is under Administration -> System -> Enable SSH server
4. Enable entware.
⁃ This step requires a little bit of work, you should follow the guide here (basically gives you somewhere to write your program to on the USB drive), in theory, you could skip this step and put the wrtbwmon either in tmp where it won’t stick through a reboot, or find somewhere it will stick (maybe /etc/storage?). Entware gives you access to a whole bunch of other software you can run on your router too.
⁃ http://code.google.com/p/rt-n56u/wiki/HowToConfigureEntware
5. Disable Hardware NAT.
⁃ This is way easier on Padavan’s firmware, go to Advanced Settings -> WAN and pick “Offload TCP for LAN”
6. Install wrtbwmon. I decided to put mine in /opt/bin/ so it was on the USB drive (though I think technically it should be somewhere else)
[code]wget http://wrtbwmon.googlecode.com/files/wrtbwmon -O /opt/bin/wrtbwmon && chmod +x /opt/bin/wrtbwmon[/code]
– This downloads it from google code servers into /opt/bin/wrtbwmon and makes it executable.
7. Create the directory for the web interface.
⁃ Padavan’s firmware has a slightly different structure for where to put html files, instead of in /www/user/ (linking to /tmp/www/user/), it has the link in /www/custom/ pointing to /opt/share/www/custom/ (which will be on the USB drive assuming you followed the instructions above)
⁃ The command for this is:
[code]mkdir /opt/share/www/custom[/code]
8. Give it a test run. To run wrtbwmon, you’ll need to run the setup first: (Updated, see this link for different interfaces)

/opt/bin/wrtbwmon setup br0
[code]/opt/bin/wrtbwmon setup eth3[/code]
⁃ Then you’ll need to run it to update it every x minutes (say every 5minutes to an hour or so) (Substitute “peak” for “offpeak” to log offpeak times)
[code]/opt/bin/wrtbwmon update /tmp/usage.db peak[/code]
⁃ Then you need to publish the results:
[code]/opt/bin/wrtbwmon publish /tmp/usage.db /www/custom/usage.htm /www/custom/users.txt[/code]9. Far out, that step was so wrong and never worked. Time for a new version of Step 9!

Set up Cron (Properly!)

Make a new file for cron_tasks (run that nano command, paste everything from the next box in, then ctrl-o to save, ctrl-x to exit):
[code]# nano /etc/storage/cron_tasks[/code]
[code]#!/bin/sh

[/code]Once you’ve changed those two files, you need to make the cron_tasks file executable, then save the settings so they’ll stick across a reboot, and then finally run the started_script.sh. The following lines will do that:
[code]# chmod 755 /etc/storage/cron_tasks
# mtd_storage.sh
# /etc/storage/started_script.sh[/code]

You can then access the traffic page at http://192.168.1.1/custom/usage.htm

Hardware NAT Issues

So I had some massive issues getting iptables to keep track of how much data was going through, I would do about a gigabyte of downloads and iptables would think that only 200-300kb had gone through. I managed to narrow it down to the fact that instead of the routing going via iptables, it would go through the hardware based NAT and it wouldn’t be accounted for (boooo).

Posted by duck

31 comments

Thank you for a very informative blog. I’m not sure if this is the right spot for a question but I’ll ask anyway. I’m using the stock Firmware: I could not for the life of me setup the cron jobs. Do you have any hints how I can set them up?

Thanks for writing this up, very useful! I got everything working but was wondering if you knew a way to automatically reset the usage statistics on the Xth day of every month? I was wanting to use this to compare my usage against what my ISP is reporting.

You could delete the database file to reset it back to defaults at the end of each month? (or, make a script to move the DB file, then rerun the setup)…..
I had issues with my one tracking so I kind of abandoned the project 😐 When I get the custom firmware loaded back on my router I might have another shot 😛
Cheers,

Hi, really what I was looking for, don’t know why it is not already in the firmware itself.
Just one question: does the “Disable Hardware NAT” thing will create bottlenecks or decrease performance of the router?
Thank you!

Not that I’ve seen. My internet here is 100mbit, I get ~96mbit direct from the fiber. Via the router I get ~96mbit with both HWNAT on and off.
The reason why I bought this router is that it can handle WAN->LAN speeds of up and over 850mbit. So, I don’t know how much having HWNAT off really affects it.

Hi! I am stuck with a problem; when I run the “setup”:
/opt/bin/wrtbwmon setup eth3
the command never returns; is it ok?
How much time is it supposed to run?
I have tried with all the interfaces but always the same behaviour.
What could be? Any idea?
Thank you!

There’s not really a lot in the way of logging for the script. If you open it up in a text editor, you can read the code for it (it’s all shell script). I guess you could go through it line by line and see what part of it is failing?

The peak/offpeak is for when you’ve got offpeak/onpeak internet, like my internet here is 100gb for between 8am and 2am, and 100gb for 2am to 8am (so 200gig total).
The script basically starts a counter watching how much traffic goes through, then next time you run it, it adds the counters to your earlier numbers, then resets the counters. You could possibly rewrite it to keep track of how much traffic goes through every time you run it (like, say traffic that has gone through in one minute if you run it every minute). I’m not sure if you’ll run into any sort of load issues on the router because of it though.

Also, if you’ve managed to get it working, keep an eye on how long it goes for before it craps out, my one behaves weird and I can’t work out why just yet.
Like, my usenet machine doesn’t appear at all, and some traffic just doesn’t end up getting counted (and at some points, it just stops working all together until I run the setup command again).

I was thinking if I could get it to accurately track the values, I could get it to send the information to my server (just a little HP N54L) to do graphing and such, but I wasn’t going to start on that until it started cooperating with me properly…

If you’re curious about watching how much traffic is going through your router at one time, it should have SNMP enabled, so using something like Peakhour on Mac, you can have a little graphy thing showing total traffic (not for individual machines, the whole router).

I have a need to pull NetFlow stats from the WAN interface on my ASUS RT-65U. Your guide comes close to what I need, but I was hoping to get more, like type of traffic that NetFlow would provide. Have you ever considered using NetFlow or sFlow? There are a few free tools out there that can be used… I have four kids and 21 devices on the inside with a 200GB bandwidth cap and we get nailed every month… Would love to be a guinea pig if you ever considered it. Thanks!

Hey, I had a go of NetFlow on my new router, but ended up not liking it. There was just way way too much info and it was difficult to use, it also required another server to do all the logging. The info I wanted to see was how much traffic each machine was using and when.

I’m in the process of rebuilding my script for my new router (here’s a screenshot: http://poopr.org/images/s3jag56cov19pnsnkmdo.png )
Compared to the old script, this one looks better, runs faster and allows you to drill down the information and work out data usage per hour/day/week etc.
My script pulls it all down into a sqlite database too, so you could in theory write a little checker script that sees if any of your kids have gone over their limit, and then blacklist/throttle their connection.

In theory, it *should* work on an Asus RT-N56U or the other sorts as it uses the same technique as the old script, though I’ll need to do a bit of testing and build an installer for it (as while the core of the script is the same, there’s a bunch of different stuff in the way it works on the different platforms)… And of course, that’ll come after I actually finish the script.

The router I’m using now is an EdgeRouter (http://www.ubnt.com/edgemax/edgerouter-lite/) They’re ~$100. You can also get them exporting netflow/sflow stats without installing any extra crap. It’s also basically a little Debian box, glorious little powerhouse machine. The only downside to it is that it doesn’t have all the features the Asus routers do (like USB ports, extra LAN ports, Wifi etc). In my case, I’ve got a gigabit switch attached to it and my house is wired up with some nice OpenMesh access points (they’re pretty cheap too, and the CloudTrax interface for them is glorious.. If you’re keen I’ll write up a review on them).

Which firmware did you run your updated code on? I updated to 3.4.3.9-091 and having problems with this script. On step 9 I couldn’t use nano and had to use vi for a text editor. It looks like there may have been some changes to how crond is called starting after 3.x.3.8-086 (see pavadan’s common tips page). If I manually tell the wrtbmon to update and publish results it will work so it has to be some bad code in the cron scripts…..

Duck, could you check your CPU usage when your script runs? I ended up making two seperate .sh files (peak and off peak) and then setting them run under crontab -e. This is my CPU usage after changing the script to run every minute: http://i.imgur.com/12knsmd.png

On my router, the script was taking between 2-5sec to do the update command, the new version I’ve been working on has that well below one second. I believe the reasoning behind it is that the shell script calls heaps of different commands (lots of sed/awk), whereas my other one calls one command and then does all the processing in php.

It’s below 50% usage in your example so it should be okay, though, I’m not sure about running it every minute (as it takes 10x the resources than running it every 10minutes)

Also, try running : time ./script.sh
It will tell you how long it takes to run, the more hosts you have, the more time it will take, but anywhere between 1-7 seconds seems normal.

Still a work in progress but I will post some results when I get it going. I am still having problems with it updating through the started_script.sh. If I run it manually it updates but just doesn’t want to run on its own. Thanks for your work on this. The edgerouter is pretty tempting…

The only downside to it is that it’s *just* a router. WAN in, 2xLAN out and a console port, but it runs Debian so you can load it up with all sorts of packages, and the community likes people hacking around with it.

Did anybody figure out how to resolve the problem of getting stuck on: /opt/bin/wrtbwmon setup eth3
It sits there forever. It doesn’t matter whether other interface is included.
I was really hopping this would work.

#1
With regards to getting stuck on /opt/bin/wrtbwmon setup eth3
I think I resolved it. I discovered why it was getting stuck.
There is a line in wrtbwmon that is not working. A command to get the name of the lan interface is not returning anything but we can easily fix it since we already now it is br0.
You need to edit the script and correct a line (make a copy of the file first with a different name in case you want to revert)
Comment this line by adding a # at the beginning
From
LAN_IFACE=$(nvram get lan_ifname)
To
#LAN_IFACE=$(nvram get lan_ifname)
AND add the following line right underneath
LAN_IFACE=br0
You can use nano or vi to make this edit. For instance nano /opt/bin/wrtbwmon.

#2
The following command didnt work for me:
mtd_storage.sh
It asked me for more parameters. I entered it in this manner and it worked:
mtd_storage.sh save

I’m having trouble with step 9 in green. The cron stuff just doesn’t work. The only way it updates the traffic is if I do step 8:
/opt/bin/wrtbwmon setup eth3
/opt/bin/wrtbwmon update /tmp/usage.db peak
/opt/bin/wrtbwmon publish /tmp/usage.db /www/custom/usage.htm /www/custom/users.txt

Can anyone help me set up the scheduler so it updates automatically? Also, I tried to troubleshoot the started_script.sh, and found out that the it does indeed create the /var/spool/cron/crontabs directory, but beyond that, it does nothing. Therefore, I think the script stops at the
Login=`nvram get http_username`
and so on.