Duck's Tech Blog

Traffic Logging with Asus RT-N56U

by on Dec.31, 2013, under Home, How-To Guides

UPDATES! 21/6/14: Use a different interface that does just WAN traffic and made the cron thing actually work. Scroll down for the green bits (padavans firmware only though).

I’ve spent the last few days fiddling around with network traffic monitoring on my modem so I can track downloads for each different computer on my network. This article is a how to guide and a set of notes about getting it working.
If you want to follow this guide, you will need an Asus RT-N56U or one of the similar models. You can also mostly follow this guide using any OpenWRT routers (but you will need to adjust as you go to suit your router). I initially set it up using the stock firmware on the RT-N56U, however I ran into issues where it was reporting the wrong amount of traffic. After I installed the Padavan’s Firmware, I still had the same problem (I later solved it by disabling hardware NAT below). In theory this technique should work with the stock firmware and I’ll detail it below as well as how to get Padavan’s Firmware up and running too.

The final result of this is to be able to see a screen like this:
Screen Shot 2013-12-31 at 11.49.33 am

Method 1: Using Stock Firmware

Sorry this method isn’t super complete as I never got it working (though you can get it working in theory by following this guide).
1. Update to the latest firmware.
⁃ Always a good idea, you can download the firmware off the Asus Website
2. Enable Telnet
⁃ You can enable telnet by going to the Administration -> System section. You can log in via telnet using the username/password you’ve got set for the web interface login and either Putty on Windows or Terminal on Mac.
⁃ Note that the Padavans Firmware allows SSH access too.
3. Install wrtbwmon (http://code.google.com/p/wrtbwmon/)
⁃ Because you can’t write to the system in the stock firmware (that I know of), this app will exist in the /tmp directory and disappear after a reboot. The br0 part here is probably wrong, See the padavan version of it!

wget http://wrtbwmon.googlecode.com/files/wrtbwmon -O /tmp/wrtbwmon && chmod +x /tmp/wrtbwmon
/tmp/wrtbwmon setup br0

– This Code will download wrtbwmon from the googlecode servers, save it into /tmp/ and make it executable. It’ll then do the initial setup of wrtbwmon.
4. Disable Hardware NAT. See Below to see the issues I had with Hardware NAT and the method this uses to track the usage. According to some forums (http://forums.smallnetbuilder.com/showthread.php?t=4317&page=9), if you enable some options like VPN passthrough or QoS, it will disable the hardware NAT and leave you on the software based one that allows the wrtbwmon to work. I didn’t get to test this because I ended up using Padavans Firmware that gives you a simple option to disable it.
5. Make a file to match MAC addresses to Names. (Optional)
⁃ This one is simple, just substitute the following with your own MAC address/Device names (The device listing in the Router web page will help you with this). Note: Using the stock firmware you need to use Upper case MAC Addresses.
⁃ Code (repeat this for each device.)

echo "MA:CA:DD:RE:SS:00,DeviceName" >> /www/user/users.txt

6. Run wrtbwmon every x minutes to update the usage.
⁃ To update the usage, you need to run:

/tmp/wrtbwmon update /tmp/usage.db peak

– This will update the usage. You should set this to run say every 15 minutes during peak hours, and replace the word peak with offpeak to do the off-peak hours. It doesn’t matter how often it is run, as it will zero the counters every time it is run (so you can run it every hour, or every minute if you like).
– After this is run, You’ll need to then write the usage data to the webpage form, which is done by the command:

/tmp/wrtbwmon publish /tmp/usage.db /www/user/usage.htm /www/user/users.txt

– This tells wrtbwmon to publish a html file to /www/user/usage.htm, using the /www/user/users.txt file which has the MAC address -> User mapping
– The /www/user/ folder is actually a symlink to /tmp/www/user (correct me if I’m wrong, I’m writing this part from memory), so it gets wiped out when you reboot the router too.
7. Setup cron to automate the above step
⁃ I didn’t get to this step, the Hardware NAT issue in step 4 had me stumped until I was on Padavan’s firmware, but the above should work *in theory* on stock firmware.
8. Work out how to make it persist through reboots.
⁃ Again, another problem I didn’t get around to sorting out, as I had moved to Padavan’s firmware by this time. I believe there’s an option somehow to use a thing called optware, which will allow you to run the program off a USB stick and store the data there. Padavan’s firmware has an easier way of doing this :P
If you work a better way of doing this (and instructions to step 7/8, please leave a comment!)
You can then access the traffic page at http://192.168.1.1/user/usage.htm

Method 2: Using Padavan’s Firmware

This is the method I ultimately ended up using after having the trouble with Hardware based NAT (Padavan’s has an easy option to turn it off).
1. Download the latest Padavan’s Firmware for your model of router
http://code.google.com/p/rt-n56u/
⁃ MAKE SURE YOU GET THE EXACT RIGHT VERSION FOR YOUR ROUTER. I don’t think it matters which one you get in terms of the aria/base/dlna version (they have slightly different additional features, I used the dlna version). There’s an N56U and N65U version, don’t get confused!
2. Install Padavan’s Firmware!
⁃ The version I grabbed was RT-N56U_3.4.3.7-072_dlna.zip
⁃ Unzip the Firmware, this will give you a .trx file like so: RT-N56U_3.4.3.7-072_dlna.trx
⁃ Go to the router home page -> System -> Firmware Update and upload the .trx file.
⁃ This will take 3-5 minutes or so to complete. Don’t interrupt it, don’t touch it, don’t touch your computer, don’t even breathe (this is the scariest step)
⁃ After the update is complete, your router will be reset to default settings. YOU CAN NOT RESTORE YOUR SETTINGS FROM THE OLD FIRMWARE USING THE .CFG FILE BACKUP THING. Just go through and redo all your settings manually.
3. Enable SSH support
⁃ This is under Administration -> System -> Enable SSH server
4. Enable entware.
⁃ This step requires a little bit of work, you should follow the guide here (basically gives you somewhere to write your program to on the USB drive), in theory, you could skip this step and put the wrtbwmon either in tmp where it won’t stick through a reboot, or find somewhere it will stick (maybe /etc/storage?). Entware gives you access to a whole bunch of other software you can run on your router too.
http://code.google.com/p/rt-n56u/wiki/HowToConfigureEntware
5. Disable Hardware NAT.
⁃ This is way easier on Padavan’s firmware, go to Advanced Settings -> WAN and pick “Offload TCP for LAN”
6. Install wrtbwmon. I decided to put mine in /opt/bin/ so it was on the USB drive (though I think technically it should be somewhere else)

wget http://wrtbwmon.googlecode.com/files/wrtbwmon -O /opt/bin/wrtbwmon && chmod +x /opt/bin/wrtbwmon

– This downloads it from google code servers into /opt/bin/wrtbwmon and makes it executable.
7. Create the directory for the web interface.
⁃ Padavan’s firmware has a slightly different structure for where to put html files, instead of in /www/user/ (linking to /tmp/www/user/), it has the link in /www/custom/ pointing to /opt/share/www/custom/ (which will be on the USB drive assuming you followed the instructions above)
⁃ The command for this is:

mkdir /opt/share/www/custom

8. Give it a test run. To run wrtbwmon, you’ll need to run the setup first: (Updated, see this link for different interfaces)

/opt/bin/wrtbwmon setup br0

/opt/bin/wrtbwmon setup eth3

⁃ Then you’ll need to run it to update it every x minutes (say every 5minutes to an hour or so) (Substitute “peak” for “offpeak” to log offpeak times)

/opt/bin/wrtbwmon update /tmp/usage.db peak

⁃ Then you need to publish the results:

/opt/bin/wrtbwmon publish /tmp/usage.db /www/custom/usage.htm /www/custom/users.txt

9. Far out, that step was so wrong and never worked. Time for a new version of Step 9!

Set up Cron (Properly!)

Make a new file for cron_tasks (run that nano command, paste everything from the next box in, then ctrl-o to save, ctrl-x to exit):

# nano /etc/storage/cron_tasks
#!/bin/sh
 
mode="$1"
case $mode in
    peak)
        /opt/bin/wrtbwmon update /tmp/usage.db peak;
        /opt/bin/wrtbwmon publish /tmp/usage.db /www/custom/usage.htm /www/custom/users.txt
    ;;
    offpeak)
        /opt/bin/wrtbwmon update /tmp/usage.db offpeak;
        /opt/bin/wrtbwmon publish /tmp/usage.db /www/custom/usage.htm /www/custom/users.txt
    ;;
    *)
        exit 0
    ;;
esac

Next, add some info to the started_script.sh file. As above, run the nano command, go to the bottom of the document, paste your code in and Ctrl-O, Ctrl-X to save/exit.

# nano /etc/storage/started_script.sh
################# user crontabs create ###################
# if app not exist
        if [ ! -f /usr/sbin/crond ]; then
            exit 0
        fi
        if [ -n "`pidof crond`" ] ; then
                # stop daemon
        killall -q crond
        fi
# create /var/spool/cron/crontabs directory
        if [ ! -d /var/spool/cron/crontabs ]; then
                mkdir -p /var/spool/cron
                cd /var/spool/cron
                mkdir -p /var/spool/cron/crontabs
        fi

Login=`nvram get http_username`

        touch /var/spool/cron/crontabs/$Login
        echo "SHELL=/bin/sh" > /var/spool/cron/crontabs/$Login
        echo "MAILTO=""" >> /var/spool/cron/crontabs/$Login
        echo "HOME=/" >> /var/spool/cron/crontabs/$Login
        echo "*/20,59 0-2 * * * /etc/storage/cron_tasks peak" >> /var/spool/cron/crontabs/$Login
        echo "*/20,59 3-8 * * * /etc/storage/cron_tasks offpeak" >> /var/spool/cron/crontabs/$Login
        echo "*/20,59 9-23 * * * /etc/storage/cron_tasks peak" >> /var/spool/cron/crontabs/$Login
        /usr/sbin/crond -l 0 -L /var/log/cron.log
##########################################################
/opt/bin/wrtbwmon setup eth3

Once you’ve changed those two files, you need to make the cron_tasks file executable, then save the settings so they’ll stick across a reboot, and then finally run the started_script.sh. The following lines will do that:

# chmod 755 /etc/storage/cron_tasks
# mtd_storage.sh
# /etc/storage/started_script.sh

You can then access the traffic page at http://192.168.1.1/custom/usage.htm

Hardware NAT Issues

So I had some massive issues getting iptables to keep track of how much data was going through, I would do about a gigabyte of downloads and iptables would think that only 200-300kb had gone through. I managed to narrow it down to the fact that instead of the routing going via iptables, it would go through the hardware based NAT and it wouldn’t be accounted for (boooo).

Links:

The Asus RT-N56U Manual: http://dlcdnet.asus.com/pub/ASUS/wireless/RT-N56U/E7822_RT_N56U_Manual_English.pdf
Padavan’s Firmware: http://code.google.com/p/rt-n56u/
Padavan’s Firmware entware Setup: http://code.google.com/p/rt-n56u/wiki/HowToConfigureEntware
Padavan’s Firmware cron Setup: https://code.google.com/p/rt-n56u/wiki/CommonTips#Using_the_built-in_scheduler_%28crond%29
wrtbwmon Homepage: http://code.google.com/p/wrtbwmon/


31 Comments for this entry

  • Jason

    Thank you for a very informative blog. I’m not sure if this is the right spot for a question but I’ll ask anyway. I’m using the stock Firmware: I could not for the life of me setup the cron jobs. Do you have any hints how I can set them up?

    Thanks.

  • duck

    Hey, I couldn’t work out how to get the cron jobs to work either. It looks like cron is there, but nothing I did would get it to work :|
    You could try a shell script with a sleep in it though?

    I haven’t done much work on it for a little while, but it’s on my list of things to do! If you work it out, leave a comment for me :D

    Cheers,

    Ducky

  • Brian

    Thanks for writing this up, very useful! I got everything working but was wondering if you knew a way to automatically reset the usage statistics on the Xth day of every month? I was wanting to use this to compare my usage against what my ISP is reporting.

  • duck

    You could delete the database file to reset it back to defaults at the end of each month? (or, make a script to move the DB file, then rerun the setup)…..
    I had issues with my one tracking so I kind of abandoned the project :| When I get the custom firmware loaded back on my router I might have another shot :P
    Cheers,

    Ducky

  • brad parsons

    Im currently trying to follow your guide and have gotten stuck at the wrtbwmon setup stage.

    it just sits there in the terminal and does nothing, any help would be appreciated!

  • duck

    Did the setup script finish? Or does it get stuck without taking you back to the console?

  • Brad parsons

    I get to

    /opt/bin/wrtbwmon setup br0
    I then get just a green square and it does nothing.

  • Camillo

    Hi, really what I was looking for, don’t know why it is not already in the firmware itself.
    Just one question: does the “Disable Hardware NAT” thing will create bottlenecks or decrease performance of the router?
    Thank you!

  • duck

    Not that I’ve seen. My internet here is 100mbit, I get ~96mbit direct from the fiber. Via the router I get ~96mbit with both HWNAT on and off.
    The reason why I bought this router is that it can handle WAN->LAN speeds of up and over 850mbit. So, I don’t know how much having HWNAT off really affects it.

  • Camillo

    Hi! I am stuck with a problem; when I run the “setup”:
    /opt/bin/wrtbwmon setup eth3
    the command never returns; is it ok?
    How much time is it supposed to run?
    I have tried with all the interfaces but always the same behaviour.
    What could be? Any idea?
    Thank you!

  • duck

    It shouldn’t take more than a few seconds to run. Sounds like there’s something going wrong! Not sure what though :|

  • Camillo

    The only difference is that I set both TCP & UDP offloading.
    Tried also to put the “wrtbwmon” in “tmp” folder but same result.
    Does this script have logs somewhere?
    Thank you!

  • duck

    There’s not really a lot in the way of logging for the script. If you open it up in a text editor, you can read the code for it (it’s all shell script). I guess you could go through it line by line and see what part of it is failing?

  • Camillo

    It seems the variable “lan_ifname” of the “wrtbwmon ” script is empty:

    LAN_IFACE=$(nvram get lan_ifname)

    /opt/home/admin # nvram get lan_ifname
    /opt/home/admin #

    then it stops in the loop for each host in the arp table…

    These are the variables I get with “lan”:
    /opt/home/admin # nvram show | grep lan
    vlan_vid_iptv=
    vlan_tag_lan1=0
    vlan_tag_lan2=0
    vlan_tag_lan3=0
    vlan_tag_lan4=0
    lan_gateway=192.168.1.1
    lan_addr6=
    lan_domain=
    lan_ipaddr_t=192.168.2.1
    lan_netmask=255.255.255.0
    http_lanport=80
    vlan_filter=0
    ether_flow_lan1=0
    lan_stp=1
    ether_flow_lan2=0
    ether_flow_lan3=0
    ether_flow_lan4=0
    ip6_lan_sfpe=4352
    ip6_lan_radv=1
    lan_hwaddr=D8:50:E6:96:07:A7
    ip6_lan_sfps=4096
    ip6_lan_auto=0
    lan_dns1=
    lan_dns2=
    vlan_pri_lan1=0
    vlan_pri_lan2=0
    vlan_pri_lan3=0
    vlan_pri_lan4=0
    ip6_lan_dhcp=1
    rt_guest_lan_isolate=1
    lan_gateway_t=192.168.2.1
    vlan_vid_cpu=
    lan_ipaddr=192.168.2.1
    preferred_lang=EN
    vlan_vid_lan1=
    vlan_vid_lan2=
    vlan_vid_lan3=
    vlan_vid_lan4=
    lan_dns_t=
    lan_netmask_t=255.255.255.0
    lan_dns_x=1
    ip6_lan_size=64
    link_lan=1
    vlan_pri_iptv=0
    vlan_pri_cpu=0
    lan_proto_x=1
    ip6_lan_sflt=1800
    ip6_lan_addr=
    ether_link_lan1=0
    ether_link_lan2=0
    ether_link_lan3=0
    ether_link_lan4=0
    wl_guest_lan_isolate=1
    lan_subnet_t=0xc0a80200

    And these with “ifname”:
    /opt/home/admin # nvram show | grep ifname
    wan0_ifname=eth3
    wan0_pppoe_ifname=ppp0
    viptv_ifname=
    wan_ifname_t=eth3
    wan_ifname=eth3
    wan_pppoe_ifname=ppp0

  • duck

    Interesting, I don’t know :|

    Here’s what I get for the nvram show | grep ifname:
    /opt/home/admin # nvram show | grep ifname
    wan0_ifname=eth3
    wan0_pppoe_ifname=ppp0
    viptv_ifname=
    lan_ifname=br0
    wan_ifname_t=eth3
    wan_ifname=eth3
    wan_pppoe_ifname=ppp0

    You could try hardcoding it to br0 and see if it works?

  • Camillo

    Hi, yeah hardcoding
    LAN_IFACE=”br0″
    in the script it definitely work.

    Do You know if it is possible to graph the logs and what do “peak” and “offpeak” means?

    Can I have a measure of the current network interface speed?
    Or it does only report the maximum speed?
    Thank you!

  • duck

    The peak/offpeak is for when you’ve got offpeak/onpeak internet, like my internet here is 100gb for between 8am and 2am, and 100gb for 2am to 8am (so 200gig total).
    The script basically starts a counter watching how much traffic goes through, then next time you run it, it adds the counters to your earlier numbers, then resets the counters. You could possibly rewrite it to keep track of how much traffic goes through every time you run it (like, say traffic that has gone through in one minute if you run it every minute). I’m not sure if you’ll run into any sort of load issues on the router because of it though.

    Also, if you’ve managed to get it working, keep an eye on how long it goes for before it craps out, my one behaves weird and I can’t work out why just yet.
    Like, my usenet machine doesn’t appear at all, and some traffic just doesn’t end up getting counted (and at some points, it just stops working all together until I run the setup command again).

    I was thinking if I could get it to accurately track the values, I could get it to send the information to my server (just a little HP N54L) to do graphing and such, but I wasn’t going to start on that until it started cooperating with me properly…

    If you’re curious about watching how much traffic is going through your router at one time, it should have SNMP enabled, so using something like Peakhour on Mac, you can have a little graphy thing showing total traffic (not for individual machines, the whole router).

  • Eric

    Thanks Camillo,

    with the fix LAN_IFACE=”bra” in wrtbwmon script, now is work.

    Thank you Duck for your how-to. Now I can find the devices they drain my internet.

  • Eric

    Hi,
    I find is not all my computer is listed. Do you have some tips to resolv this?

  • Renato

    Man, thanks a lot for this detailed post! Really helpful!

  • Jonathan Merrill

    I have a need to pull NetFlow stats from the WAN interface on my ASUS RT-65U. Your guide comes close to what I need, but I was hoping to get more, like type of traffic that NetFlow would provide. Have you ever considered using NetFlow or sFlow? There are a few free tools out there that can be used… I have four kids and 21 devices on the inside with a 200GB bandwidth cap and we get nailed every month… Would love to be a guinea pig if you ever considered it. Thanks!

  • duck

    Hey, I had a go of NetFlow on my new router, but ended up not liking it. There was just way way too much info and it was difficult to use, it also required another server to do all the logging. The info I wanted to see was how much traffic each machine was using and when.

    I’m in the process of rebuilding my script for my new router (here’s a screenshot: http://poopr.org/images/s3jag56cov19pnsnkmdo.png )
    Compared to the old script, this one looks better, runs faster and allows you to drill down the information and work out data usage per hour/day/week etc.
    My script pulls it all down into a sqlite database too, so you could in theory write a little checker script that sees if any of your kids have gone over their limit, and then blacklist/throttle their connection.

    In theory, it *should* work on an Asus RT-N56U or the other sorts as it uses the same technique as the old script, though I’ll need to do a bit of testing and build an installer for it (as while the core of the script is the same, there’s a bunch of different stuff in the way it works on the different platforms)… And of course, that’ll come after I actually finish the script.

    The router I’m using now is an EdgeRouter (http://www.ubnt.com/edgemax/edgerouter-lite/) They’re ~$100. You can also get them exporting netflow/sflow stats without installing any extra crap. It’s also basically a little Debian box, glorious little powerhouse machine. The only downside to it is that it doesn’t have all the features the Asus routers do (like USB ports, extra LAN ports, Wifi etc). In my case, I’ve got a gigabit switch attached to it and my house is wired up with some nice OpenMesh access points (they’re pretty cheap too, and the CloudTrax interface for them is glorious.. If you’re keen I’ll write up a review on them).

    Thanks :)

    Ducky

  • Brian

    Which firmware did you run your updated code on? I updated to 3.4.3.9-091 and having problems with this script. On step 9 I couldn’t use nano and had to use vi for a text editor. It looks like there may have been some changes to how crond is called starting after 3.x.3.8-086 (see pavadan’s common tips page). If I manually tell the wrtbmon to update and publish results it will work so it has to be some bad code in the cron scripts…..

  • Brian

    Duck, could you check your CPU usage when your script runs? I ended up making two seperate .sh files (peak and off peak) and then setting them run under crontab -e. This is my CPU usage after changing the script to run every minute: http://i.imgur.com/12knsmd.png

  • duck

    Hey dude, it was the latest as of the time I wrote the article Dec 31 2013, so over a year old.

    I’m actually running an EdgeRouter now, so I don’t have my old one to do any more testing with. I’m guessing with your second post that you’ve worked out what was wrong with your crontab file :P

  • duck

    On my router, the script was taking between 2-5sec to do the update command, the new version I’ve been working on has that well below one second. I believe the reasoning behind it is that the shell script calls heaps of different commands (lots of sed/awk), whereas my other one calls one command and then does all the processing in php.

    It’s below 50% usage in your example so it should be okay, though, I’m not sure about running it every minute (as it takes 10x the resources than running it every 10minutes)

    Also, try running : time ./script.sh
    It will tell you how long it takes to run, the more hosts you have, the more time it will take, but anywhere between 1-7 seconds seems normal.

    Thanks,

    Ducky

  • Brian

    Still a work in progress but I will post some results when I get it going. I am still having problems with it updating through the started_script.sh. If I run it manually it updates but just doesn’t want to run on its own. Thanks for your work on this. The edgerouter is pretty tempting…

  • duck

    The EdgeRouter is godmode in router form.

    The only downside to it is that it’s *just* a router. WAN in, 2xLAN out and a console port, but it runs Debian so you can load it up with all sorts of packages, and the community likes people hacking around with it.

  • Gabriel

    Did anybody figure out how to resolve the problem of getting stuck on: /opt/bin/wrtbwmon setup eth3
    It sits there forever. It doesn’t matter whether other interface is included.
    I was really hopping this would work.

  • Gabriel

    #1
    With regards to getting stuck on /opt/bin/wrtbwmon setup eth3
    I think I resolved it. I discovered why it was getting stuck.
    There is a line in wrtbwmon that is not working. A command to get the name of the lan interface is not returning anything but we can easily fix it since we already now it is br0.
    You need to edit the script and correct a line (make a copy of the file first with a different name in case you want to revert)
    Comment this line by adding a # at the beginning
    From
    LAN_IFACE=$(nvram get lan_ifname)
    To
    #LAN_IFACE=$(nvram get lan_ifname)
    AND add the following line right underneath
    LAN_IFACE=br0
    You can use nano or vi to make this edit. For instance nano /opt/bin/wrtbwmon.

    #2
    The following command didnt work for me:
    mtd_storage.sh
    It asked me for more parameters. I entered it in this manner and it worked:
    mtd_storage.sh save

    I will report back tomorrow. It seem to be working.

  • AgSwag

    I’m having trouble with step 9 in green. The cron stuff just doesn’t work. The only way it updates the traffic is if I do step 8:
    /opt/bin/wrtbwmon setup eth3
    /opt/bin/wrtbwmon update /tmp/usage.db peak
    /opt/bin/wrtbwmon publish /tmp/usage.db /www/custom/usage.htm /www/custom/users.txt

    Can anyone help me set up the scheduler so it updates automatically? Also, I tried to troubleshoot the started_script.sh, and found out that the it does indeed create the /var/spool/cron/crontabs directory, but beyond that, it does nothing. Therefore, I think the script stops at the
    Login=`nvram get http_username`
    and so on.

Leave a Reply

Looking for something?

Use the form below to search the site:

Still not finding what you're looking for? Drop a comment on a post or contact us so we can take care of it!

Visit our friends!

A few highly recommended friends...