Today at Browserling I was analyzing how to lower the costs for the Rackspace cloud servers. I noticed that the monthly outgoing traffic bill was higher than expected so I decided to find out which servers used the most traffic.

I opened a chat session with Rackspace's fanatical support team and asked where I could find such information. Unfortunately they didn't have outgoing bandwidth summary for each server. They did however direct me to a CSV file with thousands of lines of entries detailing every charge made to the account, including bandwidth.

I used my Perl one-liner skills and within a few minutes had the breakdown of bandwidth usage for every server. In this article I'll recreate the one-liner and turn it into a Perl program that you can use to create the traffic breakdown yourself.

CSV File

You can find the CSV file if you go to Billing and Payments, and then click on an invoice number.


The CVS file detailing every charge is located on the invoice page.

The file contains thousands of lines detailing every charge, including bandwidth, storage, and cloud servers usage. You'll need to filter the lines that contain the bandwidth charges. A Perl one-liner can easily filter the necessary lines and sum the values for every server.

Perl One-Liner

There are a dozen columns in the file separated by a comma. You can either use a proper CSV parsing module or you can write a quick hack, split the fields on the comma, sum things up, get things done and move on. That's what I did. I love getting things done.

The sixth column in the CSV file describes the charge type. For the outgoing bandwidth this field is either Legacy Server BWOUT for First generation servers or NG Server BWOUT for Next generation servers.

The tenth column shows the amount of traffic used by the server, and the eighteenth column contains the server name.

All the one-liner has to do is create a hash with the eighteenth column as the key and sum the traffic values in the tenth column. After the file has been processed all you need to do is sort the hash by values and print the key, value pairs.

perl -F, -lane '
  if ($F[5] =~ /Server BWOUT/) {
    $h{$F[17]} += $F[9]
  }
  END {
    print "$_: $h{$_}" for sort { $h{$b} <=> $h{$a} } keys %h
  }
' file.csv  

Here's the output it produces:

server1: 101.618118314453
server2: 94.1890354394532
server3: 91.4781017597657
...

I now knew which servers exactly used the most bandwidth and I could proceed to analyzing why they were sending out so much data.

Perl Program For Traffic Breakdown

One-liners are quick throw-away hacks to get the things done and move on. They don't make good programs. I rewrote the Perl one-liner in modern Perl and used a proper CSV module to parse the file.

use warnings;
use strict;

use Text::CSV;

# Rackspace Invoice CSV columns
use constant {
    ACCOUNT_NO => 0,
    BILL_NO => 1,
    BILL_START_DATE => 2,
    BILL_END_DATE => 3,
    SERVICE_TYPE => 4,
    EVENT_TYPE => 5,
    EVENT_START_DATE => 6,
    EVENT_END_DATE => 7,
    IMPACT_TYPE => 8,
    QUANTITY => 9,
    UOM => 10,
    RATE => 11,
    AMOUNT => 12,
    USAGE_RECORD_ID => 13,
    DC_ID => 14,
    REGION_ID => 15,
    RES_ID => 16,
    RES_NAME => 17,
    ATTRIBUTE_1 => 18,
    ATTRIBUTE_2 => 19,
    ATTRIBUTE_3 => 20
};

my $csvFile = shift or die "usage: $0 <csv file>";
my $csv = Text::CSV->new;

open my $file, '<', $csvFile or die "unable to open $csvFile: $!";

my %bandwidth;
while (my $row = $csv->getline($file)) {
    if ($row->[EVENT_TYPE] =~ /Server BWOUT/) {
        $bandwidth{$row->[RES_NAME]} += $row->[QUANTITY];
    }
}

for my $server (sort { $bandwidth{$b} <=> $bandwidth{$a} } keys %bandwidth) {
    printf "%s: %.02f\n", $server, $bandwidth{$server}
}

You can contribute to the project at github: rackspace-traffic.

I also created a profile at Rackspace community and I'll repost this article there. It looks like a fun place to hang out. Rackers are great people.

Until next time!

Hello everyone! I just wanted to share this new feature that I just added to Browserling. It's an URL scheme that lets you quickly access any website in any browser version.

Here's what the format for this URL scheme looks like:

http://www.browserling.com/browse/BROWSER/VERSION/URL

Let's say you want to view Hacker News in Internet Explorer 9. You then just go to:

http://www.browserling.com/browse/ie/9/http://news.ycombinator.com

Or in Firefox 3.5 (really old one):

http://www.browserling.com/browse/firefox/3.5/http://news.ycombinator.com

Or in Chrome 37:

http://www.browserling.com/browse/chrome/37/http://news.ycombinator.com

Supported browsers are ie, chrome, firefox, safari, and opera. Here's a list of all supported browser versions:

I love this feature myself. It makes it so easy to share a browser.

Here's a typical scenario - you install software from source, spend an hour figuring out the configure options, and finally run:

./configure --prefix=/a/b/c --with-X --with-Y \
  --with-Z-dir=/usr/local/Z --enable-A --disable-B

A year passes by, a new software version has come out, and you want to upgrade. To do that you need to run configure again on the new source code. However you've forgotten what the configure flags were because it's been a year. Luckily you still have the old build somewhere in your home directory.

Here's what you do. You go into the old build directory and run this:

./config.status --config

This will output the exact configure arguments that you used to configure the software. You just saved yourself an hour of work trying to remember all the arguments.

Check out this awesome trick that I just discovered. You can use console.table to format data in a nice table in Chrome. Open Chrome developer tools (press F12). Then create an array of hashes of data. The hash keys will be used as table columns, and the values will be used as table rows.

Here's a simple example. Let's simply enumerate the alphabet, convert it to upper case, and also convert it to hex. Copy this to developer console:

var data = [];
var alphabet = "abcdefghijklmnopqrsuvwxyz".split('');
alphabet.forEach(function (char) {
  data.push({
    lowerCase : char,
    upperCase : char.toUpperCase(),
    hex       : "0x" + char.charCodeAt(0).toString(16)
  });
});

console.table(data);

Here's the result:


This is the table that Chrome draws when you run console.table

Apparently console.table has been in Chrome since February 2013 but I only discovered it now. Next time I need a table, I'll just dump data in Chrome's developer console and create a screenshot.

In bash you can place redirections almost anywhere in the command. All these are equivalent:

$ > file command arg1 arg2 arg3
$ command > file arg1 arg2 arg3
$ command arg1 > file arg2 arg3
$ command arg1 arg2 > file arg3
$ command arg1 arg2 arg3 > file

Try it yourself:

$ > file echo hello
$ echo > file hello
$ echo hello > file

Pretty cool, huh?