So I participated in the 2nd annual 48-hour Node.JS Knockout competition together with James Halliday, Joshua Holbrook, and David Wee. Almost the same team as the last year. This year we called ourselves Replicants and we created a real-time code coverage heatmapping application called Heatwave!

If you like our app, please give us thumbs up, that will really help us out. You can vote for our app at Replicants team page.

Just for the record, here is a screenshot of our application (try it here):

So you can either paste the code snippet right on the site and run it and it will show live code heatmap as it runs.

Or you can upload your code via a web form and it will get stored on heatwave server and you'll get a unique url with your heatwave that you can share. Like this:

You can also use curl to upload your code. That's the smartest hackery I have seen. Idea by James Halliday. You can just do:

curl -sNT file.js heatwave.nodejitsu.com

And that will upload the code to the heatwave server and respond with info on how to see it. Like this:

$ curl -sNT foo.js heatwave.nodejitsu.com
Visit this site to run and manage the code:
    http://heatwave.nodejitsu.com/id/09306fa0

 To upload more files:
    curl -sNT file.js heatwave.nodejitsu.com/id/09306fa0/file.js

And with curl you can even upload multiple files to the same page, which is super neat.

We had really great team work. Josh, James and David hacked from Joyent and I hacked remotely from Latvia, and we communicated over IRC, just like the last year. Each of us had a separate github repo and we'd just pull from each other every now and then. That's about it.

The source code of heatwave is on github: heatwave source. Enjoy!

Looking forward to Node.js Knockout 2012!

Excellent news everyone! Last month we launched SSH tunnels for Browserling. SSH tunnels allow you to tunnel your localhost or local network straight through to Browserling, which means you can do cross-browser testing from your internal network!

I made a demo video about this awesome feature:

SSH tunnels is the first major feature that differentiates the paid plans from the free plans. In free plans you get 5 minutes of Browserling for free and no tunnels. With a paid plan you get tunnels and unlimited time.

Here is a brief technical overview of how the tunnels work.

We run the tunnel.browserling.com openssh server. The first time you use tunnels, you'll be asked to choose your ssh password and a new no-login Unix user will be created for you on the tunnel.browserling.com server. The server has ports 50000-60000 open for tunneling but they are firewalled with iptables. When you click the "open tunnel" button in the Browserling UI, the tunnel server generates a random port in this range and opens it up with iptables but only for your Browserling session. For example, it may open up tunnel.browserling.com:55555. Then it generates the ssh command for opening a reverse ssh tunnel for you. For example, if you're tunneling localhost:80 then the command that will be generated will be ssh -N -R 55555:localhost:80 your_login@tunnel.browserling.com. Now you can just copy and paste this command to the terminal, you'll get prompted for your password, and you're done. The tunnel between Browserling and your localhost:80 has been opened. Now if you visit http://tunnel.browserling.com:55555 inside of Browserling, the connection will go through the tunnel and you'll really be accessing localhost:80!

If you're on Windows, you can also easily tunnel your localhost or local network with the plink.exe program from PuTTY. It turns out that the command line arguments for plink.exe are exactly the same as for ssh. In the example above it would be plink.exe -N -R 55555:localhost:80 your_login@tunnel.browserling.com. Really cool!

We are huge fans of open-source at Browserling and we have open-sourced 40 node.js modules! I'll do a blog post about that soon. Then we are releasing Testling the next two weeks, which is an automated web testing framework for Browserling, and I am going to announce it here also. And we're adding a ton more web browsers to Browserling in the upcoming week!

If that sounds interesting, you can subscribe to my blog, follow me on twitter, or Google+. That way you'll be the first to know when we release all this goodness!

Never heard of Browserling? Read the Browserling announcement blog post, and the blog post on How I went to Silicon Valley and raised $55k seed funding for it!

Browserling is an interactive cross-browser testing site that allows you to use Internet Explorer, Firefox, Chrome, Opera and Safari from your browser. We built this amazing technology that brings virtual machines to the web and we built Browserling on top of it!

A Year of BloggingHey everyone! Another year has passed and it's now 4 years since I've been blogging here on catonmat! In this post I wish to summarize this year's statistics.

See the one year of blogging, two years of blogging and three years of blogging for the previous year statistics.

Let's start this year's stats with traffic:


Traffic statistics from Google Analytics for Jul 1, 2010 - Jul 1, 2011.

So catonmat had 1 million visitors during the last 12 months. That's 83,000 visitors per month or 2700 visitors per day. These visitors have generated 1.45 million page views, which is 120,000 page views per month, or 4000 page views per day. That's a lot less than the last year because I am now doing a startup and I don't have that much time to blog. I wrote just 16 posts last year, compared to 43 articles that I did two years ago.

Here is Feedburner statistics. I used my Feedburner graph generator to produce it.


Feedburner subscriber statistics for Jul 15, 2007 - Jul 1, 2011.

Last year I said, "if it keeps going the same way, I expect to have around 17,000 subscribers the next year." Well this didn't come true, and I have gotten only up to 13,500 subscribers. Still cool.

I am also up to 686 followers at GitHub and 1998 followers on Twitter. I didn't keep metrics of this last year and the year before but now I will. Now who's going to be my 700th follower on GitHub and who's going to be 2000th follower on twitter? :)

Next, top 10 countries my visitors came from,


Country statistics from Google Analytics for Jul 1, 2010 - Jul 1, 2011.

USA is #1 with 38% of the traffic, then India with 7.5%, then United Kingdom with 5.9%, then Canada with 4.6% and Germany with 4.3%. Poland, France, Australia, Russia and China take the bottom top 10 with less than 4.3%.

Since I wrote only 16 posts during the last year, I can list them all here, sorted by views:

Now let's have some 4 year birthday cake,

And let's meet for the cake next year again! See you!

Long story short, I went to Silicon Valley and raised $55,000 seed funding for my Browserling startup that I am doing together with James Halliday (SubStack).


James Halliday, our lead investor and advisor David Weekly, Peteris Krumins

Here is how I did it.

Last year in March, James offered me to do a startup. He had this idea of putting virtual machines on the web. I knew he was really good at functional programming and the idea was solid, and I had been wanting to do a startup for years, so it didn't take me long to agree to do a startup with him. James already had a prototype in Haskell but it was too slow, so I rewrote it in node.js and wrote several C++ node.js modules for fast encoding. In July we had a solid working prototype and I announced on my blog that I am doing the StackVM startup. We had decided to apply to Winter 2010 YC, so we hacked some more, and applied to YC in October. I was super sure we'll get accepted as I had done various interesting projects in the past for reddit, digg (this project had 100,000 visitors in 20 hours), hacker news, I had over 50 unique projects on github and my programming articles were getting millions of views. I bought a ticket to USA on October 20, before the results were even announced. Meanwhile, James took a car trip from Alaska, where he was residing at the time, to Oakland. On October 27 I took a flight from Riga, where I live, to Frankfurt and then took a flight from Frankfurt to SFO and arrived there at around 9pm on October 28. Then I took BART to Oakland and met James for the first time. He was a good guy, so everything was fine.

The YC results were to be announced on November 2. November 2 came and to our horrors they decided to pass on us and we didn't even get an interview. Here is our YC application and 2 minute YC video if anyone is interested. The video isn't great but it was like the 100th time we tried to record it and we were out of energy to do more videos so we just went with what we had at 100th take.

Fortunately for us, I had saved some money from all my previous jobs and my projects and decided to put $15,000 into our startup myself. I had been willing to do a real startup for years so I was fine with putting $15,000 in and we just started coding and not thinking about much else.

James had heard about Hackers and Founders meetup that's organized by Jonathan Nelson and Laura Nelson, so we decided to attend all the Hackers & Founders meetups. When I first met Jonathan I was really surprised cause he knew me already from this blog! He had used my xgoogle Python library for scraping Google search results for his Newsley startup. That was thrilling! The goal of this blog from the beginning was to get me well known and my blogging efforts had paid off. Success, I was known in the Silicon Valley! So we'd continue to go to Hackers & Founders meetups in SF and Mountain View, and we'd show everyone we could what we had made, and things were going really well. People loved what we were doing and we didn't worry much about anything, we'd just code and improve Browserling. We launched beta on November 24th and added paid plans in January. The moment we launched the paid plans we got our first customer, and then a few more, and then a few more, and by the end of January we had 10 paying customers already! Things were looking fantastic, but the time was ticking and the $15,000 was running out. It was now February and we had just a few $k left, so we decided it was time to raise money. At one of the Hackers & Founders meetups Jonathan introduced us to Adam Rifkin and he was really excited about StackVM and Browserling. We told Adam that we were looking for a small investment of $50,000 that would keep us going, and he introduced us to David Weekly. So we went and met with David Weekly and he was also really excited about Browserling. He was actually so excited that he wrote a check for us on the spot! Haha, how awesome is that! That's the Silicon Valley I had read about! A check on the spot!

David Weekly decided to lead the investment round for us and help us structure the form of investment. We went with a convertible note, that's the best form of investment. He also introduced us to his friend Ulrich Gall. We took a trip to meet Ulrich Gall, talked for a few hours and we had another committed investor. We actually didn't have phones so he gave me a HTC Nexus One that he had gotten at one of TED conferences. Awesome! Now we were connected, too.

David also got his girlfriend Rebecca Lipon to invest. Meanwhile Adam introduced us to Nick Heyman, who's friends with Mike Tsao. We met with Nick and Mike, and they agreed to invest as well. Mike loved that we had paying customers already.

Now we had 5 investors but we didn't yet have a company! So we had to incorporate. We met with several lawyers from Orrick, Feynwick & West, and some more, and we liked the lawyer from Fenwick & West the most, so we went with him. A few days later we incorporated as Browserling INC. Then we took another trip to meet David Weekly and we signed the investment documents with him and Ulrich. On the same day David Weekly also interviewed us on his The David Weekly podcast. We were super excited.

Next we setup our advisors. David Weekly, Jonathan Nelson, and Adam Rifkin were the most helpful and the awesome people we had met so we set them up as our advisors!

At about the same time we also joined Jonathan and Laura Nelson's new startup incubator called Co-Op. We had discussed this with Jonathan before and we had said that if he ever did an incubator that we'd join in right away. And we did. We are the Reddit of Jonathan's incubator. Also it's funny how at first we didn't know what to call the incubator, so we called it secret super ninja turtles meetup. That's what I still call it when talking with James. :)

That's the story. Now we're adding all kinds of cool features to Browserling. Just last week we launched tunnels for Browserling that allow you to access your localhost or local network from Browserling, and we're launching Testling soon, that's an automated test framework for Browserling.

I have a dozen more blog posts prepared about Browserling. Such as how we open-sourced 40 node.js libraries, how open sourcing them made Browserling easy to maintain, how Browserling works internally, details of new features such as tunnels, resolution changing, testling. If you wish to hear more about it, I suggest that you subscribe to my blog and/or follow me on twitter!

This article is part of the article series "Awk One-Liners Explained."
<- previous article next article ->

Hello everyone! I have awesome news - I have written my first e-book ever! It's called "Awk One-Liners Explained" and it's based on the "Famous Awk One-Liners Explained" article series that I wrote here on my blog a few years ago and that has been read over 800,000 times!

I went through all the one-liners in the article series, improved the explanations, fixed the mistakes, added an introduction chapter to Awk one-liners, and two new chapters on the most commonly used Awk special variables and on idiomatic Awk.

Table of Contents

The e-book is 58 pages long and contains exactly 70 well-explained Awk one-liners. It's divided into the following chapters:

  • Preface.
  • 1. Introduction to Awk One-Liners.
  • 2. Line Spacing.
  • 3. Numbering and Calculations.
  • 4. Text Conversion and Substitution.
  • 5. Selective Printing and Deleting of Certain Lines.
  • 6. String and Array Creation.
  • Appendix A: Awk Special Variables.
  • Appendix B: Idiomatic Awk.
  • Index.

What is awk?

Awk is this awesome, little program that's present on nearly ever Unix machine. It's designed to carry out various text processing tasks easily, such as numbering lines, replacing certain words, deleting and printing certain lines.

Let's take a look at several examples.

Example 1: Print the second column from a file

awk '{ print $2 }'

That's all there is to it. Awk automatically splits each line into columns and puts each column in variables $1, $2, $3, etc. This one-liner prints just the 2nd column, which is in variable $2.

You can also specify the symbol or word that you wish to split on with the -F command line switch. This switch is explained in more details in the e-book and in the last example below.

Example 2: Number lines in a file

awk '{ print NR ": " $0 }' file

The whole line itself goes into variable $0. This one-liner prints it but prepends the NR special variable and a colon ": " before it. The special variable NR always contains the current line number.

There are many other special variables and they're all explained in the e-book and summarized in the appendix.

Example 3: Count the number of words in a file

awk '{ total = total + NF } END { print total }'

Here another special variable is used. It's the NF that stands for number of fields, or number of columns, or number of words in the current line. This one-liner then just sums the total number of words up and prints them before quitting in the END block.

Example 4: Print only lines shorter than 64 characters

awk 'length < 64'

This one-liner uses the length function to determine the length of the current line. If the current line is less than 64 characters in length, then length < 64 evaluates to true that instructs awk to print the line.

Finally, let's take a look at an example that compares an Awk program with an equivalent C program. Suppose you want to print the list of all users on your system. With Awk it's as simple as this one-liner:

awk -F: '{ print $1 }' /etc/passwd

This one-liner says, "Take each line from /etc/passwd, split it on the colon and print the first field of each line." Very straightforward and easy to write once you know Awk!

Suppose you didn't know Awk. Then you'd have to write it in some other language, such as C. Compare the example above with the example in C language:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define MAX_LINE_LEN 1024

int main () {
    char line [MAX_LINE_LEN];
    FILE *in = fopen ("/etc/passwd", "r");
    if (!in) exit (EXIT_FAILURE);
    while (fgets(line, MAX_LINE_LEN, in) != NULL) {
        char *sep = strchr(line , ':');
        if (!sep) exit (EXIT_FAILURE);
        *sep = '\0';
        printf("%s\n", line);
    }
    fclose(in);
    return EXIT_SUCCESS ;
}

This is much longer and you have to compile the program first, only then you can run it. If you make any mistakes, you have to recompile it again. That's why one-liners are called one-liners! They are short, easy to write and they do one thing really well. I am pretty sure you're starting to see how mastering Awk and one-liners can make you much more efficient when working in the shell and with text files in general.

Once you read the e-book and work through the examples, you'll be able to solve the most common text processing tasks, such as joining lines in a file, numbering lines, replacing certain words and printing certain lines.

Book preview

I prepared a book preview that contains the first 11 pages of the book. It includes the table of contents, preface, introduction and the first page of the second chapter.

Buy it now!

The price of the e-book is just $5.95 and you can buy it through PayPal.

PayPal - The safer, easier way to pay online!

After you have made the payment, my automated e-book processing system will send the PDF e-book to you in a few minutes!

Testimonials

Iain Dooley, CEO and founder of Working Software LTD:

It never ceases to amaze me that, even though I spend 50% - 70% of my day using a *nix command line and have done so for the past 6 years, there are still countless thousands of useful tools and tips to learn, each with their corresponding little productivity boosts. The trouble, of course, is finding the time to organise and prioritise them, deciding which are the most important and useful to learn. "Awk One Liners Explained" is a fantastic resource of curated examples that helped me rapidly pick up a few cool tricks that have already provided many times the value I initially paid for the book. Any professional who spends time working with *nix systems can benefit from this book.

Tweet about my book!

I am really excited about my book and I would appreciate your help spreading the word via Twitter! Here is a quick link for tweeting:

My other e-books!

I am so passionate about programming and writing about programming that I have now written my second e-book called "Sed One-Liners Explained". It's written in the same style as this e-book and it explains sed, the Superman of Unix stream editing. Sed One-Liners Explained contains 100 well-explained one-liners and it's 98 pages long. Take a look!

And I am not stopping here - I am going to release several other books. My next e-book is called "Perl One-Liners Explained" and it's based on my "Famous Perl One-Liners Explained" article series.

Enjoy!

Enjoy the book and let me know what you think about it!

Sincerely,
Peteris Krumins (@pkrumins on Twitter)