Here's another short and interesting story about problems we have to deal at Browserling. This story is about getting rid of annoying pop-up dialogs in browsers, such as alerts, error dialogs and add-on update dialogs.

So this one time when we were building Testling we started getting a nasty modal dialog in Chrome that paralyzed the browser and the tests wouldn't run. It said:

Your profile can not be used because it is from a newer version of
Google Chrome.

Some features may be unavailable. Please specify a different profile
directory or use a newer version of Chrome.

Here's how that dialog looked in Chrome:


Your profile can not be used because it is from a newer version of Google Chrome.

We were isolating each browser already through Sandboxie so changing the profile directory wasn't really a solution. I decided to write a quick hack that would simply click the OK button on this dialog.

I used Spy++ to find the window class of the dialog:


Spy++ in action.

And then I wrote a Win32 C++ program that detects if this dialog is present, and if it is, it clicks the OK button to get rid of it:

#include <windows.h>
#include <cstdio>

int WINAPI
WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow)
{
    while (1) {
        HWND dialog = FindWindow("#32770", "Google Chrome");
        if (dialog) {
            printf("Found dialog: %x\n", dialog);
            HWND dialogItem = NULL;
            while (1) {
                dialogItem = FindWindowEx(dialog, dialogItem, NULL, NULL);
                if (dialogItem == NULL) {
                    break;
                }
                char windowText[255];
                GetWindowText(dialogItem, windowText, 255);
                if (strcmp(windowText, "OK") == 0) {
                    SetActiveWindow(dialog);
                    SendMessage(dialogItem, BM_CLICK, NULL, NULL);
                    break;
                }
            }
        }
        Sleep(1000);
    }
}

I pushed this program to github and called it chrome-dialog-killer.

Here's another similar problem that we faced in Browserling. We run many different FireFox versions on the same box and often FireFox wouldn't start cleanly. Instead it would complain about incompatible add-ons:

The following add-ons are not compatible with this version of
Firefox and have been disabled.
...
Firefox can check if there are compatible versions of these
add-ons available.

The incompatible add-ons dialog looked like this:


Incompatible add-ons FireFox dialog.

I tried disabling this add-on in all the FireFox version (Firefox 3 to Firefox 25) but we'd still get this alert sometimes. The browser wouldn't start and users had to click the "Don't Check" button. So, again, I used Spy++, found the dialog's windows class and wrote an even simpler program that simply sends the WM_CLOSE message to the dialog that closes it:

#include <windows.h>
#include <cstdio>

int WINAPI
WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nCmdShow)
{
    while (1) {
        HWND dialog = FindWindow("MozillaDialogClass", "FireFox Update");
        if (dialog) {
            printf("Found dialog: %x\n", dialog);
            HWND dialogItem = NULL;
            SendMessage(dialog, WM_CLOSE, 0, 0);
        }
        Sleep(2000);
    }
}

I also pushed this program to github and called it firefox-update-dialog-killer.

Similarly we can get rid of alert() dialogs in Testling as they're unnecessary and halt the JavaScript execution.

Modern GUIs and interfaces should never use modal dialogs as they provide really unpleasant user experience and make automating things difficult. Until next time.

I thought I'd write this short article about how we fixed a nasty bug at Browserling together with my friend James Halliday yesterday.

Long story short we were testing Browserling in the latest FireFox and we got this nasty exception that said:

uncaught exception: [object Object]

Here's how it looked in Firebug:


uncaught exception: [object Object]

FireBug didn't show the line number where it was thrown from and we couldn't find a way how to make it show the stack trace. We tried stepping through the code and setting breakpoints at some potential locations but didn't get us anywhere.

We were stuck and annoyed.

So we came up with a very hacky Perl one-liner that appends a console.log to almost every function definition:

perl -i -pe 's/function.*{$/$&."\n console.log($.);"/e' bundle.js

What this one-liner does is it finds all lines that have string function in them and that end with { and appends a console.log with the current line number to the beginning of the function.

For example, this code piece:

function foo (x) { 
   return 50/x;
}

Gets rewritten as:

function foo (x) { 
console.log(1);
   return 50/x;
}

Our thinking was that we should get the line number of the last function call before the exception was thrown or some function call nearby.

And it worked!

We refreshed Browserling and we got an awesome trace with lots of line numbers followed by the same uncaught exception error:

...
11248
11157
11257
11147
uncaught exception: [object Object]

Here's how it looked in FireBug:


lots of console.log's with line numbers followed by uncaught exception error

Next we checked the line 11147 in the original file (line 11914 in the updated file because the one-liner offsets line numbers) and we found the exception that we were looking for:

error = function (m) {                                                                                                 
    // Call error when something is wrong.
    throw {
        name:    'SyntaxError',
        message: m,
        at:      at,
        text:    text
    };
},

A fix quickly followed and we got Browserling working in the latest FireFox! Until next time.

We just launched Internet Explorer 11 at Browserling! IE 11 is available to all our paying customers. The free Browserling version offers IE 9. If you want to try out IE 11 for free, let me know in the comments and I'll put you in a special IE 11 user group. :)


Internet Explorer 11.0.9600.16428IS

Follow us on twitter for updates - @browserling, @testling, @substack, @pkrumins.


Try Browserling!

Browserling is an interactive cross-browser testing tool. It lets you test your website in older Internet Explorers such as IE 6, IE 7, IE 8 and IE 9, IE 10, IE 11, Firefox, Chrome, Opera and Safari!

I was just interviewed by David Farrell from Perl Tricks. We talked about Perl one-liners, my books, Perl on Windows, text editors and vim plugins. I created a backup copy of the interview here on my blog. Here's the original.

You're a really busy guy; running catonmat, Browserling and publishing books. What's your secret?

I just work like crazy. It's pretty much all I do! I'm a list person; one of my recent posts on catonmat was about how I work: I use Google Calendar to track the most important tasks; interviews, payments, important events, that kind of thing. Then I also use to-do lists, actually several to-do lists for each one of my projects - Browserling, Testling, books, blog posts, a reading list, stuff like that. Here is an example.

We're talking because you're famous for writing one liners in Perl. How did you get into that?

So there was this guy, Eric Pement and he collected hundreds of Awk one liners in this file awk1line.txt and he published it on UseNet, like 10 years ago. So I found his file and it was really interesting. I went through all of his one liners and learned Awk. Then I found that he had done the same thing for Sed (sed1line.txt) and I went through that and learned Sed. So that inspired me to create my own file for Perl, perl1line.txt and it started from there.

I read that that your post "Perl One Liners Explained" has something like 500,000 hits?

It's an article series made out of eight separate posts. It's actually more like 800,000 now for all posts combined.

Wow! Was that an outlier for you, did it make you think there was a lot of interest in this topic?

Well I had already written perl1line.txt by then. I wrote eight blog posts following the structure used by Eric Pement in his files; for example he had a section called "How to do line spacing in Awk", so I wrote one post called "How to do line spacing with Perl one liners" and so on.

Your Perl One Liners book was originally self-published, then picked up by No Starch Press. How did that happen?

It was amazing. I just got an email from Bill, who runs No Starch Press and he asked me if I wanted to turn my self-published book into a real book. And I said "yeah!" and we got it published. Here's the result - Perl One Liners book.

The new No Starch Press version has 30+ more pages of content, did No Starch help you with that?

Yes, we took the original text and improved it mainly by adding more examples. We added examples for most of the one liners. And we also added a section about running Perl on Windows.

I wanted to ask you about that. That seemed like a topic that hasn't been well-covered elsewhere?

It was really hard to write. I had to test all of the one liners on PowerShell and the command line (cmd.exe) to make sure they worked, find all of the work-arounds, handle special symbols, it was very challenging.

Was that a section that No Starch Press encouraged?

Not really. I run Windows as well, it's my primary workstation although I do have a bunch of Linux servers which I ssh into. Sometimes I need to run Perl one liners on Windows, so I thought a lot of my readers would be interested in how to run the one liners on Windows as well. We spent about a month on that section, it delayed the book for about a month and a half, because of it. It was very challenging and hard to write but it should help a lot of Windows users as it's probably the best guide to running Perl from the command line on Windows.

Would you still use Unix tools over PowerShell on a Windows?

Well, I run Windows XP and don't have PowerShell. I use Cygwin, but sometimes when I don't want to use that, I just run bash.exe from win-bash. My setup is that I have Linux server mounted through Samba as a virtual drive. So if I have to do serious editing, I throw the file onto the shared drive and shh into my Linux server, so I don't need to use the Linux tools on Windows that much.

Your book is full of interesting command line shortcuts and hidden features. Do you have a favourite one liner or code trick in Perl?

I love one liners that are like puzzles - you can't understand them just by looking at them. You have to tinker around and try them out to see what it does. For example a regular expression like: "/[ -~]/" is fun, which matches every printable ASCII character from space to tilde.

Another one liner which I like because it has no code in it is: "perl -00pe0", for paragraph slurping.

So how did you find out about that? I don't think it's even documented in perlrun.

I don't know how I came up with this one. I remember something about being on the #perl freenode IRC channel. When I was writing the book I was often on there asking people for advice. Maybe someone told me about it or I found it myself, but it's fun.

You've also written about Sed and Awk. If you know Perl one liners, do you need to learn Sed or Awk as well?

No, you don't. But sometimes when you write a Perl one liner the equivalent one liner in Awk / Sed would be shorter. For example to reference the fifth word on a line in Awk it's "$5" but in Perl it's "$F[4]", besides you'd have to turn on autosplitting and use a bunch of other command line arguments.

What text editor / IDE do you use to code in Perl?

I use Vim. I have a bunch of customizations and shortcuts that I use. I have an article series on my site called "Vim plugins you should know about" that covers this. One of my favourite plugins is "surround.vim" - if you have a single-quoted string and want to change them to double-quotes, you type: cs'" and it will change them. You can change parentheses and many other things that surround something. Another plugin I like is "matchit.vim", which extends the shift + F5 (%) parenthesis matching in Vim to match HTML tags and if/then/else statements and other constructs. Some of my other favorites are "snipmate.vim" for code snippets, "ragtag.vim" for working with HTML tags and "nerd_tree.vim" for browsing files in vim.

What about color schemes, do you have a favourite?

I don't really care about color schemes - Vim does color the code but I don't change it. I also don't care about programming fonts. The font just have to be constant width and that's it.

You've also written about EMACS before. Why do you stick with Vim over other tools?

I was working at this company once and I thought that from day one I would use EMACS (as a way of learning it) and see how it goes. After a few weeks I went back to Vim. I had to learn so many new key combinations it was hurting my productivity.

When I'm programming in C and C++, I use Visual Studio. The best thing about using Visual Studio is if I forget a command or method, I can quickly look it up using IntelliSense.

Apart from one liners, do you develop Perl applications or modules?

Perl is my preferred programming language for writing quick tools, for example: uploading / downloading videos from YouTube, or parsing HTML pages. I'm incredibly fast at that and there are so many modules that can help - I wouldn't be able to be that productive in other languages.

Have you used any Perl code analysis tools like Perl::Critic?

I have used Perl::Critic but I don't like it when someone criticizes my code! (even if it's from Damian Conway's Perl best practices book!) I just write sane code and apply most of the best practices. Other people usually don't have problems with my code.

You're a real polyglot as apart from Perl, your Github page hosts projects in C++, JavaScript, Python, OCaml, PHP! Are there any features of those languages would you like to see in Perl?

It's hard to take a feature from another language and put it into Perl as it's already very expressive and supports many different programming paradigms. I often get asked this, but I don't have a good answer! I'm very productive with Perl as it is, whereas I could definitely name features of Perl that I would like to see in other languages.

What about Perl versions, do you keep up to date?

Yeah, right now I'm using Perl 5.18 and that's all thanks to Perlbrew. I remember a few years ago before I knew about Perlbrew it was huge pain to have several different Perl versions and I always had the system Perl and maybe one local version of Perl. Managing packages was a pain, too. Once I found out about Perlbrew, I installed every version of Perl, going back to 5.6. One cool feature is "perlbrew exec" which I used to test my one liners against every version of Perl to see which ones work and don't.

What are your favourite Perl modules and tools?

I really like Try::Tiny for better exception handling, File::Slurp for quickly working with files, WWW::Mechanize and HTML::TreeBuilder for scraping the web.

Talking about tools I also like rxrx by Damian Conway (here's a demo at YAPC). It's this interactive regexp debugger. Another tool that I use is ack that's a better version of grep!

And one last final question - what is it that you're doing in your profile picture? Making a cigarette?

Haha, no! I'm actually holding a piece of scotch tape. I was making a raft that day from empty bottles so I was taping them together. It turned out to be a really good raft and it worked. Here's a photo.

Buy from No Starch Press
Use discount code CATONMAT for 30% off!

Also available at:
Amazon
Amazon Kindle
O'Reilly
Powell's
Barnes and Noble

I've fantastic news - I've become a published author! No Starch Press just published my book Perl One-Liners!

Some programs are powerful because of their thousands of lines of code, and some are powerful because of what they get done with so little. Perl is a language that harnesses the power of minimal code, letting users automate otherwise mind-numbing tasks with a single line of code.

In Perl One-Liners, I take you through more than 100 compelling one-liners that do all sorts of handy and geeky things, such as manipulate line spacing, tally column values in a table, and get a list of users on a system. This cookbook of useful, customizable, and fun scripts will even help hone your Perl coding skills, as I dissect the code to give you a deeper understanding of the language.

Table of Contents

  • Acknowledgments
  • Appendix A: Perl's Special Variables
  • Appendix B: Using Perl One-Liners on Windows
  • Appendix C: perl1line.txt
  • Index (PDF)

View the detailed Table of Contents (PDF)

What are Perl One-Liners?

Perl one-liners are small and awesome Perl programs that fit in a single line of code. They do one thing really well—like changing line spacing, numbering lines, performing calculations, converting and substituting text, deleting and printing specific lines, parsing logs, editing files in-place, calculating statistics, carrying out system administration tasks, or updating a bunch of files at once. Perl one-liners will make you a shell warrior: what took you minutes (or even hours) to solve will now take you only seconds!

Let's take a look at several Perl one-liners to see how short and powerful they are. All these one-liners are explained in greater detail in the book.

Generate a random eight-letter password

perl -le 'print map { ("a".."z")[rand 26] } 1..8'

This one-liner generates and prints a random eight-character password. It uses the list range operator .. to produce all alphabet letters from a to z (for a total of 26 letters). Then a random letter is chosen by rand 26 and this operation is repeated 8 times to produce an eight-letter password.

Sum numbers in each line

perl -MList::Util=sum -lane 'print sum @F'

This one-liner uses the sum function from the List::Util module that comes with Perl. The -a command line argument enables the automatic splitting of the current line into fields in the @F array. The splitting happens on whitespace characters by default. The -l argument ensures that print outputs a newline at the end of each line. The -n argument makes sure every line is processed. Finally, sum @F sums all the elements in the @F array, and print prints the result followed by a newline (which I added with the -l argument).

Print all lines from line 17 to line 30

perl -ne 'print if 17..30'

This one-liner uses the binary flip-flop operator ... The operator is bistable, and it works just like the line-range (comma) operator in sed, awk, and various text editors. Its value is false as long as its left operand is false. Once the left operand is true, the range operator is true until the right operand is true, after which the range operator becomes false again. As a result, this bistable operator becomes true at line 17, stays true until the 30 line, and then becomes and remains false. Combining the flip-flop operator with print if condition makes it print only lines 17 to 30.

Finding all lines in a file that appear more than once

perl -ne 'print if $a{$_}++' file

This one-liner records the lines you've seen so far in the %a hash and counts the number of times it has seen the lines. If it has already seen the line, the condition $a{$_}++ is true, so it prints the line. Otherwise it automatically creates an element that contains the current line in the %a hash and increments its value.

Create 1KB of data

perl -e 'print "a"x1024'

This one-liner repeats the letter "a" 1024 times, creating 1KB of data. Operator x is the repetition operator. This one-liner is handy when you need to generate a specific amount of data for debugging or other tasks.

As you can see, knowing how to write one-liners is very useful. Through the years, it's been one of my top priority tasks to become very efficient in the shell. Literally every day when I'm programming, I've to do all kinds of data processing tasks, changing files, verifying output, doing quick calculations, parsing data, etc. Knowing Perl one-liners makes it really fast to get things done and be really productive. With Perl One-Liners you can now become very efficient at these tasks, too!

Reviews

"By reading this book and trying the recipes, you’ll master shortly all the command-line options that can turn the Perl interpreter into a sed-awk-grep-toolboox-on-steroids."
—Alexis Sukrieh, creator of Perl Dancer (Read More)

“One of the slogans used by Perl is ‘Easy things should be easy and hard things should be possible.’ This book illustrates just how easy things can be—and how much can be done with so little code.”
—David Precious, contributor to the Perl Dancer project and various CPAN modules

“A great reference for when you want to do something with Perl without spending an hour or two trying to figure out how.”
—Sandra Henry-Stocker, IT World (Read More)

“By reading this book you can make a step toward becoming the local computer wizard, even without learning how to program.”
—Gabor Szabo, founder and editor of the Perl Weekly newsletter

“I didn't find anything to yell about. Given how much of a perfectionist bastard I am this is likely high praise.”
—Matt Trout, technical director of Shadowcat Systems

“A set of exercises for deepening your understanding of Perl.”
—John D. Cook, Singular Value Consulting

“The author is enthusiastic about the material and uses an easy writing style. Highly recommended.”
—Thrig (Jeremy Mates), Internet plumber

“These one-liners are great. Simple. Clear. Concise.”
—Jonathan Scott Duff, Perl guru

“A quick read full of useful command-line Perl programs.”
—Chris Fedde, systems engineer and Perl enthusiast

“Handy for anyone who does a lot of one-off text processing: system administrators, coders, or anyone with large amounts of data they need shifted, filtered, or interpreted.”
—Jim Davis, Perl developer

“Any Perl programmers should take the time to get to know the command like switches that make this possible. This book is a pretty good introduction to this way of using Perl.”
—David Cross, Perl Hacker, Trainer and Author (Read More)

“Seriously, folks, CompSci 101 courses might be more fun with a book like this (instead of the standard C++ tome).”
—Thomas Maher (Read More)

“Perl One-liners is a beautifully written manual, if I can call it like that, that empowers sysadmins with a Swiss Army knife to get dirty jobs done.”
—Nitin K Sookun (Read More)

“This book shows the undisputed king of one-line programs in it's full Swiss-Army-Chainsaw glory.”
—Kieran Barry (Read More)

“By the end of the book one should master enough of Perl code to write beautiful scripts.”
—Ish Sookun (Read More)

“Author pulls out a lot of interesting Perl tricks that someone can learn from. The book is worth reading just for that; it's also a short easy read.”
—John Graham-Cumming (Read More)

“If you are doing the sort of thing Perl excels at, like text mangling, this will be a book full of tools for you.”
—Justin Sherrill, DragonFly BSD (Read More)

“The presentation style is great as all the one liners are explained and alternatives presented, in keeping with the Perl idiom "There is more than one way to do it".”
—Andrew Colin Kissa, Creator of Baruwa, Fedora Linux contributor (Read More)

“Summing up, this is a useful book for admins, shell users and coders alike who need their tools concentrated into one toolbox and not spread here and there. Having the book to hand will allow you to call on Perl's full power in an instant when the need arises. Recommended”
—Nikos Vaggalis (Read More)

Book Preview

I've made Chapters 1 and 4 freely available:

You can also view full index of the book and detailed table of contents.

Buy Perl One-Liners

Perl one-liners is available from No Starch Press (use discount code CATONMAT for 30% off), Amazon, Amazon Kindle, O'Reilly, Powell's and Barnes and Noble.

Tweet about my book!

Help me spread the word about my new book! I prepared a special link that you can use to tweet about it:

What's next?

I really love writing about programming and I'll soon publish several more books. The next few are going to be about becoming skillful in bash (you can already start reading it here!), about the most effective vim tricks, a practical guide on how to be anonymous on the web, and the catonmat book.

If you enjoy my writing, you can subscribe to my blog. You can also follow me on Twitter or Google+.