I was just interviewed by David Farrell from Perl Tricks. We talked about Perl one-liners, my books, Perl on Windows, text editors and vim plugins. I created a backup copy of the interview here on my blog. Here's the original.

You're a really busy guy; running catonmat, Browserling and publishing books. What's your secret?

I just work like crazy. It's pretty much all I do! I'm a list person; one of my recent posts on catonmat was about how I work: I use Google Calendar to track the most important tasks; interviews, payments, important events, that kind of thing. Then I also use to-do lists, actually several to-do lists for each one of my projects - Browserling, Testling, books, blog posts, a reading list, stuff like that. Here is an example.

We're talking because you're famous for writing one liners in Perl. How did you get into that?

So there was this guy, Eric Pement and he collected hundreds of Awk one liners in this file awk1line.txt and he published it on UseNet, like 10 years ago. So I found his file and it was really interesting. I went through all of his one liners and learned Awk. Then I found that he had done the same thing for Sed (sed1line.txt) and I went through that and learned Sed. So that inspired me to create my own file for Perl, perl1line.txt and it started from there.

I read that that your post "Perl One Liners Explained" has something like 500,000 hits?

It's an article series made out of eight separate posts. It's actually more like 800,000 now for all posts combined.

Wow! Was that an outlier for you, did it make you think there was a lot of interest in this topic?

Well I had already written perl1line.txt by then. I wrote eight blog posts following the structure used by Eric Pement in his files; for example he had a section called "How to do line spacing in Awk", so I wrote one post called "How to do line spacing with Perl one liners" and so on.

Your Perl One Liners book was originally self-published, then picked up by No Starch Press. How did that happen?

It was amazing. I just got an email from Bill, who runs No Starch Press and he asked me if I wanted to turn my self-published book into a real book. And I said "yeah!" and we got it published. Here's the result - Perl One Liners book.

The new No Starch Press version has 30+ more pages of content, did No Starch help you with that?

Yes, we took the original text and improved it mainly by adding more examples. We added examples for most of the one liners. And we also added a section about running Perl on Windows.

I wanted to ask you about that. That seemed like a topic that hasn't been well-covered elsewhere?

It was really hard to write. I had to test all of the one liners on PowerShell and the command line (cmd.exe) to make sure they worked, find all of the work-arounds, handle special symbols, it was very challenging.

Was that a section that No Starch Press encouraged?

Not really. I run Windows as well, it's my primary workstation although I do have a bunch of Linux servers which I ssh into. Sometimes I need to run Perl one liners on Windows, so I thought a lot of my readers would be interested in how to run the one liners on Windows as well. We spent about a month on that section, it delayed the book for about a month and a half, because of it. It was very challenging and hard to write but it should help a lot of Windows users as it's probably the best guide to running Perl from the command line on Windows.

Would you still use Unix tools over PowerShell on a Windows?

Well, I run Windows XP and don't have PowerShell. I use Cygwin, but sometimes when I don't want to use that, I just run bash.exe from win-bash. My setup is that I have Linux server mounted through Samba as a virtual drive. So if I have to do serious editing, I throw the file onto the shared drive and shh into my Linux server, so I don't need to use the Linux tools on Windows that much.

Your book is full of interesting command line shortcuts and hidden features. Do you have a favourite one liner or code trick in Perl?

I love one liners that are like puzzles - you can't understand them just by looking at them. You have to tinker around and try them out to see what it does. For example a regular expression like: "/[ -~]/" is fun, which matches every printable ASCII character from space to tilde.

Another one liner which I like because it has no code in it is: "perl -00pe0", for paragraph slurping.

So how did you find out about that? I don't think it's even documented in perlrun.

I don't know how I came up with this one. I remember something about being on the #perl freenode IRC channel. When I was writing the book I was often on there asking people for advice. Maybe someone told me about it or I found it myself, but it's fun.

You've also written about Sed and Awk. If you know Perl one liners, do you need to learn Sed or Awk as well?

No, you don't. But sometimes when you write a Perl one liner the equivalent one liner in Awk / Sed would be shorter. For example to reference the fifth word on a line in Awk it's "$5" but in Perl it's "$F[4]", besides you'd have to turn on autosplitting and use a bunch of other command line arguments.

What text editor / IDE do you use to code in Perl?

I use Vim. I have a bunch of customizations and shortcuts that I use. I have an article series on my site called "Vim plugins you should know about" that covers this. One of my favourite plugins is "surround.vim" - if you have a single-quoted string and want to change them to double-quotes, you type: cs'" and it will change them. You can change parentheses and many other things that surround something. Another plugin I like is "matchit.vim", which extends the shift + F5 (%) parenthesis matching in Vim to match HTML tags and if/then/else statements and other constructs. Some of my other favorites are "snipmate.vim" for code snippets, "ragtag.vim" for working with HTML tags and "nerd_tree.vim" for browsing files in vim.

What about color schemes, do you have a favourite?

I don't really care about color schemes - Vim does color the code but I don't change it. I also don't care about programming fonts. The font just have to be constant width and that's it.

You've also written about EMACS before. Why do you stick with Vim over other tools?

I was working at this company once and I thought that from day one I would use EMACS (as a way of learning it) and see how it goes. After a few weeks I went back to Vim. I had to learn so many new key combinations it was hurting my productivity.

When I'm programming in C and C++, I use Visual Studio. The best thing about using Visual Studio is if I forget a command or method, I can quickly look it up using IntelliSense.

Apart from one liners, do you develop Perl applications or modules?

Perl is my preferred programming language for writing quick tools, for example: uploading / downloading videos from YouTube, or parsing HTML pages. I'm incredibly fast at that and there are so many modules that can help - I wouldn't be able to be that productive in other languages.

Have you used any Perl code analysis tools like Perl::Critic?

I have used Perl::Critic but I don't like it when someone criticizes my code! (even if it's from Damian Conway's Perl best practices book!) I just write sane code and apply most of the best practices. Other people usually don't have problems with my code.

You're a real polyglot as apart from Perl, your Github page hosts projects in C++, JavaScript, Python, OCaml, PHP! Are there any features of those languages would you like to see in Perl?

It's hard to take a feature from another language and put it into Perl as it's already very expressive and supports many different programming paradigms. I often get asked this, but I don't have a good answer! I'm very productive with Perl as it is, whereas I could definitely name features of Perl that I would like to see in other languages.

What about Perl versions, do you keep up to date?

Yeah, right now I'm using Perl 5.18 and that's all thanks to Perlbrew. I remember a few years ago before I knew about Perlbrew it was huge pain to have several different Perl versions and I always had the system Perl and maybe one local version of Perl. Managing packages was a pain, too. Once I found out about Perlbrew, I installed every version of Perl, going back to 5.6. One cool feature is "perlbrew exec" which I used to test my one liners against every version of Perl to see which ones work and don't.

What are your favourite Perl modules and tools?

I really like Try::Tiny for better exception handling, File::Slurp for quickly working with files, WWW::Mechanize and HTML::TreeBuilder for scraping the web.

Talking about tools I also like rxrx by Damian Conway (here's a demo at YAPC). It's this interactive regexp debugger. Another tool that I use is ack that's a better version of grep!

And one last final question - what is it that you're doing in your profile picture? Making a cigarette?

Haha, no! I'm actually holding a piece of scotch tape. I was making a raft that day from empty bottles so I was taping them together. It turned out to be a really good raft and it worked. Here's a photo.

Buy from No Starch Press
Use discount code CATONMAT for 30% off!

Also available at:
Amazon
Amazon Kindle
O'Reilly
Powell's
Barnes and Noble

I've fantastic news - I've become a published author! No Starch Press just published my book Perl One-Liners!

Some programs are powerful because of their thousands of lines of code, and some are powerful because of what they get done with so little. Perl is a language that harnesses the power of minimal code, letting users automate otherwise mind-numbing tasks with a single line of code.

In Perl One-Liners, I take you through more than 100 compelling one-liners that do all sorts of handy and geeky things, such as manipulate line spacing, tally column values in a table, and get a list of users on a system. This cookbook of useful, customizable, and fun scripts will even help hone your Perl coding skills, as I dissect the code to give you a deeper understanding of the language.

Table of Contents

  • Acknowledgments
  • Appendix A: Perl's Special Variables
  • Appendix B: Using Perl One-Liners on Windows
  • Appendix C: perl1line.txt
  • Index (PDF)

View the detailed Table of Contents (PDF)

What are Perl One-Liners?

Perl one-liners are small and awesome Perl programs that fit in a single line of code. They do one thing really well—like changing line spacing, numbering lines, performing calculations, converting and substituting text, deleting and printing specific lines, parsing logs, editing files in-place, calculating statistics, carrying out system administration tasks, or updating a bunch of files at once. Perl one-liners will make you a shell warrior: what took you minutes (or even hours) to solve will now take you only seconds!

Let's take a look at several Perl one-liners to see how short and powerful they are. All these one-liners are explained in greater detail in the book.

Generate a random eight-letter password

perl -le 'print map { ("a".."z")[rand 26] } 1..8'

This one-liner generates and prints a random eight-character password. It uses the list range operator .. to produce all alphabet letters from a to z (for a total of 26 letters). Then a random letter is chosen by rand 26 and this operation is repeated 8 times to produce an eight-letter password.

Sum numbers in each line

perl -MList::Util=sum -lane 'print sum @F'

This one-liner uses the sum function from the List::Util module that comes with Perl. The -a command line argument enables the automatic splitting of the current line into fields in the @F array. The splitting happens on whitespace characters by default. The -l argument ensures that print outputs a newline at the end of each line. The -n argument makes sure every line is processed. Finally, sum @F sums all the elements in the @F array, and print prints the result followed by a newline (which I added with the -l argument).

Print all lines from line 17 to line 30

perl -ne 'print if 17..30'

This one-liner uses the binary flip-flop operator ... The operator is bistable, and it works just like the line-range (comma) operator in sed, awk, and various text editors. Its value is false as long as its left operand is false. Once the left operand is true, the range operator is true until the right operand is true, after which the range operator becomes false again. As a result, this bistable operator becomes true at line 17, stays true until the 30 line, and then becomes and remains false. Combining the flip-flop operator with print if condition makes it print only lines 17 to 30.

Finding all lines in a file that appear more than once

perl -ne 'print if $a{$_}++' file

This one-liner records the lines you've seen so far in the %a hash and counts the number of times it has seen the lines. If it has already seen the line, the condition $a{$_}++ is true, so it prints the line. Otherwise it automatically creates an element that contains the current line in the %a hash and increments its value.

Create 1KB of data

perl -e 'print "a"x1024'

This one-liner repeats the letter "a" 1024 times, creating 1KB of data. Operator x is the repetition operator. This one-liner is handy when you need to generate a specific amount of data for debugging or other tasks.

As you can see, knowing how to write one-liners is very useful. Through the years, it's been one of my top priority tasks to become very efficient in the shell. Literally every day when I'm programming, I've to do all kinds of data processing tasks, changing files, verifying output, doing quick calculations, parsing data, etc. Knowing Perl one-liners makes it really fast to get things done and be really productive. With Perl One-Liners you can now become very efficient at these tasks, too!

Reviews

"By reading this book and trying the recipes, you’ll master shortly all the command-line options that can turn the Perl interpreter into a sed-awk-grep-toolboox-on-steroids."
—Alexis Sukrieh, creator of Perl Dancer (Read More)

“One of the slogans used by Perl is ‘Easy things should be easy and hard things should be possible.’ This book illustrates just how easy things can be—and how much can be done with so little code.”
—David Precious, contributor to the Perl Dancer project and various CPAN modules

“A great reference for when you want to do something with Perl without spending an hour or two trying to figure out how.”
—Sandra Henry-Stocker, IT World (Read More)

“By reading this book you can make a step toward becoming the local computer wizard, even without learning how to program.”
—Gabor Szabo, founder and editor of the Perl Weekly newsletter

“I didn't find anything to yell about. Given how much of a perfectionist bastard I am this is likely high praise.”
—Matt Trout, technical director of Shadowcat Systems

“A set of exercises for deepening your understanding of Perl.”
—John D. Cook, Singular Value Consulting

“The author is enthusiastic about the material and uses an easy writing style. Highly recommended.”
—Thrig (Jeremy Mates), Internet plumber

“These one-liners are great. Simple. Clear. Concise.”
—Jonathan Scott Duff, Perl guru

“A quick read full of useful command-line Perl programs.”
—Chris Fedde, systems engineer and Perl enthusiast

“Handy for anyone who does a lot of one-off text processing: system administrators, coders, or anyone with large amounts of data they need shifted, filtered, or interpreted.”
—Jim Davis, Perl developer

“Any Perl programmers should take the time to get to know the command like switches that make this possible. This book is a pretty good introduction to this way of using Perl.”
—David Cross, Perl Hacker, Trainer and Author (Read More)

“Seriously, folks, CompSci 101 courses might be more fun with a book like this (instead of the standard C++ tome).”
—Thomas Maher (Read More)

“Perl One-liners is a beautifully written manual, if I can call it like that, that empowers sysadmins with a Swiss Army knife to get dirty jobs done.”
—Nitin K Sookun (Read More)

“This book shows the undisputed king of one-line programs in it's full Swiss-Army-Chainsaw glory.”
—Kieran Barry (Read More)

“By the end of the book one should master enough of Perl code to write beautiful scripts.”
—Ish Sookun (Read More)

“Author pulls out a lot of interesting Perl tricks that someone can learn from. The book is worth reading just for that; it's also a short easy read.”
—John Graham-Cumming (Read More)

“If you are doing the sort of thing Perl excels at, like text mangling, this will be a book full of tools for you.”
—Justin Sherrill, DragonFly BSD (Read More)

“The presentation style is great as all the one liners are explained and alternatives presented, in keeping with the Perl idiom "There is more than one way to do it".”
—Andrew Colin Kissa, Creator of Baruwa, Fedora Linux contributor (Read More)

“Summing up, this is a useful book for admins, shell users and coders alike who need their tools concentrated into one toolbox and not spread here and there. Having the book to hand will allow you to call on Perl's full power in an instant when the need arises. Recommended”
—Nikos Vaggalis (Read More)

Book Preview

I've made Chapters 1 and 4 freely available:

You can also view full index of the book and detailed table of contents.

Buy Perl One-Liners

Perl one-liners is available from No Starch Press (use discount code CATONMAT for 30% off), Amazon, Amazon Kindle, O'Reilly, Powell's and Barnes and Noble.

Tweet about my book!

Help me spread the word about my new book! I prepared a special link that you can use to tweet about it:

What's next?

I really love writing about programming and I'll soon publish several more books. The next few are going to be about becoming skillful in bash (you can already start reading it here!), about the most effective vim tricks, a practical guide on how to be anonymous on the web, and the catonmat book.

If you enjoy my writing, you can subscribe to my blog. You can also follow me on Twitter or Google+.

Here's a quick tutorial on how to make unprivileged programs listen on privileged ports. The trick here is to make the unprivileged program to listen on an unprivileged port and redirect the privileged port to the unprivileged through iptables.

Here's a concrete example. Let's say you want to run a web server (on port 80) but don't want to run it as root as it has security implications. What you do instead is run your web server on port 8080 (or any other unprivileged port) and redirect port 80 to 8080 with iptables.

You'll need at least 2 iptables rules to set it up. The first rule will redirect all incoming traffic on all public interfaces from port 80 to port 8080:

iptables -A PREROUTING -t nat -p tcp --dport 80 -j REDIRECT --to-port 8080

The second rule will redirect all localhost traffic from port 80 to port 8080:

iptables -t nat -I OUTPUT -p tcp -d 127.0.0.1 --dport 80 -j REDIRECT --to-ports 8080

You might also need a third rule that will redirect all localhost traffic directed to the public IP (or hostname) of the service:

iptables -t nat -I OUTPUT -p tcp -d hostname.com --dport 80 -j REDIRECT --to-ports 8080

If you're unfamiliar with iptables, see the the frozen tux iptables tutorial. It's the best iptables tutorial out there.

I was recently interviewed by CodeProject who run a weekly interview series in which they talk with developers about their backgrounds, interests and pet peeves. I created a backup copy of the interview here on my blog. Here's the original.

Who are you? Tell us your name, where you live and work, your "job title" or business. Be as specific as comfort and employer policies allow.

My name is Peteris Krumins. I live in Riga, Latvia and I'm the co-founder of Browserling and Testling. I also run the popular programming blog catonmat.net.

What projects have you worked on? Tell us a bit about your daily work and the kinds of projects you've completed. If commercial products, names and URLs are great.

I've worked on over a hundred different projects. I tend to push all my code to GitHub. You can check my GitHub profile. I love writing code in a dozen different languages. I've an unorganized list of the project that I've done on my projects page. Currently my two largest projects are Browserling and Testling. Browserling lets you cross-browser test websites interactively in Internet Explorer and other browsers. Testling lets you run automatic JavaScript tests in various browsers.

Lately I've also started writing books. So far I've written three titles about the essential Unix tools Awk, Sed and Perl. My latest book about Perl is currently being republished by No Starch Press and will be available in October this year. My next title, currently work in progress, is about Bash. I publish all the book chapters for free on my blog. Once I've written all the chapters I put together the book and publish the announcement on my blog. People can then either buy the whole book or read it for free.

I like sticking to a plan and a daily schedule. At all times I've a list of things to do on my desk on an sheet of paper. Here's an example of a recent todo list.

As I finish things, I cross them off. I also keep a printed calendar in front of me with the most important things to do. Here's an example from June 2011.

I also sketch a lot when thinking about things. Here's an example of what my thought process looks like for one of the projects that I did in 2007.

I've been using this method since 2002 and it works for me really well.

What is your development environment? Tell us about your hardware, development tools and most-used or favorite languages and frameworks.

Here's an image from a few years ago that shows my development environment.

And here's another image from a while ago.

I use desktop PCs only. I still run Windows XP on my workstation. It's a 3.2Ghz Pentium 4 with 2 gigs of memory from 2004. It gets the job done. I use two shared screens and a Microsoft Natural keyboard. I don't enjoy laptops because of the small keyboard and the impossible-to-use mouse (touchpad) that they have. I also run two Slackware Linux servers. One as a file server and the other as a development server. The file server runs Samba and is mapped on Windows XP. I also have a bunch of servers for testing and experimenting.

My favorite development tool is Visual Studio 2008. I do most of the C and C++ development in it. Nothing beats IntelliSense and MSDN documentation. I use vim to write in JavaScript, HTML, Python, Perl and other languages. My favorite language is Perl because it's so concise and I can prototype things in it very quickly. I don't enjoy frameworks as it's often much quicker to write a basic version of the same thing than to learn how to use the framework.

What new tools, languages or frameworks are you playing with (or just interested in exploring more)?

I'm currently not playing with any new tools because I love to get things done with the existing tools. The new tools are often confusing, change too fast and/or are too experimental. I only adopt the new tools once they've become old and have survived.

What is your coding pet peeve? Favorite naming convention? Hungarian, camelCase? Indentation style? K&R, Allman, etc.?

It depends on the language that I use and the environment and context I'm working in. For example, if I'm writing a C program from scratch I stick with the_following_naming_convention and 4 spaces for a tab. However if I'm contributing to some project, I'm using the naming convention, and indentation style of the existing project so that my contributions get accepted. If I'm writing a Windows program in MCF, I stick to Hungarian notation. If I'm writing Java, I mostly use camelCase, etc.

How did you get started programming? Tell us about your first computer and programming language.

I learned a bunch of different languages at once very quickly. I learned Pascal, mIRC scripts, HTML, JavaScript, C and Visual Basic all at once. I got my first computer on September 1st, 2000. It was was a Celeron 400 with 256mb of memory and it ran Windows 98. Before that I was writing code on paper at home and trying it out at a local university. A friend of mine called Zombie also helped me a lot with learning programming and Linux.

How has the developer community - online and offline - influenced your coding? What do you like or dislike about the dev community? Are you actively using forums, Twitter, LinkedIn, GitHub or other online resources?

I don't enjoy communities in general. As soon as you're part of a community people assign all the philosophy that goes with the community to you. I like being individual.

What one piece of advice would you give to an up-and-coming programmer?

Get things done and move on.

Who would you like to see us interview?

My friend Amir Salihefendic (https://twitter.com/amix3k). We worked together at a startup called Plurk. He's an amazing developer who gets things done.

I created an ASCII cheat sheet from the table that I used in my blog post about my favorite regular expression. I know there are a billion ASCII cheat sheets out there but I wanted my own. This cheat sheet includes dec, hex, oct and bin values for ASCII characters.


download ascii cheat sheet

Also see my other cheat sheets.