Buy from No Starch Press
Use discount code CATONMAT for 30% off!

Also available at:
Amazon
Amazon Kindle
O'Reilly
Powell's
Barnes and Noble

I've fantastic news - I've become a published author! No Starch Press just published my book Perl One-Liners!

Some programs are powerful because of their thousands of lines of code, and some are powerful because of what they get done with so little. Perl is a language that harnesses the power of minimal code, letting users automate otherwise mind-numbing tasks with a single line of code.

In Perl One-Liners, I take you through more than 100 compelling one-liners that do all sorts of handy and geeky things, such as manipulate line spacing, tally column values in a table, and get a list of users on a system. This cookbook of useful, customizable, and fun scripts will even help hone your Perl coding skills, as I dissect the code to give you a deeper understanding of the language.

Table of Contents

  • Acknowledgments
  • Appendix A: Perl's Special Variables
  • Appendix B: Using Perl One-Liners on Windows
  • Appendix C: perl1line.txt
  • Index (PDF)

View the detailed Table of Contents (PDF)

What are Perl One-Liners?

Perl one-liners are small and awesome Perl programs that fit in a single line of code. They do one thing really well—like changing line spacing, numbering lines, performing calculations, converting and substituting text, deleting and printing specific lines, parsing logs, editing files in-place, calculating statistics, carrying out system administration tasks, or updating a bunch of files at once. Perl one-liners will make you a shell warrior: what took you minutes (or even hours) to solve will now take you only seconds!

Let's take a look at several Perl one-liners to see how short and powerful they are. All these one-liners are explained in greater detail in the book.

Generate a random eight-letter password

perl -le 'print map { ("a".."z")[rand 26] } 1..8'

This one-liner generates and prints a random eight-character password. It uses the list range operator .. to produce all alphabet letters from a to z (for a total of 26 letters). Then a random letter is chosen by rand 26 and this operation is repeated 8 times to produce an eight-letter password.

Sum numbers in each line

perl -MList::Util=sum -lane 'print sum @F'

This one-liner uses the sum function from the List::Util module that comes with Perl. The -a command line argument enables the automatic splitting of the current line into fields in the @F array. The splitting happens on whitespace characters by default. The -l argument ensures that print outputs a newline at the end of each line. The -n argument makes sure every line is processed. Finally, sum @F sums all the elements in the @F array, and print prints the result followed by a newline (which I added with the -l argument).

Print all lines from line 17 to line 30

perl -ne 'print if 17..30'

This one-liner uses the binary flip-flop operator ... The operator is bistable, and it works just like the line-range (comma) operator in sed, awk, and various text editors. Its value is false as long as its left operand is false. Once the left operand is true, the range operator is true until the right operand is true, after which the range operator becomes false again. As a result, this bistable operator becomes true at line 17, stays true until the 30 line, and then becomes and remains false. Combining the flip-flop operator with print if condition makes it print only lines 17 to 30.

Finding all lines in a file that appear more than once

perl -ne 'print if $a{$_}++' file

This one-liner records the lines you've seen so far in the %a hash and counts the number of times it has seen the lines. If it has already seen the line, the condition $a{$_}++ is true, so it prints the line. Otherwise it automatically creates an element that contains the current line in the %a hash and increments its value.

Create 1KB of data

perl -e 'print "a"x1024'

This one-liner repeats the letter "a" 1024 times, creating 1KB of data. Operator x is the repetition operator. This one-liner is handy when you need to generate a specific amount of data for debugging or other tasks.

As you can see, knowing how to write one-liners is very useful. Through the years, it's been one of my top priority tasks to become very efficient in the shell. Literally every day when I'm programming, I've to do all kinds of data processing tasks, changing files, verifying output, doing quick calculations, parsing data, etc. Knowing Perl one-liners makes it really fast to get things done and be really productive. With Perl One-Liners you can now become very efficient at these tasks, too!

Reviews

"By reading this book and trying the recipes, you’ll master shortly all the command-line options that can turn the Perl interpreter into a sed-awk-grep-toolboox-on-steroids."
—Alexis Sukrieh, creator of Perl Dancer (Read More)

“One of the slogans used by Perl is ‘Easy things should be easy and hard things should be possible.’ This book illustrates just how easy things can be—and how much can be done with so little code.”
—David Precious, contributor to the Perl Dancer project and various CPAN modules

“A great reference for when you want to do something with Perl without spending an hour or two trying to figure out how.”
—Sandra Henry-Stocker, IT World (Read More)

“By reading this book you can make a step toward becoming the local computer wizard, even without learning how to program.”
—Gabor Szabo, founder and editor of the Perl Weekly newsletter

“I didn't find anything to yell about. Given how much of a perfectionist bastard I am this is likely high praise.”
—Matt Trout, technical director of Shadowcat Systems

“A set of exercises for deepening your understanding of Perl.”
—John D. Cook, Singular Value Consulting

“The author is enthusiastic about the material and uses an easy writing style. Highly recommended.”
—Thrig (Jeremy Mates), Internet plumber

“These one-liners are great. Simple. Clear. Concise.”
—Jonathan Scott Duff, Perl guru

“A quick read full of useful command-line Perl programs.”
—Chris Fedde, systems engineer and Perl enthusiast

“Handy for anyone who does a lot of one-off text processing: system administrators, coders, or anyone with large amounts of data they need shifted, filtered, or interpreted.”
—Jim Davis, Perl developer

“Any Perl programmers should take the time to get to know the command like switches that make this possible. This book is a pretty good introduction to this way of using Perl.”
—David Cross, Perl Hacker, Trainer and Author (Read More)

“Seriously, folks, CompSci 101 courses might be more fun with a book like this (instead of the standard C++ tome).”
—Thomas Maher (Read More)

“Perl One-liners is a beautifully written manual, if I can call it like that, that empowers sysadmins with a Swiss Army knife to get dirty jobs done.”
—Nitin K Sookun (Read More)

“This book shows the undisputed king of one-line programs in it's full Swiss-Army-Chainsaw glory.”
—Kieran Barry (Read More)

“By the end of the book one should master enough of Perl code to write beautiful scripts.”
—Ish Sookun (Read More)

“Author pulls out a lot of interesting Perl tricks that someone can learn from. The book is worth reading just for that; it's also a short easy read.”
—John Graham-Cumming (Read More)

“If you are doing the sort of thing Perl excels at, like text mangling, this will be a book full of tools for you.”
—Justin Sherrill, DragonFly BSD (Read More)

“The presentation style is great as all the one liners are explained and alternatives presented, in keeping with the Perl idiom "There is more than one way to do it".”
—Andrew Colin Kissa, Creator of Baruwa, Fedora Linux contributor (Read More)

“Summing up, this is a useful book for admins, shell users and coders alike who need their tools concentrated into one toolbox and not spread here and there. Having the book to hand will allow you to call on Perl's full power in an instant when the need arises. Recommended”
—Nikos Vaggalis (Read More)

Book Preview

I've made Chapters 1 and 4 freely available:

You can also view full index of the book and detailed table of contents.

Buy Perl One-Liners

Perl one-liners is available from No Starch Press (use discount code CATONMAT for 30% off), Amazon, Amazon Kindle, O'Reilly, Powell's and Barnes and Noble.

Tweet about my book!

Help me spread the word about my new book! I prepared a special link that you can use to tweet about it:

What's next?

I really love writing about programming and I'll soon publish several more books. The next few are going to be about becoming skillful in bash (you can already start reading it here!), about the most effective vim tricks, a practical guide on how to be anonymous on the web, and the catonmat book.

If you enjoy my writing, you can subscribe to my blog. You can also follow me on Twitter or Google+.

Here's a quick tutorial on how to make unprivileged programs listen on privileged ports. The trick here is to make the unprivileged program to listen on an unprivileged port and redirect the privileged port to the unprivileged through iptables.

Here's a concrete example. Let's say you want to run a web server (on port 80) but don't want to run it as root as it has security implications. What you do instead is run your web server on port 8080 (or any other unprivileged port) and redirect port 80 to 8080 with iptables.

You'll need at least 2 iptables rules to set it up. The first rule will redirect all incoming traffic on all public interfaces from port 80 to port 8080:

iptables -A PREROUTING -t nat -p tcp --dport 80 -j REDIRECT --to-port 8080

The second rule will redirect all localhost traffic from port 80 to port 8080:

iptables -t nat -I OUTPUT -p tcp -d 127.0.0.1 --dport 80 -j REDIRECT --to-ports 8080

You might also need a third rule that will redirect all localhost traffic directed to the public IP (or hostname) of the service:

iptables -t nat -I OUTPUT -p tcp -d hostname.com --dport 80 -j REDIRECT --to-ports 8080

If you're unfamiliar with iptables, see the the frozen tux iptables tutorial. It's the best iptables tutorial out there.

I was recently interviewed by CodeProject who run a weekly interview series in which they talk with developers about their backgrounds, interests and pet peeves. I created a backup copy of the interview here on my blog. Here's the original.

Who are you? Tell us your name, where you live and work, your "job title" or business. Be as specific as comfort and employer policies allow.

My name is Peteris Krumins. I live in Riga, Latvia and I'm the co-founder of Browserling and Testling. I also run the popular programming blog catonmat.net.

What projects have you worked on? Tell us a bit about your daily work and the kinds of projects you've completed. If commercial products, names and URLs are great.

I've worked on over a hundred different projects. I tend to push all my code to GitHub. You can check my GitHub profile. I love writing code in a dozen different languages. I've an unorganized list of the project that I've done on my projects page. Currently my two largest projects are Browserling and Testling. Browserling lets you cross-browser test websites interactively in Internet Explorer and other browsers. Testling lets you run automatic JavaScript tests in various browsers.

Lately I've also started writing books. So far I've written three titles about the essential Unix tools Awk, Sed and Perl. My latest book about Perl is currently being republished by No Starch Press and will be available in October this year. My next title, currently work in progress, is about Bash. I publish all the book chapters for free on my blog. Once I've written all the chapters I put together the book and publish the announcement on my blog. People can then either buy the whole book or read it for free.

I like sticking to a plan and a daily schedule. At all times I've a list of things to do on my desk on an sheet of paper. Here's an example of a recent todo list.

As I finish things, I cross them off. I also keep a printed calendar in front of me with the most important things to do. Here's an example from June 2011.

I also sketch a lot when thinking about things. Here's an example of what my thought process looks like for one of the projects that I did in 2007.

I've been using this method since 2002 and it works for me really well.

What is your development environment? Tell us about your hardware, development tools and most-used or favorite languages and frameworks.

Here's an image from a few years ago that shows my development environment.

And here's another image from a while ago.

I use desktop PCs only. I still run Windows XP on my workstation. It's a 3.2Ghz Pentium 4 with 2 gigs of memory from 2004. It gets the job done. I use two shared screens and a Microsoft Natural keyboard. I don't enjoy laptops because of the small keyboard and the impossible-to-use mouse (touchpad) that they have. I also run two Slackware Linux servers. One as a file server and the other as a development server. The file server runs Samba and is mapped on Windows XP. I also have a bunch of servers for testing and experimenting.

My favorite development tool is Visual Studio 2008. I do most of the C and C++ development in it. Nothing beats IntelliSense and MSDN documentation. I use vim to write in JavaScript, HTML, Python, Perl and other languages. My favorite language is Perl because it's so concise and I can prototype things in it very quickly. I don't enjoy frameworks as it's often much quicker to write a basic version of the same thing than to learn how to use the framework.

What new tools, languages or frameworks are you playing with (or just interested in exploring more)?

I'm currently not playing with any new tools because I love to get things done with the existing tools. The new tools are often confusing, change too fast and/or are too experimental. I only adopt the new tools once they've become old and have survived.

What is your coding pet peeve? Favorite naming convention? Hungarian, camelCase? Indentation style? K&R, Allman, etc.?

It depends on the language that I use and the environment and context I'm working in. For example, if I'm writing a C program from scratch I stick with the_following_naming_convention and 4 spaces for a tab. However if I'm contributing to some project, I'm using the naming convention, and indentation style of the existing project so that my contributions get accepted. If I'm writing a Windows program in MCF, I stick to Hungarian notation. If I'm writing Java, I mostly use camelCase, etc.

How did you get started programming? Tell us about your first computer and programming language.

I learned a bunch of different languages at once very quickly. I learned Pascal, mIRC scripts, HTML, JavaScript, C and Visual Basic all at once. I got my first computer on September 1st, 2000. It was was a Celeron 400 with 256mb of memory and it ran Windows 98. Before that I was writing code on paper at home and trying it out at a local university. A friend of mine called Zombie also helped me a lot with learning programming and Linux.

How has the developer community - online and offline - influenced your coding? What do you like or dislike about the dev community? Are you actively using forums, Twitter, LinkedIn, GitHub or other online resources?

I don't enjoy communities in general. As soon as you're part of a community people assign all the philosophy that goes with the community to you. I like being individual.

What one piece of advice would you give to an up-and-coming programmer?

Get things done and move on.

Who would you like to see us interview?

My friend Amir Salihefendic (https://twitter.com/amix3k). We worked together at a startup called Plurk. He's an amazing developer who gets things done.

I created an ASCII cheat sheet from the table that I used in my blog post about my favorite regular expression. I know there are a billion ASCII cheat sheets out there but I wanted my own. This cheat sheet includes dec, hex, oct and bin values for ASCII characters.


download ascii cheat sheet

Also see my other cheat sheets.

Six Years of BloggingHey everyone! Another year has passed and it's been 6 years since I've been blogging here on catonmat! In this post I wish to summarize this year's statistics.

See the one year of blogging, two years of blogging, three years of blogging, four years of blogging, and five years of blogging for the previous year statistics.

Let's start this year's stats with traffic:



Traffic statistics from Google Analytics for Jul 1, 2012 - Jun 30, 2013.

This year: 903,468 uniques. 1,191,596 visits. 1,695,544 page views.
Last year: 988,295 uniques. 1,374,228 visits. 1,940,530 page views.
Delta: -84,827 unqiues. -182,632 visits. -244,986 page views.

I haven't been blogging as much as in the previous year and that explains the negative delta.

Feedburner stats: At 16,343 rss subscribers this year. Last year 14,362. Delta +1,981. subscribe.
Github stats: At 1400 followers this year. Last year 1070. Delta +330. follow me.
Twitter stats: At 3774 followers this year. Last year 3093. Delta +681. follow me.

Subscriptions and followers grow over time, which explains the positive delta.

Total articles written: 25. Top 10 articles:

My top 5 favorite articles:

During this year I made 4 announcements:

That's all folks! Now let's have some 6 year birthday cake,

And let's meet for the cake next year again! See you!