This article is part of the article series "Perl One-Liners Explained."
<- previous article next article ->
Perl One Liners

This is the second part of a seven-part article on famous Perl one-liners. In this part I will create various one-liners for line numbering. See part one for introduction of the series.

Famous Perl one-liners is my attempt to create "perl1line.txt" that is similar to "awk1line.txt" and "sed1line.txt" that have been so popular among Awk and Sed programmers.

The article on famous Perl one-liners will consist of at least seven parts:

The one-liners will make heavy use of Perl special variables. A few years ago I compiled all the Perl special variables in a single file and called it Perl special variable cheat-sheet. Even tho it's mostly copied out of perldoc perlvar, it's still handy to have in front of you. Print it!

Awesome news: I have written an e-book based on this article series. Check it out:

And here are today's one-liners:

Line Numbering

9. Number all lines in a file.

perl -pe '$_ = "$. $_"'

As I explained in the first one-liner, "-p" causes Perl to assume a loop around the program (specified by "-e") that reads each line of input into the " $_ " variable, executes the program and then prints the " $_ " variable.

In this one-liner I simply modify " $_ " and prepend the " $. " variable to it. The special variable " $. " contains the current line number of input.

The result is that each line gets its line number prepended.

10. Number only non-empty lines in a file.

perl -pe '$_ = ++$a." $_" if /./'

Here we employ the "action if condition" statement that executes "action" only if "condition" is true. In this case the condition is a regular expression "/./", which matches any character except newline (that is, it matches a non-empty line); and the action is " $_ = ++$a." $_" ", which prepends variable " $a " incremented by one to the current line. As we didn't use strict pragma, $a was created automatically.

The result is that at each non-empty line " $a " gets incremented by one and prepended to that line. And at each empty line nothing gets modified and the empty line gets printed as is.

11. Number and print only non-empty lines in a file (drop empty lines).

perl -ne 'print ++$a." $_" if /./'

This one-liner uses the "-n" program argument that places the line in " $_ " variable and then executes the program specified by "-e". Unlike "-p", it does not print the line after executing code in "-e", so we have to call "print" explicitly to get it printed.

The one-liner calls "print" only on lines that have at least one character in them. And exactly like in the previous one-liner, it increments the line number in variable " $a " by one for each non-empty line.

The empty lines simply get ignored and never get printed.

12. Number all lines but print line numbers only non-empty lines.

perl -pe '$_ = "$. $_" if /./'

This one-liner is similar to one-liner #10. Here I modify the " $_ " variable that holds the entire line only if the line has at least one character. All other lines (empty ones) get printed without line numbers.

13. Number only lines that match a pattern, print others unmodified.

perl -pe '$_ = ++$a." $_" if /regex/'

Here we again use the "action if condition" statement but the condition in this case is a pattern (regular expression) "/regex/". The action is the same as in one-liner #10. I don't want to repeat, see #10 for explanation.

14. Number and print only lines that match a pattern.

perl -ne 'print ++$a." $_" if /regex/'

This one-liner is almost exactly like #11. The only difference is that it prints numbered lines that match only "/regex/".

15. Number all lines, but print line numbers only for lines that match a pattern.

perl -pe '$_ = "$. $_" if /regex/'

This one-liner is similar to the previous one-liner and to one-liner #12. Here the line gets its line number prepended if it matches a /regex/, otherwise it just gets printed without a line number.

16. Number all lines in a file using a custom format (emulate cat -n).

perl -ne 'printf "%-5d %s", $., $_'

This one-liner uses the formatted print "printf" function to print the line number together with line. In this particular example the line numbers are left aligned on 5 char boundary.

Some other nice format strings are "%5d" that right-aligns line numbers on 5 char boundary and "%05d" that zero-fills and right-justifies the line numbers.

Here my Perl printf cheat sheet might come handy that lists all the possible format specifiers.

17. Print the total number of lines in a file (emulate wc -l).

perl -lne 'END { print $. }'

This one-liner uses the "END" block that Perl probably took as a feature from Awk language. The END block gets executed after the program has executed. In this case the program is the hidden loop over the input that was created by the "-n" argument. After it has looped over the input, the special variable " $. " contains the number of lines there was in the input. The END block prints this variable. The " -l " parameter sets the output record separator for "print" to a newline (so that we didn't have to print "$.\n").

Another way to do the same is:

perl -le 'print $n=()=<>'

This is a tricky one, but easy to understand if you know about Perl contexts. In this one-liner the " ()=<> " part causes the <> operator (the diamond operator) to evaluate in list context, that causes the diamond operator to read the whole file in a list. Next, " $n " gets evaluated in scalar context. Evaluating a list in a scalar context returns the number of elements in the list. Thus the " $n=()=<> " construction is equal to the number of lines in the input, that is number of lines in the file. The print statement prints this number out. The " -l " argument makes sure a newline gets added after printing out this number.

This is the same as writing the following, except longer:

perl -le 'print scalar(()=<>)'

And completely obvious version:

perl -le 'print scalar(@foo=<>)'

Yet another way to do it:

perl -ne '}{print $.'

This one-liner uses the eskimo operator "}{" in conjunction with "-n" command line argument. As I explained in one-liner #11, the "-n" argument forces Perl to assume a " while(<>) { } " loop around the program. The eskimo operator forces Perl to escape the loop, and the program turns out to be:

while (<>) {
}{                    # eskimo operator here
    print $.;
}

It's easy to see that this program just loops over all the input and after it's done doing so, it prints the " $. ", which is the number of lines in the input.

18. Print the number of non-empty lines in a file.

perl -le 'print scalar(grep{/./}<>)'

This one-liner uses the "grep" function that is similar to the grep Unix command. Given a list of values, " grep {condition} " returns only those values that match condition. In this case the condition is a regular expression that matches at least one character, so the input gets filtered and the "grep{/./}" returns all lines that were non empty. To get the number of characters we evaluate the list in scalar context and print the result. (As I mentioned in the previous one-liner list in scalar context evaluates to number of elements in the list).

A golfer's version of this one-liner would be to replace "scalar()" with " ~~ " (double bitwise negate), thus it can be shortened:

perl -le 'print ~~grep{/./}<>'

This can be made even shorter:

perl -le 'print~~grep/./,<>'

19. Print the number of empty lines in a file.

perl -lne '$a++ if /^$/; END {print $a+0}'

Here I use variable $a to count how many empty lines have I encountered. Once I have finished looping over all the lines, I print the value of $a in the END block. I use " $a+0 " construction to make sure " 0 " gets output if no lines were empty.

I could have also modified the previous one-liner:

perl -le 'print scalar(grep{/^$/}<>)'

Or written it with " ~~ ":

perl -le 'print ~~grep{/^$/}<>'

These last two versions are not as effective, as they would read the whole file in memory. Where as the first one would do it line by line.

20. Print the number of lines in a file that match a pattern (emulate grep -c).

perl -lne '$a++ if /regex/; END {print $a+0}'

This one-liner is basically the same as the previous one, except it increments the line counter $a by one in case a line matches a regular expression /regex/.

Perl one-liners explained e-book

I've now written the "Perl One-Liners Explained" e-book based on this article series. I went through all the one-liners, improved explanations, fixed mistakes and typos, added a bunch of new one-liners, added an introduction to Perl one-liners and a new chapter on Perl's special variables. Please take a look:

Have Fun!

Have fun with these one-liners. These were really easy this time. The next part is going to be about various calculations.

Can you think of other numbering operations that I did not include here?

This article is part of the article series "Perl One-Liners Explained."
<- previous article next article ->

Comments

July 31, 2009, 16:32

How about printing the line numbers of those lines matching a regex? I wanted to do that the other day, since I was searching for control characters in a file that I couldn't find as easily with vi.

James Permalink
July 31, 2009, 20:49

@Michael - you'd probably want to use Ack (http://betterthangrep.com/) for that task, but here's a one-liner solution in the spirit of this article:

perl -pe '$_ = (/regex/ ? "$.:\t$_" : "")'
July 31, 2009, 20:50

Michael, I already provided a solution for that, see #12 and #13!

James Permalink
July 31, 2009, 22:29

@Peteris - thanks for the great article! I enjoyed reading it. I saw examples #12 and #13, but I noticed that they do not print out the line numbers. Instead they simply number the lines that matched. So, let's say that the 5th instance of a match in the file exists on line 52. Examples #12 and #13 will print out "5" for the number instead of "52". That's why I added the additional example in the comments here to print out the actual line number of the matching lines.

July 31, 2009, 22:49

James, oh, I see. Here are solutions for that:

perl -pe '$_ = "$. $_" if /regex/'

Or if you don't want to print lines that did not match at all, then:

perl -ne 'print "$. $_" if /regex/'

I will modify the article to include them, thanks for the comment!

July 31, 2009, 23:00

Thank you for these perl 1 liners juggling!

Why not adding an alternative to:
perl -le 'print~~grep/./,'

as another template example?...

perl -le 'print"TOTAL NON EMPTY LINES",~~grep!/^$/,'

It illustrates the negative regex and expands the alternatives a little further, just by using your examples.

Thank's
alberto

August 16, 2013, 09:51

straw cap What do you think of this provision of the Cap & Trade Bill? Will this be good for the economy?
<A>hiphop cap

SteveC Permalink
August 02, 2009, 04:57

grep has a -n option, you know. (I mean, I'm sure you do know.) cat does too, these days.

August 05, 2009, 21:40

SteveC, I surely do know that but this is an article on Perl, and not other tools. The goal is to create perl1line.txt, similar to awk1line.txt and sed1line.txt.

poletti Permalink
August 25, 2009, 16:27

A trick that is very useful in one-liners is using the closing-opening braces and avoid END blocks. For example, this:

perl -lne '$a++ if /regex/; END {print $a+0}'

can be turned into this:

perl -lne '$a++ if /regex/}{print $a+0'

And there's more! You can actually turn the "n" into a "p" and save a few more keystrokes using $_:

perl -lpe '$a++ if /regex/}{$_=$a+0'

This takes advantage of the code that is automagically put around your code (more or less...):

while () {
your-code-here
;print; # if -p, otherwise no print
}

which becomes:

while () {
$a++ if /regex/}{$_=$a+0
;print;
}

i.e.

while () {
$a++ if /regex/
}

{
$_=$a+0;
print;
}

I seem to remember that Abigail is to be credited for this.

thisisraja Permalink
September 21, 2011, 00:01

crazy! guys...

michael.wang Permalink
January 08, 2010, 02:30

The use of }{ operator is wonderful....

nanker Permalink
January 15, 2011, 16:42

hey I loved your SED oneliners and I can make much better use of SED having read them. I am running into a few things I would like to do with file manipulation I can find a solution for with sed and have been thinking Perl might be what I need.

In specific, I want to search a file for a particular string and the line number on which it occurs... say line 5...

5 <font>

read in the line number, add 1 (or any number I like) and enter that number as the value for the (HTML) font tags...

5 <font>6

I guess I should first ask if there is indeed a way in SED this can be easily accomplished? if not how could I do this in perl?

I am not a programmer, I get done what I need to and very little more, so the more plain the answer the better for.

currently when I run into these kinds of things I just end up fixing them manually, but I know there are better ways. I am thinking I should just dive in and spend the time to learn enough perl work from the command line in place of SED. Is that a reasonable progression do you think, if I am editting files almost daily with little tasks like this?

thanks for the help and any advice

March 20, 2014, 02:11

I was very happy that I discovered this website. I needed to thank you for this excellent information . best forex trading platform I have bookmark this page and after work I read interesting things to update myself and to refresh my mind

Leave a new comment

(why do I need your e-mail?)

(Your twitter name, if you have one. (I'm @pkrumins, btw.))

Type the first letter of your name: (just to make sure you're a human)

Please preview the comment before submitting to make sure it's OK.

Advertisements