This article is part of the article series "Node.JS Modules You Should Know About."
<- previous article next article ->

node logoHey everyone! This is the fourth post in my new node.js modules you should know about article series.

The first post was about dnode - the freestyle rpc library for node, the second was about optimist - the lightweight options parser for node, the third was about lazy - lazy lists for node.

This time I'll introduce you to a very awesome module called request by Mikeal Rogers. Request is the swiss army knife of HTTP streaming.

Check this out:

var fs = require('fs')
var request = require('request');

request('http://google.com/doodle.png').pipe(fs.createWriteStream('doodle.png'))

Pow! You just streamed the response of HTTP request to http://google.com/doodle.png into doodle.png local file!

Here is more awesome stuff:

var fs = require('fs')
var request = require('request');

fs.readStream('file.json').pipe(request.put('http://mysite.com/obj.json'))

Pow! It streamed your local file file.json to http://mysite.com/obj.json as HTTP PUT request!

var request = require('request');

request.get('http://google.com/img.png').pipe(request.put('http://mysite.com/img.png'))

Pow! This just streamed a HTTP GET from http://google.com/img.png to HTTP PUT to http://mysite.com/img.png.

At Browserling we use this module for streaming data to and from couchdb. Here is an example that saves a JSON document at mikeal's test couchdb:

var request = require('request')
var rand = Math.floor(Math.random()*100000000).toString()

request({
  method: 'PUT',
  uri: 'http://mikeal.iriscouch.com/testjs/' + rand,
  multipart: [
    {
      'content-type': 'application/json',
      'body': JSON.stringify({
        foo: 'bar',
        _attachments: {
          'message.txt': {
            follows: true,
            length: 18,
            'content_type': 'text/plain'
           }
         }
       })
    },
    { body: 'I am an attachment' }
  ] 
}, function (error, response, body) {
  if(response.statusCode == 201){
    console.log('document saved as: http://mikeal.iriscouch.com/testjs/'+ rand);
  } else {
    console.log('error: '+ response.statusCode);
    console.log(body);
  }
})

Install it via npm, as always:

npm install request

Sponsor this blog series!

Doing a node.js company and want your ad to appear in the series? The ad will go out to 14,000 rss subscribers, 7,000 email subscribers, and it will get viewed by thousands of my blog visitors! Email me and we'll set it up!

See ya!

If you love these articles, subscribe to my blog for more, follow me on Twitter to find about my adventures, and watch me produce code on GitHub!

This article is part of the article series "Node.JS Modules You Should Know About."
<- previous article next article ->

node logoHey everyone! This is the third post in my new node.js modules you should know about article series.

The first post was about dnode - the freestyle rpc library for node, the second was about optimist - the lightweight options parser for node.

This time I'll introduce you to one of my own modules called node-lazy - lazy lists module for node.

Basically you create a new lazy object, and pump data into it via data events (it's an event emitter). Then you can manipulate this data via chaining by using various functional programming methods.

Here is a quick example. Here we create a new lazy object and define a filter that returns only even integers, then we take just 5 elements, then we apply map on them, and finally we join (think of threads) the result in a list:

var Lazy = require('lazy');

var lazy = new Lazy;
lazy
  .filter(function (item) {
    return item % 2 == 0
  })
  .take(5)
  .map(function (item) {
    return item*2;
  })
  .join(function (xs) {
    console.log(xs);
  });

You can return this object from your function and then later when someone pumps data into it via data events, it will do the computation.

For example, if you do this:

[0,1,2,3,4,5,6,7,8,9,10].forEach(function (x) {
  lazy.emit('data', x);
});
setTimeout(function () { lazy.emit('end') }, 100);

Then the output will be produced by the console.log once 5 elements have reached the bottom of the chain.

The output is: [0, 4, 8, 12, 16].

Here is a real world example from node-iptables (another of my modules):

var Lazy = require('lazy');
var spawn = require('child_process').spawn;
var iptables = spawn('iptables', ['-L', '-n', '-v']);

Lazy(iptables.stdout)
    .lines
    .map(String)
    .skip(2) // skips the two lines that are iptables header
    .map(function (line) {
        // packets, bytes, target, pro, opt, in, out, src, dst, opts
        var fields = line.trim().split(/\s+/, 9);
        return {
            parsed : {
                packets : fields[0],
                bytes : fields[1],
                target : fields[2],
                protocol : fields[3],
                opt : fields[4],
                in : fields[5],
                out : fields[6],
                src : fields[7],
                dst : fields[8]
            },
            raw : line.trim()
        };
    });

This code fragment takes output from iptables -L -n -v and converts it into a data structure for later use.

Here a new Lazy object is created from an existing stream - the iptables.stdout stream. Next, the special lines getter is called that splits the stream on \n char and converts it into a line-stream. Then this stream is mapped onto String constructor to convert it to a string. Next, first two lines are skipped via skip(2) and then all the other lines are converted into a data structure via map.

You can also create all kinds of ranges with node-lazy, including infinite ranges. Check this out, ranges.js:

var Lazy = require('lazy');

Lazy.range('1..20').join(function (xs) {
    console.log(xs);
});

Lazy.range('444..').take(10).join(function (xs) {
    console.log(xs);
});

Lazy.range('2,4..20').take(10).join(function (xs) {
    console.log(xs);
});

When you run it:

$ node ranges.js 
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19 ]
[ 2, 4, 6, 8, 10, 12, 14, 16, 18 ]
[ 444, 445, 446, 447, 448, 449, 450, 451, 452, 453 ]

Here are all the possible ranges that node-lazy supports:

Lazy.range('10..')       - infinite range starting from 10
Lazy.range('(10..')      - infinite range starting from 11
Lazy.range(10)           - range from 0 to 9
Lazy.range(-10, 10)      - range from -10 to 9 (-10, -9, ... 0, 1, ... 9)
Lazy.range(-10, 10, 2)   - range from -10 to 8, skipping every 2nd element (-10, -8, ... 0, 2, 4, 6, 8)
Lazy.range(10, 0, 2)     - reverse range from 10 to 1, skipping every 2nd element (10, 8, 6, 4, 2)
Lazy.range(10, 0)        - reverse range from 10 to 1
Lazy.range('5..50')      - range from 5 to 49
Lazy.range('50..44')     - range from 50 to 45
Lazy.range('1,1.1..4')   - range from 1 to 4 with increment of 0.1 (1, 1.1, 1.2, ... 3.9)
Lazy.range('4,3.9..1')   - reverse range from 4 to 1 with decerement of 0.1
Lazy.range('[1..10]')    - range from 1 to 10 (all inclusive)
Lazy.range('[10..1]')    - range from 10 to 1 (all inclusive)
Lazy.range('[1..10)')    - range from 1 to 9
Lazy.range('[10..1)')    - range from 10 to 2
Lazy.range('(1..10]')    - range from 2 to 10
Lazy.range('(10..1]')    - range from 9 to 1
Lazy.range('(1..10)')    - range from 2 to 9
Lazy.range('[5,10..50]') - range from 5 to 50 with a step of 5 (all inclusive)

Install it via npm, as always:

npm install lazy

Awesome sauce!

If you love these articles, subscribe to my blog for more, follow me on Twitter to find about my adventures, and watch me produce code on GitHub!

Sponsor this blog series!

Doing a node.js company and want your ad to appear in the series? The ad will go out to 14,000 rss subscribers, 7,000 email subscribers, and it will get viewed by thousands of my blog visitors! Email me and we'll set it up!

This article is part of the article series "Node.JS Modules You Should Know About."
<- previous article next article ->

node logoHey everyone! This is the second post in my new node.js modules you should know about article series.

The first post was about dnode - the freestyle rpc library for node.

This time I'll introduce you to node-optimist - the lightweight options parser library. This library is also written by James Halliday (SubStack), my co-founder of Browserling and Testling.

Wonder how lightweight an options parser can be? Check this out:

var argv = require('optimist').argv;

And you're done! All options have been parsed for you and have been put in argv.

Here are various use cases. First off, it supports long arguments:

#!/usr/bin/env node
var argv = require('optimist').argv;

if (argv.rif - 5 * argv.xup > 7.138) {
    console.log('Buy more riffiwobbles');
}
else {
    console.log('Sell the xupptumblers');
}

Now you can run this script with --rif and --xup arguments like this:

$ ./xup.js --rif=55 --xup=9.52
Buy more riffiwobbles

$ ./xup.js --rif 12 --xup 8.1
Sell the xupptumblers

I know you want to buy more riffiwobbles and sell your xupptumblers.

Next, it supports short args:

#!/usr/bin/env node
var argv = require('optimist').argv;
console.log('(%d,%d)', argv.x, argv.y);

You can use -x and -y as arguments:

$ ./short.js -x 10 -y 21
(10,21)

Then node-optimist supports boolean arguments, both short, long and grouped:

#!/usr/bin/env node
var argv = require('optimist').argv;

if (argv.s) {
    console.log(argv.fr ? 'Le chat dit: ' : 'The cat says: ');
}
console.log(
    (argv.fr ? 'miaou' : 'meow') + (argv.p ? '.' : '')
);

And now you can invoke the script with various options:

$ ./bool.js -s
The cat says: meow

$ ./bool.js -sp
The cat says: meow.

$ ./bool.js -sp --fr
Le chat dit: miaou.

Next, you can easily get to non-hypenated options via argv._:

#!/usr/bin/env node
var argv = require('optimist').argv;

console.log('(%d,%d)', argv.x, argv.y);
console.log(argv._);

Here are use cases for non-hypenated options:

$ ./nonopt.js -x 6.82 -y 3.35 moo
(6.82,3.35)
[ 'moo' ]

$ ./nonopt.js foo -x 0.54 bar -y 1.12 baz
(0.54,1.12)
[ 'foo', 'bar', 'baz' ]

Optimist also comes with .usage() and .demand() functions:

#!/usr/bin/env node
var argv = require('optimist')
    .usage('Usage: $0 -x [num] -y [num]')
    .demand(['x','y'])
    .argv;

console.log(argv.x / argv.y);

Here arguments x and y are required and if they are not passed, the usage will be printed automatically:

$ ./divide.js -x 55 -y 11
5

$ node ./divide.js -x 4.91 -z 2.51
Usage: node ./divide.js -x [num] -y [num]

Options:
  -x  [required]
  -y  [required]

Missing required arguments: y

Optimist also supports default arguments via .default():

#!/usr/bin/env node
var argv = require('optimist')
    .default('x', 10)
    .default('y', 10)
    .argv
;
console.log(argv.x + argv.y);

Here x and y default to 10:

$ ./default_singles.js -x 5
15

Enjoy this stranger:

Alternatively you can use isaacs's nopt that can enforce data types on arguments and can be used to easily handle a lot of arguments. Or you can use nomnom that noms your args and gives them back to you in a hash.

If you love these articles, subscribe to my blog for more, follow me on Twitter to find about my adventures, and watch me produce code on GitHub!

Sponsor this blog series!

Doing a node.js company and want your ad to appear in the series? The ad will go out to 14,000 rss subscribers, 7,000 email subscribers, and it will get viewed by thousands of my blog visitors! Email me and we'll set it up!

This article is part of the article series "Node.JS Modules You Should Know About."
<- previous article next article ->

node logoHey everyone! I am starting a new article series called node.js modules you should know about. I have been using node for over 2 years now and I built Browserling startup using node so I know just about everything about it. I also have written about 20 node.js modules myself (see my github).

In this series I will go through a few dozen of node.js modules, give examples and explain where it's useful.

The first module in the series is dnode. Dnode is freestyle rpc library and it's written by James Halliday (SubStack) ― co-founder of Browserling and Testling.

Here is what it is. This is the server.js:

var dnode = require('dnode');

var server = dnode({
    mul : function (n, m, cb) { cb(n * m) }
});
server.listen(5050);

And here is the client.js:

var dnode = require('dnode');

dnode.connect(5050, function (remote) {
    remote.mul(10, 20, function (n) {
        console.log('10 * 20 = ' + n);
    });
});

Now when you run client.js, you get the output:

$ node client.js
200

See what it did? It called the mul function at server side from the client side and passed it arguments 10 and 20. They got multiplied at server side and the result got sent back to the client by calling cb.

It's important to stress that no code was passed along, all this happened purely through references. You can see the implementation dnode protocol in dnode-protocol github repo.

Here is a more complex example, where client calls server, which calls client again, which passes the result back to server, which then calls client and prints the result.

server.js:

var dnode = require('dnode');

var server = dnode(function (client) {
    this.calculate = function (n, m, cb) {
        client.div(n*m, function (res) {
            cb(res+1)
        });
    }
});
server.listen(5050);

client.js:

var dnode = require('dnode');

var client = dnode({
    div : function (n, cb) {
       cb(n/5);
    }
});

client.connect(5050, function (remote) {
    remote.calculate(10, 20, function (n) {
        console.log('the result is ' + n);
    });
});

When you run the client, you'll get result 41. Here is what happens. First you connect to dnode server at port 5050. Once you're connected, dnode client calls calculate function on server side and passes it arguments 10 and 20 and a callback function that prints the result. Now when the server receives the arguments 10 and 20, it multiplies them together and calls the client's div function, that divides the result by 5. The result is returned back to the server and it adds 1 to it and calls the original callback that prints the result.

We use dnode everywhere at Browserling. Every service is a dnode server and they are all interconnected. For example, the authentication is a dnode server. We can bring it down and update, while the rest of the site is up. Really awesome.

You can install dnode through npm:

npm install dnode

And since dnode has a well defined protocol, you can implement it in any language! Here are dnode implementations in Perl, Ruby, PHP, Java.

Enjoy this rapping turtle!

If you love these articles, subscribe to my blog for more, follow me on Twitter to find about my adventures, and watch me produce code on GitHub!

Sponsor this blog series!

Doing a node.js company and want your ad to appear in the series? The ad will go out to 14,000 rss subscribers, 7,000 email subscribers, and it will get viewed by thousands of my blog visitors! Email me and we'll set it up!

This article is part of the article series "Perl One-Liners Explained."
<- previous article next article ->

Perl One LinersThe ultimate goal of the Perl One-Liners Explained article series was to release the perl1line.txt file. Last week I finished the series and today I am happy to announce perl1line.txt - a collection of handy Perl one-liner scripts.

The perl1line.txt file contains over a hundred short Perl one-line scripts for various text processing tasks. The file processing tasks include: changing file spacing, numbering lines, doing calculations, creating strings and arrays, converting and substituting text, selective printing and deleting of certain lines and text filtering and modifications through regular expressions.

The latest version of perl1line.txt is always at:

  • http://www.catonmat.net/download/perl1line.txt

Here is the full table of contents of perl1line.txt:

  • File Spacing.
  • Line Numbering.
  • Calculations.
  • String Creation and Array Creation.
  • Text Conversion and Substitution.
  • Selective Printing and Deleting of Certain Lines.
  • Handy Regular Expressions.

You can send me bug fixes and updates via GitHub. I put the file in its own perl1line.txt repository. I also accept translations. Send them in!

Awesome news: I have written an e-book based on the one-liners in this file. Check it out:

Here is the whole file of perl1line.txt at version 1.0:

Useful One-Line Scripts for Perl                     Nov 14 2011 | version 1.0
--------------------------------                     -----------   -----------

Compiled by Peteris Krumins (peter@catonmat.net, @pkrumins on Twitter)
http://www.catonmat.net -- good coders code, great reuse

Latest version of this file is always at:

    http://www.catonmat.net/download/perl1line.txt

This file is also available in other languages:
    (None at the moment.)
    Please email me peter@catonmat.net if you wish to translate it.

I am also writing "Perl One-Liners Explained" ebook that's based on
this file. It explains all the one-liners here. Get it soon at:

    http://www.catonmat.net/blog/perl-book/

These one-liners work both on UNIX systems and Windows. Most likely your
UNIX system already has Perl. For Windows get the Strawberry Perl at:

    http://www.strawberryperl.com/

Table of contents:

    1. File Spacing
    2. Line Numbering
    3. Calculations
    4. String Creation and Array Creation
    5. Text Conversion and Substitution
    6. Selective Printing and Deleting of Certain Lines    
    7. Handy Regular Expressions


FILE SPACING 
------------

# Double space a file
perl -pe '$\="\n"'
perl -pe 'BEGIN { $\="\n" }'
perl -pe '$_ .= "\n"'
perl -pe 's/$/\n/'

# Double space a file, except the blank lines
perl -pe '$_ .= "\n" unless /^$/'
perl -pe '$_ .= "\n" if /\S/'

# Triple space a file
perl -pe '$\="\n\n"'
perl -pe '$_.="\n\n"'

# N-space a file
perl -pe '$_.="\n"x7'

# Add a blank line before every line
perl -pe 's//\n/'

# Remove all blank lines
perl -ne 'print unless /^$/'
perl -lne 'print if length'
perl -ne 'print if /\S/'

# Remove all consecutive blank lines, leaving just one
perl -00 -pe ''
perl -00pe0

# Compress/expand all blank lines into N consecutive ones
perl -00 -pe '$_.="\n"x4'


LINE NUMBERING
--------------

# Number all lines in a file
perl -pe '$_ = "$. $_"'

# Number only non-empty lines in a file
perl -pe '$_ = ++$a." $_" if /./'

# Number and print only non-empty lines in a file (drop empty lines)
perl -ne 'print ++$a." $_" if /./'

# Number all lines but print line numbers only non-empty lines
perl -pe '$_ = "$. $_" if /./'

# Number only lines that match a pattern, print others unmodified
perl -pe '$_ = ++$a." $_" if /regex/'

# Number and print only lines that match a pattern
perl -ne 'print ++$a." $_" if /regex/'

# Number all lines, but print line numbers only for lines that match a pattern
perl -pe '$_ = "$. $_" if /regex/'

# Number all lines in a file using a custom format (emulate cat -n)
perl -ne 'printf "%-5d %s", $., $_'

# Print the total number of lines in a file (emulate wc -l)
perl -lne 'END { print $. }'
perl -le 'print $n=()=<>'
perl -le 'print scalar(()=<>)'
perl -le 'print scalar(@foo=<>)'
perl -ne '}{print $.'

# Print the number of non-empty lines in a file
perl -le 'print scalar(grep{/./}<>)'
perl -le 'print ~~grep{/./}<>'
perl -le 'print~~grep/./,<>'

# Print the number of empty lines in a file
perl -lne '$a++ if /^$/; END {print $a+0}'
perl -le 'print scalar(grep{/^$/}<>)'
perl -le 'print ~~grep{/^$/}<>'

# Print the number of lines in a file that match a pattern (emulate grep -c)
perl -lne '$a++ if /regex/; END {print $a+0}'


CALCULATIONS
------------

# Check if a number is a prime
perl -lne '(1x$_) !~ /^1?$|^(11+?)\1+$/ && print "$_ is prime"'

# Print the sum of all the fields on a line
perl -MList::Util=sum -alne 'print sum @F'

# Print the sum of all the fields on all lines
perl -MList::Util=sum -alne 'push @S,@F; END { print sum @S }'
perl -MList::Util=sum -alne '$s += sum @F; END { print $s }'

# Shuffle all fields on a line
perl -MList::Util=shuffle -alne 'print "@{[shuffle @F]}"'
perl -MList::Util=shuffle -alne 'print join " ", shuffle @F'

# Find the minimum element on a line
perl -MList::Util=min -alne 'print min @F'

# Find the minimum element over all the lines
perl -MList::Util=min -alne '@M = (@M, @F); END { print min @M }'
perl -MList::Util=min -alne '$min = min @F; $rmin = $min unless defined $rmin && $min > $rmin; END { print $rmin }'

# Find the maximum element on a line
perl -MList::Util=max -alne 'print max @F'

# Find the maximum element over all the lines
perl -MList::Util=max -alne '@M = (@M, @F); END { print max @M }'

# Replace each field with its absolute value
perl -alne 'print "@{[map { abs } @F]}"'

# Find the total number of fields (words) on each line
perl -alne 'print scalar @F'

# Print the total number of fields (words) on each line followed by the line
perl -alne 'print scalar @F, " $_"'

# Find the total number of fields (words) on all lines
perl -alne '$t += @F; END { print $t}'

# Print the total number of fields that match a pattern
perl -alne 'map { /regex/ && $t++ } @F; END { print $t }'
perl -alne '$t += /regex/ for @F; END { print $t }'
perl -alne '$t += grep /regex/, @F; END { print $t }'

# Print the total number of lines that match a pattern
perl -lne '/regex/ && $t++; END { print $t }'

# Print the number PI to n decimal places
perl -Mbignum=bpi -le 'print bpi(n)'

# Print the number PI to 39 decimal places
perl -Mbignum=PI -le 'print PI'

# Print the number E to n decimal places
perl -Mbignum=bexp -le 'print bexp(1,n+1)'

# Print the number E to 39 decimal places
perl -Mbignum=e -le 'print e'

# Print UNIX time (seconds since Jan 1, 1970, 00:00:00 UTC)
perl -le 'print time'

# Print GMT (Greenwich Mean Time) and local computer time
perl -le 'print scalar gmtime'
perl -le 'print scalar localtime'

# Print local computer time in H:M:S format
perl -le 'print join ":", (localtime)[2,1,0]'

# Print yesterday's date
perl -MPOSIX -le '@now = localtime; $now[3] -= 1; print scalar localtime mktime @now'

# Print date 14 months, 9 days and 7 seconds ago
perl -MPOSIX -le '@now = localtime; $now[0] -= 7; $now[4] -= 14; $now[7] -= 9; print scalar localtime mktime @now'

# Calculate factorial of 5
perl -MMath::BigInt -le 'print Math::BigInt->new(5)->bfac()'
perl -le '$f = 1; $f *= $_ for 1..5; print $f'

# Calculate greatest common divisor (GCM)
perl -MMath::BigInt=bgcd -le 'print bgcd(@list_of_numbers)'

# Calculate GCM of numbers 20 and 35 using Euclid's algorithm
perl -le '$n = 20; $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $m'

# Calculate least common multiple (LCM) of numbers 35, 20 and 8
perl -MMath::BigInt=blcm -le 'print blcm(35,20,8)'

# Calculate LCM of 20 and 35 using Euclid's formula: n*m/gcd(n,m)
perl -le '$a = $n = 20; $b = $m = 35; ($m,$n) = ($n,$m%$n) while $n; print $a*$b/$m'

# Generate 10 random numbers between 5 and 15 (excluding 15)
perl -le '$n=10; $min=5; $max=15; $, = " "; print map { int(rand($max-$min))+$min } 1..$n'

# Find and print all permutations of a list
perl -MAlgorithm::Permute -le '$l = [1,2,3,4,5]; $p = Algorithm::Permute->new($l); print @r while @r = $p->next'

# Generate the power set
perl -MList::PowerSet=powerset -le '@l = (1,2,3,4,5); for (@{powerset(@l)}) { print "@$_" }'

# Convert an IP address to unsigned integer
perl -le '$i=3; $u += ($_<<8*$i--) for "127.0.0.1" =~ /(\d+)/g; print $u'
perl -le '$ip="127.0.0.1"; $ip =~ s/(\d+)\.?/sprintf("%02x", $1)/ge; print hex($ip)'
perl -le 'print unpack("N", 127.0.0.1)'
perl -MSocket -le 'print unpack("N", inet_aton("127.0.0.1"))'

# Convert an unsigned integer to an IP address
perl -MSocket -le 'print inet_ntoa(pack("N", 2130706433))'
perl -le '$ip = 2130706433; print join ".", map { (($ip>>8*($_))&0xFF) } reverse 0..3'
perl -le '$ip = 2130706433; $, = "."; print map { (($ip>>8*($_))&0xFF) } reverse 0..3'


STRING CREATION AND ARRAY CREATION
----------------------------------

# Generate and print the alphabet
perl -le 'print a..z'
perl -le 'print ("a".."z")'
perl -le '$, = ","; print ("a".."z")'
perl -le 'print join ",", ("a".."z")'

# Generate and print all the strings from "a" to "zz"
perl -le 'print ("a".."zz")'
perl -le 'print "aa".."zz"'

# Create a hex lookup table
@hex = (0..9, "a".."f")

# Convert a decimal number to hex using @hex lookup table
perl -le '$num = 255; @hex = (0..9, "a".."f"); while ($num) { $s = $hex[($num%16)&15].$s; $num = int $num/16 } print $s'
perl -le '$hex = sprintf("%x", 255); print $hex'
perl -le '$num = "ff"; print hex $num'

# Generate a random 8 character password
perl -le 'print map { ("a".."z")[rand 26] } 1..8'
perl -le 'print map { ("a".."z", 0..9)[rand 36] } 1..8'

# Create a string of specific length
perl -le 'print "a"x50'

# Create a repeated list of elements
perl -le '@list = (1,2)x20; print "@list"'

# Create an array from a string
@months = split ' ', "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
@months = qw/Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec/

# Create a string from an array
@stuff = ("hello", 0..9, "world"); $string = join '-', @stuff

# Find the numeric values for characters in the string
perl -le 'print join ", ", map { ord } split //, "hello world"'

# Convert a list of numeric ASCII values into a string
perl -le '@ascii = (99, 111, 100, 105, 110, 103); print pack("C*", @ascii)'
perl -le '@ascii = (99, 111, 100, 105, 110, 103); print map { chr } @ascii'

# Generate an array with odd numbers from 1 to 100
perl -le '@odd = grep {$_ % 2 == 1} 1..100; print "@odd"'
perl -le '@odd = grep { $_ & 1 } 1..100; print "@odd"'

# Generate an array with even numbers from 1 to 100
perl -le '@even = grep {$_ % 2 == 0} 1..100; print "@even"'

# Find the length of the string
perl -le 'print length "one-liners are great"'

# Find the number of elements in an array
perl -le '@array = ("a".."z"); print scalar @array'
perl -le '@array = ("a".."z"); print $#array + 1'


TEXT CONVERSION AND SUBSTITUTION
--------------------------------

# ROT13 a string
'y/A-Za-z/N-ZA-Mn-za-m/'

# ROT 13 a file
perl -lpe 'y/A-Za-z/N-ZA-Mn-za-m/' file

# Base64 encode a string
perl -MMIME::Base64 -e 'print encode_base64("string")'
perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' file

# Base64 decode a string
perl -MMIME::Base64 -le 'print decode_base64("base64string")'
perl -MMIME::Base64 -ne 'print decode_base64($_)' file

# URL-escape a string
perl -MURI::Escape -le 'print uri_escape($string)'

# URL-unescape a string
perl -MURI::Escape -le 'print uri_unescape($string)'

# HTML-encode a string
perl -MHTML::Entities -le 'print encode_entities($string)'

# HTML-decode a string
perl -MHTML::Entities -le 'print decode_entities($string)'

# Convert all text to uppercase
perl -nle 'print uc'
perl -ple '$_=uc'
perl -nle 'print "\U$_"'

# Convert all text to lowercase
perl -nle 'print lc'
perl -ple '$_=lc'
perl -nle 'print "\L$_"'

# Uppercase only the first word of each line
perl -nle 'print ucfirst lc'
perl -nle 'print "\u\L$_"'

# Invert the letter case
perl -ple 'y/A-Za-z/a-zA-Z/'

# Camel case each line
perl -ple 's/(\w+)/\u$1/g'
perl -ple 's/(?<!['])(\w+)/\u\1/g'

# Strip leading whitespace (spaces, tabs) from the beginning of each line
perl -ple 's/^[ \t]+//'
perl -ple 's/^\s+//'

# Strip trailing whitespace (space, tabs) from the end of each line
perl -ple 's/[ \t]+$//'

# Strip whitespace from the beginning and end of each line
perl -ple 's/^[ \t]+|[ \t]+$//g'

# Convert UNIX newlines to DOS/Windows newlines
perl -pe 's|\n|\r\n|'

# Convert DOS/Windows newlines to UNIX newlines
perl -pe 's|\r\n|\n|'

# Convert UNIX newlines to Mac newlines
perl -pe 's|\n|\r|'

# Substitute (find and replace) "foo" with "bar" on each line
perl -pe 's/foo/bar/'

# Substitute (find and replace) all "foo"s with "bar" on each line
perl -pe 's/foo/bar/g'

# Substitute (find and replace) "foo" with "bar" on lines that match "baz"
perl -pe '/baz/ && s/foo/bar/'


SELECTIVE PRINTING AND DELETING OF CERTAIN LINES
------------------------------------------------

# Print the first line of a file (emulate head -1)
perl -ne 'print; exit'

# Print the first 10 lines of a file (emulate head -10)
perl -ne 'print if $. <= 10'
perl -ne '$. <= 10 && print'

# Print the last line of a file (emulate tail -1)
perl -ne '$last = $_; END { print $last }'
perl -ne 'print if eof'

# Print the last 10 lines of a file (emulate tail -10)
perl -ne 'push @a, $_; @a = @a[@a-10..$#a]; END { print @a }'

# Print only lines that match a regular expression
perl -ne '/regex/ && print'

# Print only lines that do not match a regular expression
perl -ne '!/regex/ && print'

# Print the line before a line that matches a regular expression
perl -ne '/regex/ && $last && print $last; $last = $_'

# Print the line after a line that matches a regular expression
perl -ne 'if ($p) { print; $p = 0 } $p++ if /regex/'

# Print lines that match regex AAA and regex BBB in any order
perl -ne '/AAA/ && /BBB/ && print'

# Print lines that don't match match regexes AAA and BBB
perl -ne '!/AAA/ && !/BBB/ && print'

# Print lines that match regex AAA followed by regex BBB followed by CCC
perl -ne '/AAA.*BBB.*CCC/ && print'

# Print lines that are 80 chars or longer
perl -ne 'print if length >= 80'

# Print lines that are less than 80 chars in length
perl -ne 'print if length < 80'

# Print only line 13
perl -ne '$. == 13 && print && exit'

# Print all lines except line 27
perl -ne '$. != 27 && print'
perl -ne 'print if $. != 27'

# Print only lines 13, 19 and 67
perl -ne 'print if $. == 13 || $. == 19 || $. == 67'
perl -ne 'print if int($.) ~~ (13, 19, 67)' 

# Print all lines between two regexes (including lines that match regex)
perl -ne 'print if /regex1/../regex2/'

# Print all lines from line 17 to line 30
perl -ne 'print if $. >= 17 && $. <= 30'
perl -ne 'print if int($.) ~~ (17..30)'
perl -ne 'print if grep { $_ == $. } 17..30'

# Print the longest line
perl -ne '$l = $_ if length($_) > length($l); END { print $l }'

# Print the shortest line
perl -ne '$s = $_ if $. == 1; $s = $_ if length($_) < length($s); END { print $s }'

# Print all lines that contain a number
perl -ne 'print if /\d/'

# Find all lines that contain only a number
perl -ne 'print if /^\d+$/'

# Print all lines that contain only characters
perl -ne 'print if /^[[:alpha:]]+$/

# Print every second line
perl -ne 'print if $. % 2'

# Print every second line, starting the second line
perl -ne 'print if $. % 2 == 0'

# Print all lines that repeat
perl -ne 'print if ++$a{$_} == 2'

# Print all unique lines
perl -ne 'print unless $a{$_}++'


HANDY REGULAR EXPRESSIONS
-------------------------

# Match something that looks like an IP address
/^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$/
/^(\d{1,3}\.){3}\d{1,3}$/

# Test if a number is in range 0-255
/^([0-9]|[0-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])$/

# Match an IP address
my $ip_part = qr|([0-9]|[0-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])|;
if ($ip =~ /^($ip_part\.){3}$ip_part$/) {
 say "valid ip";
}

# Check if the string looks like an email address
/.+@.+\..+/

# Check if the string is a decimal number
/^\d+$/
/^[+-]?\d+$/
/^[+-]?\d+\.?\d*$/

# Check if the string is a hexadecimal number
/^0x[0-9a-f]+$/i

# Check if the string is an octal number
/^0[0-7]+$/

# Check if the string is binary
/^[01]+$/

# Check if a word appears twice in the string
/(word).*\1/

# Increase all numbers by one in the string
$str =~ s/(\d+)/$1+1/ge

# Extract HTTP User-Agent string from the HTTP headers
/^User-Agent: (.+)$/

# Match printable ASCII characters
/[ -~]/

# Match unprintable ASCII characters
/[^ -~]/

# Match text between two HTML tags
m|<strong>([^<]*)</strong>|
m|<strong>(.*?)</strong>|

# Replace all <b> tags with <strong>
$html =~ s|<(/)?b>|<$1strong>|g

# Extract all matches from a regular expression
my @matches = $text =~ /regex/g;


PERL ONE-LINERS EXPLAINED E-BOOK
--------------------------------

I am writing an ebook based on the one-liners in this file. If you wish to
support my work and learn more about these one-liners, you can get a copy
of my ebook soon at:

    http://www.catonmat.net/blog/perl-book/

The ebook is based on the 7-part article series that I wrote on my blog.
In the ebook I reviewed all the one-liners, improved explanations and added
new ones.

You can read the original article series here:

    http://www.catonmat.net/blog/perl-one-liners-explained-part-one/
    http://www.catonmat.net/blog/perl-one-liners-explained-part-two/
    http://www.catonmat.net/blog/perl-one-liners-explained-part-three/
    http://www.catonmat.net/blog/perl-one-liners-explained-part-four/
    http://www.catonmat.net/blog/perl-one-liners-explained-part-five/
    http://www.catonmat.net/blog/perl-one-liners-explained-part-six/
    http://www.catonmat.net/blog/perl-one-liners-explained-part-seven/


CREDITS
-------

Andy Lester       http://www.petdance.com
Shlomi Fish       http://www.shlomifish.com
Madars Virza      http://www.madars.org


FOUND A BUG? HAVE ANOTHER ONE-LINER?
------------------------------------

Email bugs and new one-liners to me at peter@catonmat.net!


HAVE FUN
--------

I hope you found these one-liners useful. Have fun!

#---end of file---

Perl One-Liners Explained Article Series

If you're curious how the one-liners work, take a look at the Perl One-Liners Explained article series. I have explained all of them in the following articles:

The next, and final, thing I am doing with these one-liners is releasing Perl One-Liners Explained e-book. Stay tuned!

Update: I finished writing the e-book. Check it out!

Awk and Sed One-Liners Explained

I based the perl1line.txt file on the famous awk1line.txt and sed1line.txt files. That's how I actually learned awk and sed. I studied these two files inside out and learned everything I could about sed and awk. Later I decided to contribute back to community and explained all the one-liners in these two article series:

Take a look at these article series. The Awk one has been read over 800,000 times and the sed one over 500,000 times!

Enjoy!

Enjoy and see you later!