Another Ten One-Liners from CommandLineFu Explained

Another week and another top ten one-liners from commandlinefu explained.

This is the third post in the series already, covering one-liners 21-30. See the previous two posts for the introduction of the series and one-liners 1-20:

• Part I: Top Ten One-Liners from CommandLineFu Explained
• Part II: The Next Ten One-Liners from CommandLineFu Explained

Update: Russian translation now available.

#21. Display currently mounted file systems nicely

$ mount | column -t

The file systems are not that important here. The column -t command is what is important. It takes the input and formats it into multiple columns so that all columns were aligned vertically.

Here is how the mounted filesystem list looks without column -t command:

$ mount

/dev/root on / type ext3 (rw)
/proc on /proc type proc (rw)
/dev/mapper/lvmraid-home on /home type ext3 (rw,noatime)

And now with column -t command:

$ mount | column -t

/dev/root                 on  /      type  ext3   (rw)
/proc                     on  /proc  type  proc   (rw)
/dev/mapper/lvmraid-home  on  /home  type  ext3   (rw,noatime)

You can improve this one-liner now by also adding column titles:

$ (echo "DEVICE - PATH - TYPE FLAGS" && mount) | column -t

DEVICE                    -   PATH   -     TYPE   FLAGS
/dev/root                 on  /      type  ext3   (rw)
/proc                     on  /proc  type  proc   (rw)
/dev/mapper/lvmraid-home  on  /home  type  ext3   (rw,noatime)

Columns 2 and 4 are not really necessary. We can use awk text processing utility to get rid of them:

$ (echo "DEVICE PATH TYPE FLAGS" && mount | awk '$2=$4="";1') | column -t

DEVICE                    PATH   TYPE   FLAGS
/dev/root                 /      ext3   (rw)
/proc                     /proc  proc   (rw)
/dev/mapper/lvmraid-home  /home  ext3   (rw,noatime)

Finally, we can make it an alias so that we always enjoyed the nice output from mount. Let's call this alias nicemount:

$ nicemount() { (echo "DEVICE PATH TYPE FLAGS" && mount | awk '$2=$4="";1') | column -t; }

Let's try it out:

$ nicemount

DEVICE                    PATH   TYPE   FLAGS
/dev/root                 /      ext3   (rw)
/proc                     /proc  proc   (rw)
/dev/mapper/lvmraid-home  /home  ext3   (rw,noatime)

It works!

#22. Run the previous shell command but replace every "foo" with "bar"

$ !!:gs/foo/bar

I explained this type of one-liners in one-liner #5 already. Please take a look for a longer discussion.

To summarize, what happens here is that the !! recalls the previous executed shell command and :gs/foo/bar substitutes (the :s flag) all (the g flag) occurrences of foo with bar. The !! construct is called an event designator.

#23. Top for files

$ watch -d -n 1 'df; ls -FlAt /path'

This one-liner watches for file changes in directory /path. It uses the watch command that executes the given command periodically. The -d flag tells watch to display differences between the command calls (so you saw what files get added or removed in /path). The -n 1 flag tells it to execute the command every second.

The command to execute is df; ls -FlAt /path that is actually two commands, executed one after other. First, df outputs the filesystem disk space usage, and then ls -FlAt lists the files in /path. The -F argument to ls tells it to classify files, appending */=>@| to the filenames to indicate whether they are executables *, directories /, sockets =, doors >, symlinks @, or named pipes |. The -l argument lists all files, -A hides . and .., and -t sorts the files by time.

Special note about doors - they are Solaris thing that act like pipes, except they launch the program that is supposed to be the receiving party. A plain pipe would block until the other party opens it, but a door launches the other party itself.

Actually the output is nicer if you specify -h argument to df so it was human readable. You can also join the arguments to watch together, making them -dn1. Here is the final version:

$ watch -dn1 'df -h; ls -FlAt /path'

Another note - -d in BSD is --differences

#24. Mount a remote folder through SSH

$ sshfs name@server:/path/to/folder /path/to/mount/point

That's right, you can mount a remote directory locally via SSH! You'll first need to install two programs however:

• FUSE that allows to implement filesystems in userspace programs, and
• sshfs client that uses FUSE and sftp (secure ftp - comes with OpenSSH, and is on your system already) to access the remote host.

And that's it, now you can use sshfs to mount remote directories via SSH.

To unmount, use fusermount:

fusermount -u /path/to/mount/point

#25. Read Wikipedia via DNS

$ dig +short txt <keyword>.wp.dg.cx

This is probably the most interesting one-liner today. David Leadbeater created a DNS server, which when queried the TXT record type, returns a short plain-text version of a Wikipedia article. Here is his presentation on he did it.

Here is an example, let's find out what "hacker" means:

$ dig +short txt hacker.wp.dg.cx

"Hacker may refer to: Hacker (computer security), someone involved
in computer security/insecurity, Hacker (programmer subculture), a
programmer subculture originating in the US academia in the 1960s,
which is nowadays mainly notable for the free software/" "open
source movement, Hacker (hobbyist), an enthusiastic home computer
hobbyist http://a.vu/w:Hacker"

The one-liner uses dig, the standard sysadmin's utility for DNS troubleshooting to do the DNS query. The +short option makes it output only the returned text response, and txt makes it query the TXT record type.

This one-liner is actually alias worthy, so let's make an alias:

wiki() { dig +short txt $1.wp.dg.cx; }

Try it out:

$ wiki hacker

"Hacker may refer to: Hacker (computer security), ..."

It works!

If you don't have dig, you may also use host that also performs DNS lookups:

host -t txt hacker.wp.dg.cx

#26. Download a website recursively with wget

$ wget --random-wait -r -p -e robots=off -U Mozilla www.example.com

This one-liner does what it says. Here is the explanation of the arguments:

• --random-wait - wait between 0.5 to 1.5 seconds between requests.
• -r - turn on recursive retrieving.
• -e robots=off - ignore robots.txt.
• -U Mozilla - set the "User-Agent" header to "Mozilla". Though a better choice is a real User-Agent like "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729)".

Some other useful options are:

• --limit-rate=20k - limits download speed to 20kbps.
• -o logfile.txt - log the downloads.
• -l 0 - remove recursion depth (which is 5 by default).
• --wait=1h - be sneaky, download one file every hour.

#27. Copy the arguments of the most recent command

ALT + . (or ESC + .)

This keyboard shortcut works in shell's emacs editing mode only, it copies the last argument form the last command to the current command. Here is an example:

$ echo a b c
a b c

$ echo <Press ALT + .>
$ echo c

If you repeat the command, it copies the last argument from the command before the last, then if you repeat again, it copies the last argument from command before the command before the last, etc.

Here is an example:

$ echo 1 2 3
1 2 3
$ echo a b c
a b c

$ echo <Press ALT + .>
$ echo c

$ echo <Press ALT + .> again
$ echo 3

However, if you wish to get 1st or 2nd or n-th argument, use the digit-argument command ALT + 1 (or ESC + 1) or ALT + 2 (or ESC +2), etc. Here is an example:

$ echo a b c
a b c

$ echo <Press ALT + 1> <Press ALT + .>
$ echo a
a

$ echo <Press ALT + 2> <Press ALT + .>
$ echo b
b

See my article on Emacs Editing Mode Keyboard Shortcuts for a tutorial and a cheat sheet of all the shortcuts.

#28. Execute a command without saving it in the history

$ <space>command

This one-liner works at least on bash, I haven't tested other shells.

If you start your command by a space, it won't be saved to bash history (~/.bash_history file). This behavior is controlled by $HISTIGNORE shell variable. Mine is set to HISTIGNORE="&:[ ]*", which means don't save repeated commands to history, and don't save commands that start with a space to history. The values in $HISTIGNORE are colon-separated.

If you're interested, see my article "The Definitive Guide to Bash Command Line History" for a short tutorial on how to work with shell history and a summary cheat sheet.

#29. Show the size of all sub folders in the current directory

$ du -h --max-depth=1

The --max-depth=1 causes du to summarize disk usage statistics for directories that are depth 1 from the current directory, that is, all directories in the current directory. The -h argument makes the summary human-readable, that is, displays 5MB instead of 5242880 (bytes).

If you are interested in both sub folder size and file size in the current directory, you can use the shorter:

$ du -sh *

#30. Display the top ten running processes sorted by memory usage

$ ps aux | sort -nk +4 | tail

This is certainly not the best way to display the top ten processes that consume the most memory, but, hey, it works.

It takes the output of ps aux, sorts it by 4th column numerically and then uses tail to output the last then lines which happen to be the processes with the biggest memory consumption.

If I was to find out who consumes the most memory, I'd simply use htop or top and not ps.

Bonus one-liner: Start an SMTP server

python -m smtpd -n -c DebuggingServer localhost:1025

This one-liner starts an SMTP server on port 1025. It uses Python's standard library smtpd (specified by -m smtpd) and passes it three arguments - -n, -c DebuggingServer and localhost:1025.

The -n argument tells Python not to setuid (change user) to "nobody" - it makes the code run under your user.

The -c DebuggingServer argument tells Python to use DebuggingServer class as the SMTP implementation that prints each message it receives to stdout.

The localhost:1025 argument tells Python to start the SMTP server on locahost, port 1025.

However, if you wish to start it on the standard port 25, you'll have to use sudo command, because only root is allowed to start services on ports 1-1024. These are also known as privileged ports.

sudo python -m smtpd -n -c DebuggingServer localhost:25

This one-liner was coined by Evan Culver. Thanks to him!

That's it for today,

but be sure to come back the next time for "Yet Another Ten One-Liners from CommandLineFu Explained!"