This article is part of the article series "CommandLineFu One-Liners Explained."
<- previous article next article ->
CommandLineFu Explained

Another week and another top ten one-liners from commandlinefu explained.

This is the third post in the series already, covering one-liners 21-30. See the previous two posts for the introduction of the series and one-liners 1-20:

Update: Russian translation now available.

#21. Display currently mounted file systems nicely

$ mount | column -t

The file systems are not that important here. The column -t command is what is important. It takes the input and formats it into multiple columns so that all columns were aligned vertically.

Here is how the mounted filesystem list looks without column -t command:

$ mount

/dev/root on / type ext3 (rw)
/proc on /proc type proc (rw)
/dev/mapper/lvmraid-home on /home type ext3 (rw,noatime)

And now with column -t command:

$ mount | column -t

/dev/root                 on  /      type  ext3   (rw)
/proc                     on  /proc  type  proc   (rw)
/dev/mapper/lvmraid-home  on  /home  type  ext3   (rw,noatime)

You can improve this one-liner now by also adding column titles:

$ (echo "DEVICE - PATH - TYPE FLAGS" && mount) | column -t

DEVICE                    -   PATH   -     TYPE   FLAGS
/dev/root                 on  /      type  ext3   (rw)
/proc                     on  /proc  type  proc   (rw)
/dev/mapper/lvmraid-home  on  /home  type  ext3   (rw,noatime)

Columns 2 and 4 are not really necessary. We can use awk text processing utility to get rid of them:

$ (echo "DEVICE PATH TYPE FLAGS" && mount | awk '$2=$4="";1') | column -t

DEVICE                    PATH   TYPE   FLAGS
/dev/root                 /      ext3   (rw)
/proc                     /proc  proc   (rw)
/dev/mapper/lvmraid-home  /home  ext3   (rw,noatime)

Finally, we can make it an alias so that we always enjoyed the nice output from mount. Let's call this alias nicemount:

$ nicemount() { (echo "DEVICE PATH TYPE FLAGS" && mount | awk '$2=$4="";1') | column -t; }

Let's try it out:

$ nicemount

DEVICE                    PATH   TYPE   FLAGS
/dev/root                 /      ext3   (rw)
/proc                     /proc  proc   (rw)
/dev/mapper/lvmraid-home  /home  ext3   (rw,noatime)

It works!

#22. Run the previous shell command but replace every "foo" with "bar"

$ !!:gs/foo/bar

I explained this type of one-liners in one-liner #5 already. Please take a look for a longer discussion.

To summarize, what happens here is that the !! recalls the previous executed shell command and :gs/foo/bar substitutes (the :s flag) all (the g flag) occurrences of foo with bar. The !! construct is called an event designator.

#23. Top for files

$ watch -d -n 1 'df; ls -FlAt /path'

This one-liner watches for file changes in directory /path. It uses the watch command that executes the given command periodically. The -d flag tells watch to display differences between the command calls (so you saw what files get added or removed in /path). The -n 1 flag tells it to execute the command every second.

The command to execute is df; ls -FlAt /path that is actually two commands, executed one after other. First, df outputs the filesystem disk space usage, and then ls -FlAt lists the files in /path. The -F argument to ls tells it to classify files, appending */=>@| to the filenames to indicate whether they are executables *, directories /, sockets =, doors >, symlinks @, or named pipes |. The -l argument lists all files, -A hides . and .., and -t sorts the files by time.

Special note about doors - they are Solaris thing that act like pipes, except they launch the program that is supposed to be the receiving party. A plain pipe would block until the other party opens it, but a door launches the other party itself.

Actually the output is nicer if you specify -h argument to df so it was human readable. You can also join the arguments to watch together, making them -dn1. Here is the final version:

$ watch -dn1 'df -h; ls -FlAt /path'

Another note - -d in BSD is --differences

#24. Mount a remote folder through SSH

$ sshfs name@server:/path/to/folder /path/to/mount/point

That's right, you can mount a remote directory locally via SSH! You'll first need to install two programs however:

  • FUSE that allows to implement filesystems in userspace programs, and
  • sshfs client that uses FUSE and sftp (secure ftp - comes with OpenSSH, and is on your system already) to access the remote host.

And that's it, now you can use sshfs to mount remote directories via SSH.

To unmount, use fusermount:

fusermount -u /path/to/mount/point

#25. Read Wikipedia via DNS

$ dig +short txt <keyword>.wp.dg.cx

This is probably the most interesting one-liner today. David Leadbeater created a DNS server, which when queried the TXT record type, returns a short plain-text version of a Wikipedia article. Here is his presentation on he did it.

Here is an example, let's find out what "hacker" means:

$ dig +short txt hacker.wp.dg.cx

"Hacker may refer to: Hacker (computer security), someone involved
in computer security/insecurity, Hacker (programmer subculture), a
programmer subculture originating in the US academia in the 1960s,
which is nowadays mainly notable for the free software/" "open
source movement, Hacker (hobbyist), an enthusiastic home computer
hobbyist http://a.vu/w:Hacker"

The one-liner uses dig, the standard sysadmin's utility for DNS troubleshooting to do the DNS query. The +short option makes it output only the returned text response, and txt makes it query the TXT record type.

This one-liner is actually alias worthy, so let's make an alias:

wiki() { dig +short txt $1.wp.dg.cx; }

Try it out:

$ wiki hacker

"Hacker may refer to: Hacker (computer security), ..."

It works!

If you don't have dig, you may also use host that also performs DNS lookups:

host -t txt hacker.wp.dg.cx

#26. Download a website recursively with wget

$ wget --random-wait -r -p -e robots=off -U Mozilla www.example.com

This one-liner does what it says. Here is the explanation of the arguments:

  • --random-wait - wait between 0.5 to 1.5 seconds between requests.
  • -r - turn on recursive retrieving.
  • -e robots=off - ignore robots.txt.
  • -U Mozilla - set the "User-Agent" header to "Mozilla". Though a better choice is a real User-Agent like "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729)".

Some other useful options are:

  • --limit-rate=20k - limits download speed to 20kbps.
  • -o logfile.txt - log the downloads.
  • -l 0 - remove recursion depth (which is 5 by default).
  • --wait=1h - be sneaky, download one file every hour.

#27. Copy the arguments of the most recent command

ALT + . (or ESC + .)

This keyboard shortcut works in shell's emacs editing mode only, it copies the last argument form the last command to the current command. Here is an example:

$ echo a b c
a b c

$ echo <Press ALT + .>
$ echo c

If you repeat the command, it copies the last argument from the command before the last, then if you repeat again, it copies the last argument from command before the command before the last, etc.

Here is an example:

$ echo 1 2 3
1 2 3
$ echo a b c
a b c

$ echo <Press ALT + .>
$ echo c

$ echo <Press ALT + .> again
$ echo 3

However, if you wish to get 1st or 2nd or n-th argument, use the digit-argument command ALT + 1 (or ESC + 1) or ALT + 2 (or ESC +2), etc. Here is an example:

$ echo a b c
a b c

$ echo <Press ALT + 1> <Press ALT + .>
$ echo a
a

$ echo <Press ALT + 2> <Press ALT + .>
$ echo b
b

See my article on Emacs Editing Mode Keyboard Shortcuts for a tutorial and a cheat sheet of all the shortcuts.

#28. Execute a command without saving it in the history

$ <space>command

This one-liner works at least on bash, I haven't tested other shells.

If you start your command by a space, it won't be saved to bash history (~/.bash_history file). This behavior is controlled by $HISTIGNORE shell variable. Mine is set to HISTIGNORE="&:[ ]*", which means don't save repeated commands to history, and don't save commands that start with a space to history. The values in $HISTIGNORE are colon-separated.

If you're interested, see my article "The Definitive Guide to Bash Command Line History" for a short tutorial on how to work with shell history and a summary cheat sheet.

#29. Show the size of all sub folders in the current directory

$ du -h --max-depth=1

The --max-depth=1 causes du to summarize disk usage statistics for directories that are depth 1 from the current directory, that is, all directories in the current directory. The -h argument makes the summary human-readable, that is, displays 5MB instead of 5242880 (bytes).

If you are interested in both sub folder size and file size in the current directory, you can use the shorter:

$ du -sh *

#30. Display the top ten running processes sorted by memory usage

$ ps aux | sort -nk +4 | tail

This is certainly not the best way to display the top ten processes that consume the most memory, but, hey, it works.

It takes the output of ps aux, sorts it by 4th column numerically and then uses tail to output the last then lines which happen to be the processes with the biggest memory consumption.

If I was to find out who consumes the most memory, I'd simply use htop or top and not ps.

Bonus one-liner: Start an SMTP server

python -m smtpd -n -c DebuggingServer localhost:1025

This one-liner starts an SMTP server on port 1025. It uses Python's standard library smtpd (specified by -m smtpd) and passes it three arguments - -n, -c DebuggingServer and localhost:1025.

The -n argument tells Python not to setuid (change user) to "nobody" - it makes the code run under your user.

The -c DebuggingServer argument tells Python to use DebuggingServer class as the SMTP implementation that prints each message it receives to stdout.

The localhost:1025 argument tells Python to start the SMTP server on locahost, port 1025.

However, if you wish to start it on the standard port 25, you'll have to use sudo command, because only root is allowed to start services on ports 1-1024. These are also known as privileged ports.

sudo python -m smtpd -n -c DebuggingServer localhost:25

This one-liner was coined by Evan Culver. Thanks to him!

That's it for today,

but be sure to come back the next time for "Yet Another Ten One-Liners from CommandLineFu Explained!"

This article is part of the article series "CommandLineFu One-Liners Explained."
<- previous article next article ->

Comments

These tips were great! Just the perfect bite-size format and usefulness to be helpful in getting me to learn bash. Thanks :)

April 21, 2010, 05:21

You're welcome! :)

April 20, 2010, 23:50

Absolutely excellent one-liners. I've been using Linux for a long time, and it still delights me when I see a command or option I didn't know. Somehow all these years I've missed the 'column' command. And the max-depth option to du? Genius. Thanks for putting this together

April 21, 2010, 05:22

You're also welcome!

Romans Permalink
April 21, 2010, 01:29

Perfect, tnx!

Jay Permalink
April 21, 2010, 04:40

A few problems with these one-liners:
not every 'mount' formats options like (opt1,opt2), some use (opt1, opt2), which impacts column -t output on filesystems with lots of options;
the event indicator !! is a bash-ism;
watch seems to be linux/GNU only, not available elsewhere, and on FreeBSD it is a totally different command:
watch(8) - snoop on another tty line;
--max-depth is a GNU du extension;
sort's -k option takes a initial number, i.e. '4', not '+4' which is an invalid field specification (but I guess it works with GNU sort).

April 21, 2010, 04:51

Great comments. I'll incorporate them in the article! Thank you! And if you ever read my future posts, I'd appreciate more comments of this style!

FlatCap Permalink
April 21, 2010, 04:50

> The values in $HISTIGNORE are column-separated.

Typo. They're colon-separated.

April 21, 2010, 05:20

Thanks for spotting. Fixed now!

bluszcz Permalink
April 21, 2010, 05:06

About SMTP, I found one very useful for HTTP:

python -m SimpleHTTPServer [port_number]

It's serving locadirectory with index.

April 21, 2010, 05:19

Yep, explained it in part one, one-liner #2 already. :)

bluszcz Permalink
April 21, 2010, 10:46

Haha, I had no idea.

April 21, 2010, 09:21

Thx Peter for next part from Linux shells one-line series :) ... really great. Good luck!

April 21, 2010, 16:37

Thank you!

April 21, 2010, 10:50

I have used du -sh * in the past for listing the sizes of the subdirectories and files under the current directory. The same effect (of listing files too) can be achieved with the given --max-depth argument by adding --all also (du -s --all --max-depth=1), but we are lazy and less typing is better, isn't it? :)

April 21, 2010, 16:37

Make it an alias then!

alias dum='du --max-depth=1'
alias duma='dum -a'
NitinS Permalink
April 21, 2010, 10:56

Loved your article, especially #28. Execute a command without saving it in the history

April 21, 2010, 16:38

Happy to be of service. :)

April 21, 2010, 15:53

sshfs is going to save me so much time. Great work. when's your book coming out? :)

April 21, 2010, 16:35

I have several planned, I can't say precisely when, because I haven't yet focused exclusively on putting them together. But if you stick around, you'll know. :)

Mark Permalink
April 21, 2010, 19:31

Thanks, some good tips here. I know ssh pretty well but I have not seen sshfs before. That's very handy.

April 21, 2010, 20:56

I am glad you like them!

April 22, 2010, 10:18

Nice article! I was wondering what the default values are for $HISTIGNORE. This is not defined by default on my system, (Debain Sid), but does ignore a space by default. I was wondering if there are any other characters ignored by default without having to dig through the source. Can't find any documentation.

Thanks!

Rick

April 22, 2010, 23:07

It's not defined (empty) by default! That's why it ignored a space.

April 22, 2010, 12:01

One of my favorites: to edit any env variable

print -s "PATH=$PATH"

then edit the last command (zsh with vi keybindings): ESC-

April 22, 2010, 12:08

Almost forgot the follow up function to make it easier:
foo()
{
eval "print -s ${1?no variable}=\'\$$1\'"
}

then do this:
X='xyzzy plugh'

foo X

then edit your history

April 22, 2010, 23:06

Hmm.. Can you explain this a bit more? I don't understand it.

May 19, 2010, 13:21

What he's doing is first he sets PATH to itself (eval PATH=$PATH), which leaves him with something like
PATH=/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin
in his history. Then he just uses the editor to modify this and run it again, thus setting PATH to the modified value.

nima0102 Permalink
April 23, 2010, 12:26

Thanks a looooooooooooooooooooooooooooooooooooot :)

April 24, 2010, 02:09

You're welcooooooooooooooooome!

jklowden Permalink
April 23, 2010, 15:21

wiki() { host -t txt "$1.wp.dg.cx"; }

You need to quote the servername if the argument can contain whitespace. Cf. "wiki 'mad men' ".

April 23, 2010, 23:26

This is better -- replace all whitespace with _

wiki() { dig +short txt $(sed 's/ /_/g'<<<$*).wp.dg.cx; }
_pingu Permalink
April 25, 2010, 11:11

I've tried:
#27. Copy the arguments of the most recent command

But it doesn't work.

If I do:
$ echo ESC 1
I get:
(arg: 1)

But I do not get the a like expected. Why?
I've read your other examples for emacs editing mode.They work fine.

April 25, 2010, 13:08

Try:

echo ESC 1 ESC .
_pingu Permalink
April 25, 2010, 13:57

Ok that works. Thanks!

kangu Permalink
April 27, 2010, 08:10

Admirable! thanks Peter!

freenode/yitz_ Permalink
April 28, 2010, 04:24

#21
The ( ) force a new subshell, where { } doesn't.
The && test seems pointless. I'd use ;
nicemount() is a function, not an alias. It could be made an alias.
The echo stuff can be put into awk with a BEGIN{} block and a print.

mount | awk 'BEGIN{ print "DEVICE PATH TYPE FLAGS" } { print $1, $3, $5, $6 }' | column -t

April 28, 2010, 08:53

To show the size of all subfolders, but to sort them by size, I often do :

du -sk | sort -n

the "-k" show results in kilobytes (maybe you'd prefer '-m' for megabytes)

the "sort -n" is to sort the list (taking care of numerals).

May 28, 2010, 20:34

Thanks a lot, especially for #26

Bastiaan Permalink
June 11, 2010, 08:11

#23. Top for files: I rather use inotify if available

Bastiaan Permalink
June 11, 2010, 08:12

ps. great explains btw ;-) forgot to say that!

argv Permalink
July 16, 2010, 08:06

someone actually did a DNS-based reverse polish notation calculator using the TXT RR. with EDNS, if the servers supprt it, the possibilities don't stop there. i love UDP. it's just fast and simple.

speaking of RPN, will you ever write about FORTH peteris?

July 18, 2010, 05:14

How did that calculator work?

I have not planned to write about FORTH at the moment because I haven't ever played with the language. But I may do it sometime and then write about it.

argv Permalink
July 30, 2010, 06:37

I'll tell you if you write an article on FORTH. Deal?

I promise it will not be a waste of time at all. Terse. Stack machine. Scriptable. Very powerful. And it probably will give you ideas.

My understanding is that FORTH will be part of the laptops to be given to underprivileged children in developing nations. If kids start learning FORTH, it could change everything. FORTH is, in effect, "assemblylanguagescript". It is as malleable as is possible. As we know, all other languages are just ways of avoiding the tedium of assembly. But if young people start using assembly again (i.e. "rediscover it" through FORTH), all other languages will be inevitably viewed as slow and limited. HL languages cannot compete with well-written assembly.

There are over *2000* computer languages and still more being created. Hardware performance acclerates via Moore's Law but software lags behind. Much of the power of today's CPU's is not utilised. How long can this go on?

Your writing is lucid and uncoloured by bias. Please write about FORTH.

June 21, 2011, 09:15

Display top CPU user

ps aux | sort -n +2 | tail -1
just adding minus one :D

January 16, 2014, 05:02

nice post..
#28. Execute a command without saving it in the history

from,,
Kenshusei/japan intership progam

January 16, 2014, 05:02

nice post..
#28. Execute a command without saving it in the history

from,,
Kenshusei/japan intership progam

January 16, 2014, 05:05

These tips were great!
Thanks for putting this together,,

January 16, 2014, 05:05

These tips were great!
Thanks for putting this together,,

January 16, 2014, 05:05

These tips were great!
Thanks for putting this together,,

January 16, 2014, 05:09

How do I find out size of a directory on Linux operating systems using command line options?

August 05, 2014, 15:36

I have not planned to write about FORTH at the moment because I haven't ever played with the language.

from

very great post , helped me more
thanks for sharing

Leave a new comment

(why do I need your e-mail?)

(Your twitter name, if you have one. (I'm @pkrumins, btw.))

Type the word "quake3_212": (just to make sure you're a human)

Please preview the comment before submitting to make sure it's OK.

Advertisements