The Definitive Guide to Bash Command Line History

Let me teach you how to work efficiently with command line history in bash.

This tutorial comes with a downloadable cheat sheet that summarizes (and expands on) topics covered here (scroll to the end for a download link).

In case you are a first time reader, this is the 3rd part of the article series on working efficiently in bourne again shell. Previously I have written on how to work efficiently in vi and emacs command editing modes by using predefined keyboard shortcuts (both articles come with cheat sheets of predefined shortcuts).

First, lets review some basic keyboard shortcuts for navigating around previously typed commands.

As you remember, bash offers two modes for command editing - emacs mode and vi mode. In each of these editing modes the shortcuts for retrieving history are different.

Suppose you had executed the following commands:

$ echo foo bar baz
$ iptables -L -n -v -t nat
$ ... lots and lots more commands
$ echo foo foo foo
$ perl -wle 'print q/hello world/'
$ awk -F: '{print$1}' /etc/passwd
$

and you wanted to execute the last command (awk -F …).

You could certainly hit the up arrow and live happily along, but do you really want to move your hand that far away?

If you are in emacs mode just try CTRL-p which fetches the previous command from history list (CTRL-n for the next command).

In vi mode try CTRL-[ (or ESC) (to switch to command mode) and ‘h‘ (’j‘ for the next command).

There is another, equally quick, way to do that by using bash’s history expansion mechanism - event designators. Typing ‘!!‘ will execute the previous command (more about event designators later).

Now, suppose that you wanted to execute ‘iptables -L -n -v -t nat‘ command again without retyping it.

A naive user would, again, just keep hitting up-arrow key until he/she finds the command. But that’s not the way hackers work. Hackers love to work quickly and efficiently. Forget about arrow keys and page-up, page-down, home and end keys. They are completely useless and, as I said, they are too far off from the main part of the keyboard anyway.

In emacs mode try CTRL-r and type a few first letters of ‘iptables‘, like ‘ipt‘. That will display the last iptables command you executed. In case you had more than one iptables commands executed in between, hitting CTRL-r again will display older entries. In case you miss the right command and move too deep into history list, you can reverse the search direction by hitting CTRL-s (don’t forget that by default CTRL-s stops the output to the terminal and you’ll get an effect of “frozen” terminal (hit CTRL-q to “unfreeze”), see stty command to change this behavior).

In vi mode the same CTRL-r and CTRL-s still work but there is another way more specific to vi mode.
Switch to command mode by hitting CTRL-[ or ESC and hit ‘/‘, then type a first few characters of ‘iptables’ command, like ‘ipt’ and hit return. Bash will display the most recent match found in history. To navigate around use ‘n‘ or just plain ‘/‘ to repeat the search in the same direction, and ‘N‘ or ‘?‘ to repeat the search in opposite direction!

With event designators you may execute only the most recently executed command matching (or starting with) ’string’.

Try ‘!iptables‘ history expansion command which refers to the most recent command starting with ‘iptables’.

Another way is to use bash’s built in ‘history‘ command then grep for a string of interest and finally use an event designator in form ‘!N‘, where N is an integer which refers to N-th command in command history list.

For example,

$ history | grep 'ipt'
  2    iptables -L -n -v -t nat
$ !2     # will execute the iptables command

I remembered another way to execute N-th command in history list in vi editing mode. Type ‘N‘ (command number) and then ‘G‘, in this example ‘2G‘

Listing and Erasing Command History

Bash provides a built-in command ‘history‘ for viewing and erasing command history.

Suppose that we are still working with the same example:

$ echo foo bar baz
$ iptables -L -n -v -t nat
$ ... lots and lots more commands
$ echo foo foo foo
$ perl -wle 'print q/hello world/'
$ awk -F: '{print$1}' /etc/passwd
$

Typing ‘history‘ will display all the commands in bash history alongside with line numbers:

  1    echo foo bar baz
  2    iptables -L -n -v -t nat
  ...  lots and lots more commands
  568  echo foo foo foo
  569  perl -wle 'print q/hello world/'
  570  awk -F: '{print$1}' /etc/passwd

Typing ‘history N‘, where N is an integer, will display the last N commands in the history.
For example, ‘history 3‘ will display:

  568  echo foo foo foo
  569  perl -wle 'print q/hello world/'
  570  awk -F: '{print$1}' /etc/passwd

history -c will clear the history list and history -d N will delete a history entry N.

By default, the history list is kept in user’s home directory in a file ‘.bash_history‘.

History Expansion

History expansion is done via so-called event designators and word designators. Event designators can be used to recall previously executed commands (events) and word designators can be used to extract command line arguments from the events. Optionally, various modifiers can be applied to the extracted arguments.

Event designators are special commands that begin with a ‘!‘ (there is also one that begins with a ‘^‘), they may follow a word designator and one or more modifiers. Event designators, word designators and modifiers are separated by a colon ‘:‘.

Event Designators

Lets look at a couple of examples to see how the event designators work.

Event designator ‘!!‘ can be used to refer to the previous command, for example,

$ echo foo bar baz
foo bar baz
$ !!
foo bar baz

Here the ‘!!‘ executed the previous ‘echo foo bar baz‘ command.

Event designator ‘!N‘ can be used to refer to the N-th command.
Suppose you listed the history and got the following output:

  1    echo foo foo foo
  2    iptables -L -n -v -t nat
  ...  lots and lots more commands
  568  echo bar bar bar
  569  perl -wle 'print q/hello world/'
  570  awk -F: '{print$1}' /etc/passwd

Then the event designator ‘!569‘ will execute ‘perl …‘ command, and ‘!1‘ will execute ‘echo foo foo foo‘ command!

Event designator ‘!-N‘ refers to current command line minus N. For example,

$ echo foo bar baz
foo bar baz
$ echo a b c d e
a b c d e
$ !-2
foo bar baz

Here the event designator ‘!-2‘ executed a one before the previous command, or current command line minus 2.

Event designator ‘!string‘ refers to the most recent command starting with ‘string‘. For example,

$ awk --help
$ perl --help

Then the event designator ‘!p‘ or ‘!perl‘ or ‘!per‘ will execute the ‘perl –help‘ command. Similarly, ‘!a‘ will execute the awk command.

An event designator ‘!?string?‘ refers to a command line containing (not necessarily starting with) ‘string‘.

Perhaps the most interesting event designator is the one in form ‘^string1^string2^‘ which takes the last command, replaces string1 with string2 and executes it. For example,

$ ehco foo bar baz
bash: ehco: command not found
$ ^ehco^echo^
foo bar baz

Here the ‘^ehco^echo^‘ designator replaced the incorrectly typed ‘ehco‘ command with the correct ‘echo‘ command and executed it.

Word Designators and Modifiers

Word designators follow event designators separated by a colon. They are used to refer to some or all of the parameters on the command referenced by event designator.

For example,

$ echo a b c d e
a b c d e
$ echo !!:2
b

This is the simplest form of a word designator. ‘:2‘ refers to the 2nd argument of the command (3rd word). In general ‘:N‘ refers to Nth argument of the command ((N+1)-th word).

Word designators also accept ranges, for example,

$ echo a b c d e
a b c d e
$ echo !!:3-4
c d

There are various shortcuts, such as, ‘:$‘ to refer to the last argument, ‘:^‘ to refer to the first argument, ‘:*‘ to refer to all the arguments (synonym to ‘:1-$‘), and others. See the cheat sheet for a complete list.

Modifiers can be used to modify the behavior of a word designators. For example:

$ tar -xvzf software-1.0.tgz
software-1.0/file
...
$ cd !!:$:r
software-1.0$

Here the ‘r‘ modifier was applied to a word designator which picked the last argument from the previous command line. The ‘r‘ modifier removed the trailing suffix ‘.tgz’.

The ‘h‘ modifier removes the trailing pathname component, leaving the head:

$ echo /usr/local/apache
/usr/local/apache
$ echo !!:$:h
/usr/local

The ‘e‘ modifier removes all but the trailing suffix:

$ ls -la /usr/src/software-4.2.messy-Extension
...
$ echo /usr/src/*!!:$:e
/usr/src/*.messy-Extension    # ls could have been used instead of echo

Another interesting modifier is the substitute ‘:s/old/new/‘ modifier which substitutes new for old. It can be used in conjunction with ‘g‘ modifier to do global substitution. For example,

$ ls /urs/local/software-4.2 /urs/local/software-4.3
/usr/bin/ls: /urs/local/software-4.2: No such file or directory
/usr/bin/ls: /urs/local/software-4.3: No such file or directory
$ !!:gs/urs/usr/
...

This example replaces all occurances of ‘urs’ to ‘usr’ and makes the command correct.

There are a few other modifiers, such as ‘p‘ modifier which prints the resulting command after history expansion but does not execute it. See the cheat sheet for all of the modifiers.

Modifying History Behavior

Bash allows you to modify which commands get stored in the history list, the file where they get stored, the number of commands that get stored, and a few other options.

These options are controlled by setting HISTFILE, HISTFILESIZE, HISTIGNORE and HISTSIZE environment variables.

HISTFILE, as the name suggests, controls where the history file gets saved.
For example,

$ export HISTFILE=/home/pkrumins/todays_history

will save the commands to a file /home/pkrumins/todays_history

Set it to /dev/null or unset it to avoid getting your history list saved.

HISTFILESIZE controls how many history commands to keep in HISTFILE.
For example,

$ export HISTFILESIZE=1000

will keep the last 1000 history commands.

HISTSIZE controls how many history commands to keep in the history list of current session.
For example,

$ export HISTSIZE=42

will keep 42 last commands in the history of current session.

If this number is less than HISTFILESIZE, only that many commands will get written to HISTFILE.

HISTIGNORE controls the items which get ignored and do not get saved. This variable takes a list of colon separated patterns. Pattern ‘&’ (ampersand) is special in a sense that it matches the previous history command.

There is a trick to make history ignore the commands which begin with a space. The pattern for that is “[ ]*”

For example,

$ export HISTIGNORE="&:[ ]*:exit"

will make bash ignore duplicate commands, commands that begin with a space, and the ‘exit’ command.

There are several other options of interest controlled by the built-in ‘shopt‘ command.

The options may be set by specifying ‘-s‘ parameter to the ‘shopt‘ command, and may be unset by specifying ‘-u‘ parameter.

Option ‘histappend‘ controls how the history list gets written to HISTFILE, setting the option will append history list of current session to HISTFILE, unsetting it (default) will make HISTFILE get overwritten each time.

For example, to set this option, type:

$ shopt -s histappend

And to unset it, type:

$ shopt -u histappend

Option ‘histreedit‘ allows users to re-edit a failed history substitution.

For example, suppose you had typed:

$ echo foo bar baz

and wanted to substitute ‘baz’ for ‘test’ with the ^baz^test^ event designator , but you made a mistake and typed ^boo^test^. This would lead to a substitution failure because the previous command does not contain string ‘boo’.

If you had this option turned on, bash would put the erroneous ^baz^test^ event designator back on the command line as if you had typed it again.

Finally, option ‘histverify‘ allows users to verify a substituted history expansion.

Based on the previous example, suppose you wanted to execute that ‘echo’ command again by using the ‘!!’ event designator. If you had this option on, bash would not execute the ‘echo’ command immediately but would first put it on command line so that you could see if it had made the correct substitution.

Tuning the Command Prompt

Here is how my command prompt looks:

Wed Jan 30@07:07:03
pkrumins@catonmat:1002:2:~$

The first line displays the date and time the command prompt was displayed so I could keep track of commands back in time.
The second line displays username, hostname, global history number and current command number.

The global history number allows me to quickly use event designators.

My PS1, primary prompt display variable looks like this:

PS1='\d@\t\n\u@\h:\!:\#:\w$ '

Bash History Cheat Sheet

Here is a summary cheat sheet for working effectively with bash history.

This cheat sheet includes:

• History editing keyboard shortcuts (emacs and vi mode),
• History expansion summary - event designators, word designators and modifiers,
• Shell variables and `shopt’ options to modify history behavior,
• Examples