This article is part of the article series "Bash One-Liners Explained."
<- previous article next article ->

This is the second part of the Bash One-Liners Explained article series. In this part I'll show you how to do various string manipulations with bash. I'll use only the best bash practices, various bash idioms and tricks. I want to illustrate how to get various tasks done with just bash built-in commands and bash programming language constructs.

See the first part of the series for introduction. After I'm done with the series I'll release an ebook (similar to my ebooks on awk, sed, and perl), and also bash1line.txt (similar to my perl1line.txt).

Also see my other articles about working fast in bash from 2007 and 2008:

Let's start.

Part II: Working With Strings

1. Generate the alphabet from a-z

$ echo {a..z}

This one-liner uses brace expansion. Brace expansion is a mechanism for generating arbitrary strings. This one-liner uses a sequence expression of the form {x..y}, where x and y are single characters. The sequence expression expands to each character lexicographically between x and y, inclusive.

If you run it, you get all the letters from a-z:

$ echo {a..z}
a b c d e f g h i j k l m n o p q r s t u v w x y z

2. Generate the alphabet from a-z without spaces between characters

$ printf "%c" {a..z}

This is an awesome bash trick that 99.99% bash users don't know about. If you supply a list of items to the printf function it actually applies the format in a loop until the list is empty! printf as a loop! There is nothing more awesome than that!

In this one-liner the printf format is "%c", which means "a character" and the arguments are all letters from a-z separated by space. So what printf does is it iterates over the list outputting each character after character until it runs out of letters.

Here is the output if you run it:


This output is without a terminating newline because the format string was "%c" and it doesn't include \n. To have it newline terminated, just add $'\n' to the list of chars to print:

$ printf "%c" {a..z} $'\n'

$'\n' is bash idiomatic way to represent a newline character. printf then just prints chars a to z, and the newline character.

Another way to add a trailing newline character is to echo the output of printf:

$ echo $(printf "%c" {a..z})

This one-liner uses command substitution, which runs printf "%c" {a..z} and replaces the command with its output. Then echo prints this output and adds a newline itself.

Want to output all letters in a column instead? Add a newline after each character!

$ printf "%c\n" {a..z}



Want to put the output from printf in a variable quickly? Use the -v argument:

$ printf -v alphabet "%c" {a..z}

This puts abcdefghijklmnopqrstuvwxyz in the $alphabet variable.

Similarly you can generate a list of numbers. Let's say from 1 to 100:

$ echo {1..100}


1 2 3 ... 100

Alternatively, if you forget this method, you can use the external seq utility to generate a sequence of numbers:

$ seq 1 100

3. Pad numbers 0 to 9 with a leading zero

$ printf "%02d " {0..9}

Here we use the looping abilities of printf again. This time the format is "%02d ", which means "zero pad the integer up to two positions", and the items to loop through are the numbers 0-9, generated by the brace expansion (as explained in the previous one-liner).


00 01 02 03 04 05 06 07 08 09

If you use bash 4, you can do the same with the new feature of brace expansion:

$ echo {00..09}

Older bashes don't have this feature.

4. Produce 30 English words

$ echo {w,t,}h{e{n{,ce{,forth}},re{,in,fore,with{,al}}},ither,at}

This is an abuse of brace expansion. Just look at what this produces:

when whence whenceforth where wherein wherefore wherewith wherewithal whither what then thence thenceforth there therein therefore therewith therewithal thither that hen hence henceforth here herein herefore herewith herewithal hither hat

Crazy awesome!

Here is how it works - you can produce permutations of words/symbols with brace expansion. For example, if you do this,

$ echo {a,b,c}{1,2,3}

It will produce the result a1 a2 a3 b1 b2 b3 c1 c2 c3. It takes the first a, and combines it with {1,2,3}, producing a1 a2 a3. Then it takes b and combines it with {1,2,3}, and then it does the same for c.

So this one-liner is just a smart combination of braces that when expanded produce all these English words!

5. Produce 10 copies of the same string

$ echo foo{,,,,,,,,,,}

This one-liner uses the brace expansion again. What happens here is foo gets combined with 10 empty strings, so the output is 10 copies of foo:

foo foo foo foo foo foo foo foo foo foo foo

6. Join two strings

$ echo "$x$y"

This one-liner simply concatenates two variables together. If the variable x contains foo and y contains bar then the result is foobar.

Notice that "$x$y" were quoted. If we didn't quote it, echo would interpret the $x$y as regular arguments, and would first try to parse them to see if they contain command line switches. So if $x contains something beginning with -, it would be a command line argument rather than an argument to echo:

y=" foo"
echo $x$y



Versus the correct way:

y=" foo"
echo "$x$y"


-n foo

If you need to put the two joined strings in a variable, you can omit the quotes:


7. Split a string on a given character

Let's say you have a string foo-bar-baz in the variable $str and you wish to split it on the dash and iterate over it. You can simply combine IFS with read to do it:

$ IFS=- read -r x y z <<< "$str"

Here we use the read x command that reads data from stdin and puts the data in the x y z variables. We set IFS to - as this variable is used for field splitting. If multiple variable names are specified to read, IFS is used to split the line of input so that each variable gets a single field of the input.

In this one-liner $x gets foo, $y gets bar, $z gets baz.

Also notice the use of <<< operator. This is the here-string operator that allows strings to be passed to stdin of commands easily. In this case string $str is passed as stdin to read.

You can also put the split fields and put them in an array:

$ IFS=- read -ra parts <<< "foo-bar-baz"

The -a argument to read makes it put the split words in the given array. In this case the array is parts. You can access array elements through ${parts[0]}, ${parts[1]}, and ${parts[0]}. Or just access all of them through ${parts[@]}.

8. Process a string character by character

$ while IFS= read -rn1 c; do
    # do something with $c
done <<< "$str"

Here we use the -n1 argument to read command to make it read the input character at a time. Similarly we can use -n2 to read two chars at a time, etc.

9. Replace "foo" with "bar" in a string

$ echo ${str/foo/bar}

This one-liner uses parameter expansion of form ${var/find/replace}. It finds the string find in var and replaces it with replace. Really simple!

To replace all occurrences of "foo" with "bar", use the ${var//find/replace} form:

$ echo ${str//foo/bar}

10. Check if a string matches a pattern

$ if [[ $file = *.zip ]]; then
    # do something

Here the one-liner does something if $file matches *.zip. This is a simple glob pattern matching, and you can use symbols * ? [...] to do matching. Code * matches any string, ? matches a single char, and [...] matches any character in ... or a character class.

Here is another example that matches if answer is Y or y:

$ if [[ $answer = [Yy]* ]]; then
    # do something

11. Check if a string matches a regular expression

$ if [[ $str =~ [0-9]+\.[0-9]+ ]]; then
    # do something

This one-liner tests if the string $str matches regex [0-9]+\.[0-9]+, which means match a number followed by a dot followed by number. The format for regular expressions is described in man 3 regex.

12. Find the length of the string

$ echo ${#str}

Here we use parameter expansion ${#str} which returns the length of the string in variable str. Really simple.

13. Extract a substring from a string

$ str="hello world"
$ echo ${str:6}

This one-liner extracts world from hello world. It uses the substring expansion. In general substring expansion looks like ${var:offset:length}, and it extracts length characters from var starting at index offset. In our one-liner we omit the length that makes it extract all characters starting at offset 6.

Here is another example:

$ echo ${str:7:2}



14. Uppercase a string

$ declare -u var
$ var="foo bar"

The declare command in bash declares variables and/or gives them attributes. In this case we give the variable var attribute -u, which upper-cases its content whenever it gets assigned something. Now if you echo it, the contents will be upper-cased:

$ echo $var

Note that -u argument was introduced in bash 4. Similarly you can use another feature of bash 4, which is the ${var^^} parameter expansion that upper-cases a string in var:

$ str="zoo raw"
$ echo ${str^^}



15. Lowercase a string

$ declare -l var
$ var="FOO BAR"

Similar to the previous one-liner, -l argument to declare sets the lower-case attribute on var, which makes it always be lower-case:

$ echo $var
foo bar

The -l argument is also available only in bash 4 and later.

Another way to lowercase a string is to use ${var,,} parameter expansion:

$ str="ZOO RAW"
$ echo ${str,,}


zoo raw


Enjoy the article and let me know in the comments what you think about it! If you think that I forgot some interesting bash one-liners related to string operations, let me know in the comments also!

This article is part of the article series "Bash One-Liners Explained."
<- previous article next article ->

I love being super fast in the shell so I decided to do a new article series called Bash One-Liners Explained. It's going to be similar to my other article series - Awk One-Liners Explained, Sed One-Liners Explained, and Perl One-Liners Explained. After I'm done with this bash series, I'll release an e-book by the same title, just as I did with awk, sed, and perl series. The e-book will be available as a pdf and in mobile formats (mobi and epub). I'll also be releasing bash1line.txt, similar to perl1line.txt that I made for the perl series.

In this series I'll use the best bash practices, various bash idioms and tricks. I want to illustrate how to get various tasks done with just bash built-in commands and bash programming language constructs.

Also see my other articles about working fast in bash:

Let's start.

Part I: Working With Files

1. Empty a file (truncate to 0 size)

$ > file

This one-liner uses the output redirection operator >. Redirection of output causes the file to be opened for writing. If the file does not exist it is created; if it does exist it is truncated to zero size. As we're not redirecting anything to the file it remains empty.

If you wish to replace the contents of a file with some string or create a file with specific content, you can do this:

$ echo "some string" > file

This puts the string "some string" in the file.

2. Append a string to a file

$ echo "foo bar baz" >> file

This one-liner uses a different output redirection operator >>, which appends to the file. If the file does not exist it is created. The string appended to the file is followed by a newline. If you don't want a newline appended after the string, add the -n argument to echo:

$ echo -n "foo bar baz" >> file

3. Read the first line from a file and put it in a variable

$ read -r line < file

This one-liner uses the built-in bash command read and the input redirection operator <. The read command reads one line from the standard input and puts it in the line variable. The -r parameter makes sure the input is read raw, meaning the backslashes won't get escaped (they'll be left as is). The redirection command < file opens file for reading and makes it the standard input to the read command.

The read command removes all characters present in the special IFS variable. IFS stands for Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read built-in command. By default IFS contains space, tab, and newline, which means that the leading and trailing tabs and spaces will get removed. If you wish to preserve them, you can set IFS to nothing for the time being:

$ IFS= read -r line < file

This will change the value of IFS just for this command and will make sure the first line gets read into the line variable really raw with all the leading and trailing whitespaces.

Another way to read the first line from a file into a variable is to do this:

$ line=$(head -1 file)

This one-liner uses the command substitution operator $(...). It runs the command in ..., and returns its output. In this case the command is head -1 file that outputs the first line of the file. The output is then assigned to the line variable. Using $(...) is exactly the same as `...`, so you could have also written:

$ line=`head -1 file`

However $(...) is the preferred way in bash as it's cleaner and easier to nest.

4. Read a file line-by-line

$ while read -r line; do
    # do something with $line
done < file

This is the one and only right way to read lines from a file one-by-one. This method puts the read command in a while loop. When the read command encounters end-of-file, it returns a positive return code (code for failure) and the while loop stops.

Remember that read trims leading and trailing whitespace, so if you wish to preserve it, clear the IFS variable:

$ while IFS= read -r line; do
    # do something with $line
done < file

If you don't like the to put < file at the end, you can also pipe the contents of the file to the while loop:

$ cat file | while IFS= read -r line; do
    # do something with $line

5. Read a random line from a file and put it in a variable

$ read -r random_line < <(shuf file)

There is no clean way to read a random line from a file with just bash, so we'll need to use some external programs for help. If you're on a modern Linux machine, then it comes with the shuf utility that's in GNU coreutils.

This one-liner uses the process substitution <(...) operator. This process substitution operator creates an anonymous named pipe, and connects the stdout of the process to the write part of the named pipe. Then bash executes the process, and it replaces the whole process substitution expression with the filename of the anonymous named pipe.

When bash sees <(shuf file) it opens a special file /dev/fd/n, where n is a free file descriptor, then runs shuf file with its stdout connected to /dev/fd/n and replaces <(shuf file) with /dev/fd/n so the command effectively becomes:

$ read -r random_line < /dev/fd/n

Which reads the first line from the shuffled file.

Here is another way to do it with the help of GNU sort. GNU sort takes the -R option that randomizes the input.

$ read -r random_line < <(sort -R file)

Another way to get a random line in a variable is this:

$ random_line=$(sort -R file | head -1)

Here the file gets randomly sorted by sort -R and then head -1 takes the first line.

6. Read the first three columns/fields from a file into variables

$ while read -r field1 field2 field3 throwaway; do
    # do something with $field1, $field2, and $field3
done < file

If you specify more than one variable name to the read command, it shall split the line into fields (splitting is done based on what's in the IFS variable, which contains a whitespace, a tab, and a newline by default), and put the first field in the first variable, the second field in the second variable, etc., and it will put the remaining fields in the last variable. That's why we have the throwaway variable after the three field variables. if we didn't have it, and the file had more than three columns, the third field would also get the leftovers.

Sometimes it's shorter to just write _ for the throwaway variable:

$ while read -r field1 field2 field3 _; do
    # do something with $field1, $field2, and $field3
done < file

Or if you have a file with exactly three fields, then you don't need it at all:

$ while read -r field1 field2 field3; do
    # do something with $field1, $field2, and $field3
done < file

Here is an example. Let's say you wish to find out number of lines, number of words, and number of bytes in a file. If you run wc on a file you get these 3 numbers plus the filename as the fourth field:

$ cat file-with-5-lines
x 1
x 2
x 3
x 4
x 5

$ wc file-with-5-lines
 5 10 20 file-with-5-lines

So this file has 5 lines, 10 words, and 20 chars. We can use the read command to get this info into variables:

$ read lines words chars _ < <(wc file-with-5-lines)

$ echo $lines
$ echo $words
$ echo $chars

Similarly you can use here-strings to split strings into variables. Let's say you have a string "20 packets in 10 seconds" in a $info variable and you want to extract 20 and 10. Not too long ago I'd have written this:

$ packets=$(echo $info | awk '{ print $1 }')
$ time=$(echo $info | awk '{ print $4 }')

However given the power of read and our bash knowledge, we can now do this:

$ read packets _ _ time _ <<< "$info"

Here the <<< is a here-string, which lets you pass strings directly to the standard input of commands.

7. Find the size of a file, and put it in a variable

$ size=$(wc -c < file)

This one-liner uses the command substitution operator $(...) that I explained in one-liner #3. It runs the command in ..., and returns its output. In this case the command is wc -c < file that prints the number of chars (bytes) in the file. The output is then assigned to size variable.

8. Extract the filename from the path

Let's say you have a /path/to/file.ext, and you wish to extract just the filename file.ext. How do you do it? A good solution is to use the parameter expansion mechanism:

$ filename=${path##*/}

This one-liner uses the ${var##pattern} parameter expansion. This expansion tries to match the pattern at the beginning of the $var variable. If it matches, then the result of the expansion is the value of $var with the longest matching pattern deleted.

In this case the pattern is */ which matches at the beginning of /path/to/file.ext and as it's a greedy match, the pattern matches all the way till the last slash (it matches /path/to/). The result of this expansion is then just the filename file.ext as the matched pattern gets deleted.

9. Extract the directory name from the path

This is similar to the previous one-liner. Let's say you have a /path/to/file.ext, and you wish to extract just the path to the file /path/to. You can use the parameter expansion again:

$ dirname=${path%/*}

This time it's the ${var%pattern} parameter expansion that tries to match the pattern at the end of the $var variable. If the pattern matches, then the result of the expansion is the value of $var shortest matching pattern deleted.

In this case the pattern is /*, which matches at the end of /path/to/file.ext (it matches /file.ext). The result then is just the dirname /path/to as the matched pattern gets deleted.

10. Make a copy of a file quickly

Let's say you wish to copy the file at /path/to/file to /path/to/file_copy. Normally you'd write:

$ cp /path/to/file /path/to/file_copy

However you can do it much quicker by using the brace expansion {...}:

$ cp /path/to/file{,_copy}

Brace expansion is a mechanism by which arbitrary strings can be generated. In this particular case /path/to/file{,_copy} generates the string /path/to/file /path/to/file_copy, and the whole command becomes cp /path/to/file /path/to/file_copy.

Similarly you can move a file quickly:

$ mv /path/to/file{,_old}

This expands to mv /path/to/file /path/to/file_old.


Enjoy the article and let me know in the comments what you think about it! If you think that I forgot some interesting bash one-liners related to file operations, let me know in to comments also!

We've super exciting news at Browserling Inc. We've reached 100 paying customers! Actually as of publishing this article we're at 129 customers! We've about 50 companies and 80 individuals using Browserling. It took us nearly two years to get here.

Here is a small sample of companies that use Browserling.

PubNub, Onswipe, Iris Couch, Teambox, Fusionbox, Coderwall, Blekko, Teespring, PBWorks

Thank you all!

Browserling is an interactive cross-browser testing tool. It lets you test your website in older Internet Explorers such as IE 6, IE 7, IE 8 and IE 9, Firefox, Chrome, Opera and Safari!

Try Browserling!

We just launched real Internet Explorers for Browserling. They include IE6, IE7, IE8 and IE9. The free plan includes IE9 and our paid plan lets you use IE6, IE7 and IE8. Before this upgrade we were using IETester as a replacement for IE{6,7,8}, which was super buggy and we had a lot of complaints about not having real IEs. So now we have them!

We run the IE6, IE7 and IE8 on Windows Server 2003, and the IE9 runs on Windows Server 2008, all on Rackspace Cloud!

Here are the screenshots of IE versions that we run.

Internet Explorer 6:

Internet Explorer 6.0.3790.3959

Internet Explorer 7:

Internet Explorer 7.0.5730.13

Internet Explorer 8:

Internet Explorer 8.0.6001.18702

Internet Explorer 9:

Internet Explorer 9.0.8112.16421

Browserling is an interactive cross-browser testing tool. It lets you test your website in older Internet Explorers such as IE 6, IE 7, IE 8 and IE 9, Firefox, Chrome, Opera and Safari!

Try Browserling!

If you liked this post, you'll also like the post on how I went to Silicon Valley and raised $55,000 for Browserling.

This is the introduction to perl one-liners. Originally I wrote this introduction for my third e-book, however later I decided to make it a part of the free e-book preview and republish it here as this article.

Introduction to Perl one-liners

Perl one-liners are small and awesome Perl programs that fit in a single line of code and they do one thing really well. These things include changing line spacing, numbering lines, doing calculations, converting and substituting text, deleting and printing certain lines, parsing logs, editing files in-place, doing statistics, carrying out system administration tasks, updating a bunch of files at once, and many more. Perl one-liners will make you the shell warrior. Anything that took you minutes to solve, will now take you seconds!

Let's look at several examples to get more familiar with one-liners. Here is one:

perl -pi -e 's/you/me/g' file

This one-liner replaces all occurrences of the text you with me in the file file. Very useful if you ask me. Imagine you are on a remote server and have this file and you need to do the replacement. You can either open it in text editor and execute find-replace or just do it through command line and, bam, be done with it.

The -e argument is the best argument. It allows you to specify the Perl code to be executed right on the command line. In this one-liner the code says, do the substitution (s/find/replace/flags command) and replace you with me globally (g flag). The -p argument makes sure the code gets executed on every line, and that the line gets printed out after that. The -i argument makes sure that file gets edited in-place, meaning Perl opens the file, executes the substitution for each line, prints the output to a temporary file, and then replaces the original file.

How about doing the same replacement in multiple files? Just specify them on the command line!

perl -pi -e 's/you/me/g' file1 file2 file3

Now let's do the same replacement only on lines that match we. It's as simple as this:

perl -pi -e 's/you/me/g if /we/' file

Here we use the conditional if /we/. This makes sure that s/you/me/g gets executed only on lines that match the regular expression /we/. The regular expression here can be anything. Let's say you want to execute the substitution only on lines that have digits in them. You can then use the /\d/ regular expression that matches numbers:

perl -pi -e 's/you/me/g if /\d/' file

Now how about finding all repeated lines in a file?

perl -ne 'print if $a{$_}++' file

This one-liner records the lines seen so far in the %a hash and keeps the counter of how many times it has seen the same line. The $a{$_}++ creates elements in the %a hash automagically. When it sees a repeated line, the value of that hash element is defined and greater than zero, so if $a{$_} is true, and it prints the line. This one-liner also uses the -n command line argument that loops over the input but unlike -p doesn't print the lines automatically, so you have to use print explicitly.

How about numbering lines? Super simple! Perl has the $. special variable that automatically maintains the current line number. You can just print it out together with the line:

perl -ne 'print "$. $_"'

You can also achieve the same by using -p argument and modifying the $_ variable, which contains the entire line:

perl -pe '$_ = "$. $_"'

Here each line gets replaced by the string "$. $_", which is the current line number followed by the line itself.

How about we combine the previous two one-liners and create one that numbers repeated lines? Here we go:

perl -ne 'print "$. $_" if $a{$_}++'

Now let's do something different. Let's sum up all the numbers on each line. We'll use the sum function from the List::Util CPAN module. You can install it as easily as running perl -MCPAN -e'install List::Util'.

perl -MList::Util=sum -alne 'print sum @F'

The -MList::Util command line argument imports the List::Util module, and the =sum part of it imports the sum function from it. Next -a enables automatic splitting of fields into the @F array. The -l argument makes sure that print outputs a newline at the end of each line. Finally the sum @F sums up all the elements in the @F list and print prints it out, followed by a newline (that was added by the -l argument).

How about finding the date 1299 days ago? That's also solvable by a simple one-liner:

perl -MPOSIX -le '
  @now = localtime;
  $now[3] -= 1299;
  print scalar localtime mktime @now

This one-liner didn't quite fit in one line, but that's just because my blog has narrow space for content. What happens here is we modify the 4th element of the structure returned by localtime, which happens to be days. So we just subtract 1299 days from the current day. Then we reassembles it into a new time through localtime mktime @now and print it in scalar context that prints human readable time.

How about generating an 8 letter password? Again, solvable elegantly with a one-liner:

perl -le 'print map { (a..z)[rand 26] } 1..8'

The a..z generates a list of letters from a to z (total 26). Then we randomly choose one of the characters through generating a random number in range 0-25, and then we repeat this whole process 8 times!

Here is another one. Suppose you want to quickly find the decimal number that corresponds to an IP address. You can use the unpack function and find it really quickly:

perl -le 'print unpack("N",'

This one-liner uses a vstring, which is a version literal. The IP address is treated as a vstring, which is basically the numbers 127, 0, 0, 1 concatenated together. Next the unpack function unpacks them to a single decimal number.

Btw, I once created a cheat sheet with all the pack/unpack format strings. Get it here: perl pack, unpack cheat sheet.

Now how about calculations? Let's find the sum of the numbers in the first column:

perl -lane '$sum += $F[0]; END { print $sum }'

Here the lines automatically get split up into fields through the -a argument. The fields can be now accessed through the @F array. The first element of the array, $F[0], is the first column. So all we have to do is sum all the columns up through $sum += $F[0]. When the Perl program ends its job, it executes any code in the special END block. In this case we print out the total sum there. Really easy!

Now let's find out how many packets have gone through all the iptables rules. It's this simple:

iptables -L -nvx | perl -lane '
  $_ = $F[0]; $pkts += $_ if /^\d/; END { print $pkts }

The iptables program outputs the packets as the first field. All we do is check if the first field is numeric (because it also outputs labels header), and if so, sum the packets up on the first column, just like in the previous one-liner.

How about getting a list of the names of all users on the system?

perl -a -F: -lne 'print $F[4]' /etc/passwd

Combining -a with -F argument allows you to specify the character that lines should be split on. In this case it's the colon, which is the record separator of /etc/passwd. Next we just print the 5th field $F[4], which is the real name of the user. Really quick and easy.

As you can see, knowing Perl one-liners lets you accomplish many tasks quickly. If you wish to learn more and become really fast in the shell, I suggest you get a copy of my "Perl One-Liners Explained" e-book. The e-book contains 130 unique one-liners and many of them are presented in several different ways, so the total number of one-liners in this book is over 200.

If you enjoy my writing, you can subscribe to my blog, follow me on Twitter or Google+.