I was reading Knuth's "Selected Papers on Computer Science" and in chapter 13 "The IBM 650: An Appreciation from the Field" he had included a photo of a young himself from 1958 at his first computer.

Young Donald Knuth, age 20, at his first IBM 650 computer in 1958.

Don Knuth's first computer, as the chapter's title suggests, was the IBM 650. At that time Knuth had a part-time job helping the statisticians at Case Institute of Technology to draw graphs and do keypunching. Soon a strange new machine was installed across the hall, that the student newspaper called a "giant brain". It was actually an IBM 650.

One afternoon someone at the institute explained some of the machine's internal code to a bunch of freshmen, including Knuth. It all sounded mysterious to him, but it seemed to make a bit of sense, so he got ahold of some manuals. His first chance to try the machine came a few weeks later, when one of the upperclassmen at the fraternity needed to know the five roots of a particular fifth degree equation. Knuth decided to compute the roots using the 650. This was his first program on the 650.

Later Knuth realized how lucky he was to have had such a good first encounter with computers. The polynomial problem was well matched to his mathematical knowledge and interests, and he had a chance for hands-on experience, pushing buttons on the machine and seeing it punch the cards containing the answers.

Knuth's first large program was a tic-tac-toe that learned to play by remembering the relative desirability or undesirability of each position that it had ever encountered. The hardest part, as Knuth writes, was figuring out how to keep one digit of memory for each possible configuration of the board. The machine had just 2000 words of memory, each 10 digits long, plus a sign bit.

Next Knuth proceeded to writing a program that would find prime factors. The idea was that a person could set up any 10-digit number in the console switched and start his program, which would punch the corresponding prime factors on a card and stop. Knuth spent several weeks on this problem, rewriting his program several times. The final program took 11 minutes to determine that the number 9999999967 was prime.

The next major Knuth's program was the improvement of SOAP (Symbolic Optimal Assembly Program) and SOAP II assembly languages. He wrote SOAP III and learned about "creeping featurism," where each of his friends would suggest different things they wanted in an assembler. The program used all 2000 words of memory. Knuth doesn't think he could have gotten by with only 1999 words, because he had spent considerable time finding every last bit of space by using terrible tricks, such as using a single instruction on a specific address that would cause 4 side effects.

Along the way Knuth wrote a lot fun programs. One of the competitions between the students was to do as much as possible with programs that would fit on a single punch card - which had room for only eight instructions. One of the unsolved problems was to take the 10-digit number on the console and to reverse its digits from left to right, then display the answer and stop; nobody could figure out how to do this on a signle card. But one day Knuth proudly marched up to the machine and made a demonstration: he read in a card, then dialled the number 0123456789 on the console, and started the machine. Sure enough, it stopped, displaying the number 9876543210. Everybody applauded. He didn't explain until later that his card would display the number 9876543210 regardless of what number appeared on the console switches. :)

But there is more to the story. One day the IBM 650 machine got an extra set of console switches, that were called register 8004 (top row of switches in the picture). It turned out that nine instructions on an extended 650 were sufficient to reverse the digits of a number, and the ninth instruction could be put into the switches. Therefore he was able to solve the problem without cheating.

Knuth was very close with IBM 650. One night he missed a date with his wife-to-be, because he was so engrossed in debugging that he had forgotten all about the time.

The 650 provided Knuth with solid instruction in the art of programming. It was directly related to the topics of the first two technical articles he ever submitted for publication. Therefore it's not surprising at all that he decided to dedicate The Art of Computer Programming books to the IBM 650 computer. "This series of books is affectionately dedicated to the Type 650 computer once installed at Case Institute of Technology, in remembrance of many pleasant evenings."

**The End.**

If you liked this story, you may want to get Knuth's book "Selected Papers on Computer Science". The collection focuses on Knuth's publications that are addressed primarily to a general audience than to specialists.

**14**Comments February 24, 2010

# MIT Linear Algebra, Lecture 6: Column Space and Null Space

This is the sixth post in an article series about MIT's course "Linear Algebra". In this post I will review lecture six on **column spaces** and **null spaces** of matrices. The lecture first reviews vector spaces and subspaces and then looks at the result of intersect and union of vector subspaces, and finds out when Ax=b and Ax=0 can be solved.

Here is a list of the previous posts in this article series:

- Lecture 1: Geometry of Linear Equations
- Lecture 2: Elimination with Matrices
- Lecture 3: Matrix Multiplication and Inverse Matrices
- Lecture 4: A=LU Factorization
- Lecture 5: Vector Spaces and Subspaces

## Lecture 6: Column Space and Null Space

Lecture six starts with a reminder of what the vector space requirements are. If vectors **v** and **w** are in the space, then the result of adding them and multiplying them by a number stays in the space. In other words, all linear combinations of **v** and **w** stay in the space.

For example, the 3-dimensional space **R**^{3} is a vector space. You can take any two vectors in it, add them, and multiply them by a number and they will still be in the same **R**^{3} space.

Next, the lecture reminds subspaces. A subspace of some space is a set of vectors (including the **0** vector) that satisfies the same two requirements. If vectors **v** and **w** are in the subspace, then all linear combinations of **v** and **w** are in the subspace.

For example, some subspaces of **R**^{3} are:

- Any plane P through the origin (0, 0, 0).
- Any line L through the origin (0, 0, 0).

See the previous, lecture 5, on more examples of spaces on subspaces.

Now suppose we have two subspaces of **R**^{3} - plane P and line L. Is the union P∪L a subspace? **No.** Because if we take some vector in P and some vector in L and add them together, we go outside of P and L and that does not satisfy the requirements of a subspace.

What about intersection P∩L? Is P∩L a subspace? **Yes.** Because it's either the zero vector or L.

In general, given two subspaces S and T, union S∪T is a not a subspace and intersection S∩T is a subspace.

The lecture now turns to **column spaces of matrices**. The notation for a column space of a matrix A is C(A).

For example, given this matrix,

The column space C(A) is all the linear combination of the first (1, 2, 3, 4), the second (1, 1, 1, 1) and the third column (2, 3, 4, 5). That is, C(A) = { a·(1, 2, 3, 4) + b·(1, 1, 1, 1) + c·(2, 3, 4, 5) }. In general, **the column space C(A) contains all the linear combinations of columns of A**.

A thing to note here is that C(A) is a subspace of **R**^{4} because the vectors contain 4 components.

Now the key question, does C(A) fill the whole **R**^{4}? **No.** Because there are only three columns (to fill the whole **R**^{4} we would need exactly 4 columns) and also because the third column (2, 3, 4, 5) is a sum of first column (1, 2, 3, 4) and second column (1, 1, 1, 1).

From this question follows another question, the most important question in the lecture - Does Ax=b have a solution for every right-hand side vector b? **No.** Because the columns are not linearly independent (the third can be expressed as first+second)! Therefore the column space C(A) is actually a two-dimensional subspace of **R**^{4}.

Another important question arises - For which right-hand sides b can this system be solved? The answer is: **Ax=b can be solved if and only if b is in the column space C(A)**! It's because Ax is a combination of columns of A. If b is not in this combination, then there is simply no way we can express it as a combination.

That's why we are interested in column spaces of matrices. They show when can systems of equation Ax=b be solved.

Now the lecture turns to **null spaces of matrices**. The notation for a null space of a matrix A is N(A).

Let's keep the same matrix A:

The null space N(A) contains something completely different than C(N). **N(A) contains all solutions x's that solve Ax=0**. In this example, N(A) is a subspace of **R**^{3}.

Let's find the null space of A. We need to find all x's that solve Ax=0. The first one, obviously, is x = (0, 0, 0). Another one is x = (1, 1, -1). In general all x's (c, c, -c) solve Ax=0. The vector (c, c, -c) can be rewritten as c·(1, 1, -1).

Note that the null space c·(1, 1, -1) is a line in **R**^{3}.

The lecture ends with a proof that solutions x to Ax=0 always give a subspace. The first thing to show in the proof is that if x is a solution and x' is a solution, then their sum x + x' is a solution:

We need to show that if Ax=0 and Ax' = 0 then A(x + x') = 0. This is very simple. Matrix multiplication allows to separate A(x + x') into Ax + Ax' = 0. But Ax=0 and Ax'=0. Therefore 0 + 0 = 0.

The second thing to show is that if x is a solution, then c·x is a solution:

We need to show that if Ax=0 then A(c·x)=0. This is again very simple. Matrix multiplication allows to bring c from A(c·x) outside c·A(x) = c·0 = 0.

That's it. We have proven that solutions x to Ax=0 always form a subspace.

Here is the video of the sixth lecture:

Direct link: http://www.youtube.com/watch?v=8o5Cmfpeo6g

Topics covered in lecture six:

- [01:00] Vector space requirements.
- [02:10] Example of spaces R
^{3}. - [02:40] Subspaces of spaces.
- [03:00] A plane P is a subspace of R
^{3}. - [03:50] A line L is a subspace of R
^{3}. - [04:40] Union of P and L.
- [07:30] Intersection of P and L.
- [09:00] Intersection of two subspaces S and T.
- [11:50] The column space C(A) of a matrix A.
- [16:20] Does Ax=b have a solution for every b?
- [19:45] Which b's allow Ax=b to be solved?
- [23:50] Can solve Ax=b exactly when b is in C(A).
- [28:50] The null space N(A) of a matrix A.
- [37:00] Why is the null space a vector space?
- [37:30] A proof that the null space is always a vector space.
- [41:50] Do the solutions to Ax=b form a subspace?

Here are my notes of lecture six:

Have fun with this lecture! The next post is going to be about general theory of solving equations Ax=0, pivot variables and special solutions.

PS. This course is taught from Introduction to Linear Algebra textbook. Get it here:

In this post I'll derive the Y-combinator and explain all the steps taken. The key to understanding the Y-combinator is knowing precisely what it does. I am sure many people never understand Y-combinator because of this very reason, or because the introduction is too lengthy to comprehend. Therefore I'll give the shortest possible explanation of Y-combinator before I derive it so that you know what the end result should be.

The Y-combinator allows an anonymous function to call itself, that is, **it allows anonymous function recursion**. Since anonymous functions don't have names, they can't refer to themselves easily. The Y-combinator is a clever construct helps them to refer to themselves. That's it. That's all the Y-combinator does. Remember that.

I'll use the Scheme programming language to derive the Y-combinator. My derivation is based on the one in "The Little Schemer" book. I took the examples from the book, filled in the missing steps and explained the steps in more details.

Here it is.

Suppose we want to write the `length`

function, which, given a list, returns the number of elements in it. It's really easy if we can give the function a name:

```
(define length
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list)))))))
```

But now suppose that we can't give names - we can only use anonymous functions:

```
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (??? (cdr list)))))))
```

Suddenly there is no way this anonymous function can refer to itself. What do we put in ** ???**? We can't refer to the name

`length`

anymore because it's an anonymous function, which doesn't have any name. One thing we can try is to put the function itself in the place of **:**

`???`

```
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (
(lambda (list) ; the
(cond ;
((null? list) 0) ; function
(else ;
(add1 (??? (cdr list)))))) ; itself
(cdr list))))))
```

This is not much better, we are still left with ** ???**. But there is a bright side to it, this function can determine lengths of lists with one element and no elements. Let's try inserting the same anonymous function in place of

**again:**

`???`

```
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (??? (cdr list))))))
(cdr list))))))
(cdr list))))))
```

Now this function can determine lengths of lists with 0, 1 and 2 elements. If we continue this way, we can construct the `length`

function for lists with a certain number of elements. But this is not what we want, we the real `length`

function that works for lists with any number of elements.

As we all know, repeating code is not a good thing. Let's try to factor out the repetitions and rewrite the original anonymous function slightly. Instead of leaving `???`

in the code, let's pass it to an anonymous function via an argument called `length`

.

```
((lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list)))))))
???)
```

Notice that this code invokes the first `lambda (length)`

function and passes it `???`

as the `length`

argument. This function in turn returns the original anonymous function:

```
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (??? (cdr list))))))
```

Now let's try constructing the previous two length functions that work for lists with a maximum of one and two elements. First the function for lists with a maximum of one element:

```
((lambda (f)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (f (cdr list)))))))
((lambda (g)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (g (cdr list)))))))
???))
```

Here `???`

first gets passed to `lambda (g)`

, which returns the original anonymous function, which then gets passed to `lambda (f)`

, which in turn returns the function that works for lists with one element:

```
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (??? (cdr list))))))
(cdr list))))))
```

Similarly, the following code returns a function that works for lists with a maximum of two elements:

```
((lambda (f)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (f (cdr list)))))))
((lambda (g)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (g (cdr list)))))))
((lambda (h)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (h (cdr list)))))))
???)))
```

Since the argument names `f`

, `g`

,`h`

in `(lambda (f) ...)`

, `(lambda (g) ...)`

, `(lambda (h) ...)`

are independent, we can rename all of them to `length`

, to make it look more similar to the `length`

function:

```
((lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list)))))))
((lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list)))))))
((lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list)))))))
???)))
```

We are still repeating code. The obvious thing we can do about it is create an anonymous function called `mk-length`

(make length) that creates this code for us:

```
((lambda (mk-length)
(mk-length ???))
(lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list))))))))
```

This is pretty tricky. Observe how `(lambda (mk-length) ...)`

gets the `(lambda (length) ...)`

function passed as the `mk-length`

argument, which in turn accepts `???`

as an argument and returns our original anonymous function:

```
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (??? (cdr list))))))
```

Now let's try constructing length functions for lists of length one and two. For the list of length one, we just need to apply `mk-length`

on itself:

((lambda (mk-length) (mk-length (mk-length ???))) (lambda (length) (lambda (list) (cond ((null? list) 0) (else (add1 (length (cdr list))))))))

Let's go through this code. First `(lambda (length) ...)`

gets passed to the `lambda (mk-length)`

function as the `mk-length`

argument. Then it applies the result of `(mk-length ???)`

(which is the original anonymous function) to the `mk-length`

. This produces our well known function that works on lists with one or none elements:

```
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (??? (cdr list))))))
(cdr list))))))
```

Similarly, by adding another call to `mk-length`

, we get the function that works for lists with two or less elements:

```
((lambda (mk-length)
(mk-length
(mk-length
(mk-length ???)))
(lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list))))))))
```

Now, since we don't know what `???`

is, let's just call it `mk-length`

and see what happens:

```
((lambda (mk-length)
(mk-length mk-length))
(lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list))))))))
```

Nothing bad actually happens, we still get the same original anonymous function, except in place of `???`

we have the function:

```
(lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list)))))))
```

Notice also that argument names `mk-length`

and `length`

in `lambda (mk-length)`

and `lambda (length)`

are independent. Therefore we can rename `length`

to `mk-length`

to remind that the first argument to `mk-length`

is `mk-length`

:

```
((lambda (mk-length)
(mk-length mk-length))
(lambda (mk-length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (mk-length (cdr list))))))))
```

And now comes the key trick. We replace `mk-length`

with a self-application `(mk-length mk-length)`

:

```
((lambda (mk-length)
(mk-length mk-length))
(lambda (mk-length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 ((mk-length mk-length) (cdr list))))))))
```

If you try this code out, you'll see that it returns the length of a list for any list. Here is an example of applying it to a list of 10 elements:

```
(((lambda (mk-length)
(mk-length mk-length))
(lambda (mk-length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 ((mk-length mk-length) (cdr list))))))))
'(a b c d e f g h i j))
```

The function works because it keeps adding recursive uses by passing `mk-length`

to itself, just as it is about to expire. This is not yet the Y-combinator, but we have successfully managed to recursively call an anonymous function. Now we need to massage the code a bit to separate the anonymous function from the self-applicative code.

The first step is to move the self-applicative code out as much as possible:

((lambda (mk-length) (mk-length mk-length)) (lambda (mk-length)((lambda (length) (lambda (list) (cond ((null? list) 0) (else (add1 (length (cdr list)))))))(lambda (x) ((mk-length mk-length) x)))))

Now the code in bold looks very much like the `length`

function we need! Let's move it outside as well:

((lambda (le) ((lambda (mk-length) (mk-length mk-length)) (lambda (mk-length) (le (lambda (x) ((mk-length mk-length) x))))))(lambda (length) (lambda (list) (cond ((null? list) 0) (else (add1 (length (cdr list))))))))

There we have it! The anonymous `length`

function has been separated from the self-applicative call. The code in **bold** is the `length`

function, and the code not-in-bold is the Y-combinator!

Now let's rename `mk-length`

to `f`

and join the lines to make it more compact:

```
((lambda (le)
((lambda (f) (f f))
(lambda (f)
(le (lambda (x) ((f f) x))))))
(lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list))))))))
```

And we are done. We can now write down the definition of Y-combinator:

```
(define Y
(lambda (le)
((lambda (f) (f f))
(lambda (f)
(le (lambda (x) ((f f) x)))))))
```

And we can apply it to our anonymous `length`

function:

```
((Y (lambda (length)
(lambda (list)
(cond
((null? list) 0)
(else
(add1 (length (cdr list))))))))
'(a b c d e f g h i j))
; ==> 10
```

This is one of many derivations of the applicative-order Y-combinator. See the publication "The Why of Y" for another derivation.

I am not just a Linux enthusiast, I also happen to use Windows quite often. In fact, Windows is my primary desktop from which I connect to all the other boxes and do my work on. During the years of Windows usage, I have accumulated a list of must-have Windows programs that I wouldn't be able to work without. Some of them are commercial, some are freeware, but that doesn't matter. What matters is how productive you are with your setup. If you're really productive on Linux with your own set of tools, it's perfectly fine and you have done a great job of finding the best tools of trade.

## 1. Total Commander.

I can't imagine working on a computer without Total Commander. Absolutely must-have software. Total Commander is what separates boys from men. Total Commander is probably the #1 reason why I don't use other operating system on my desktop. Tabs, a great GUI interface and customize-everything-configuration make it superior to all the other file managers. If I switch the desktop OS, I'll have to write my own clone of Total Commander or I wouldn't be able to work with the computer. Total commander is shareware.

## 2. TrueCrypt.

Because you don't want anyone to know what you store on your disks. TrueCrypt the best disk encryption software for Windows. TrueCrypt is freeware.

## 3. SecureCRT and Putty.

SecureCRT is the SSH client to use. Again, the primary reason I use it is because it has tab support. Putty comes second as it doesn't have tabs. SecureCRT is shareware. Putty is freeware.

## 4. WinSCP.

Forget all your FTP clients. FTP is dead and unsecure. I use WinSCP to send all my files over a secure connection. WinSCP is freeware.

## 5. Cygwin.

Since Windows has never had a usable shell, the only way to get one is to run a local ssh server via Cygwin and connect to localhost via SecureCRT. This way you get the best possible shell. Cygwin is freeware.

## 6. VMWare Workstation.

I have been using VMWare Workstation since 2004 and have never had a single problem. It's the best virtualization software ever written and nothing comes close. VMWare Workstation is shareware.

## 7. Synergy.

Synergy allows you to share your mouse and keyboard via network with other computers (no need for a KVM switch). It just works. Synergy is freeware.

## 8. Beyond compare.

The greatest visual diff tool ever written. Period. Beyond compare is shareware.

## 9. Tclock2.

Tclock2 is freeware that allows you to customize the windows clock in the systray. I set it to `dddd\nyyyy.mm.dd\nhh:nn:ss`

.

## 10. Auto Hotkeys.

With Auto Hotkeys you can remap your keyboard and script shortcuts. For one, the first thing to do is to remap ESC to CAPS LOCK so that you could work in Vim. Auto Hotkeys is freeware.

## 11. Locate32.

Locate32 is to Windows what locate is to Linux. It helps you find all your files in an instant. Locate32 is freeware.

## 12. allSnap.

allSnap snaps your windows one next to each other, i.e., it aligns them nicely. allSnap is freeware.

## 13. Fineprint.

Fineprint is the best printer proxy ever. Allows you to print multiple pages per sheet even much more smartly than your printer does. Plus it has a real print preview. Fineprint is shareware.

## 14. Launchy.

I can't imagine working without Launchy. It's QuickSilver for Windows. It allows to quickly launch programs by pressing Alt+Space and typing in the first few chars of the program's name. Launchy is freeware.

## 15. ClipX.

ClipX maintains a paste buffer. Use Winkey+V to paste from it. If you haven't used it, it will change the way you work with your computer. ClipX is freeware.

## 16. DU Meter.

Du Meter measures your network usage and shows nice plots of it. I like it. Du Meter is shareware.

## 17. Taskbar Shuffle.

Taskbar Shuffle allows you to rearrange taskbar tabs and systray icons. Must-have. Taskbar Shuffle is freeware.

## 18. UltraMon.

If you're on a multi-screen setup, like me, UltraMon adds taskbars to your other screens. It's productivity super-booster. UltraMon is shareware.

## 19. Sumatra PDF.

Adobe Reader got bloated and died 5 years ago. Foxit Reader got bloated died 2 years ago. I hope Sumatra PDF reader doesn't die. Sumatra PDF is freeware.

## 20. KeePass.

KeePass is a secure password manager. I probably have 10 thousand passwords in it secured by one master password. Keepass is freeware.

## 21. RoboForm.

While KeePass keeps your passwords secure in a file, RoboForm allows you to dynamically submit them to various website forms. It also stores them under the master password. RoboForm is shareware.

## 22. WinRAR.

WinRAR is the best archivator for Windows. I have been using it since I was born. 7-Zip doesn't come close. WinRAR is shareware.

## 23. Inkscape.

Inkscape is a descent vector graphics editor. It's free.

## 24. FFDShow.

FFDShow is a codec library. If you wish to watch all kinds of videos on your computer, it's a must-have. FFDShow is freeware.

## 25. Media Player Classic.

MPC is the best video player for windows. It doesn't have skins or any other useless bullshit. MPC is freeware.

## 26. Real Alternative.

Real Alternative will use Media Player Classic to play RealMedia files. Real Alternative is freeware.

## 27. Quicktime Alternative.

Quicktime Alternative is to QuickTime files what Real Alternative is to RealMedia files. Freeware.

## 28. AviSynth.

AviSynth is a video processing programming language. If you wish to speed your videos up, it's a must-have. AviSynth is freeware.

## 29. FFMpeg.

FFMpeg is a must-have for converting videos and working with AviSynth scripts. FFMpeg is freeware.

## 30. VirtualDub.

VirtualDub is another must-have software for editing videos on Windows. VirtualDub is freeware.

## 31. Dependency Walker.

Sometimes programs don't work because of DLL hell. Dependency Walker helps you to diagnose and solve these problems. Dependency Walker is freeware.

## 32. Most of the Sysinternals tools.

Autoruns, regmon, procmon, procexp, tcpview, to name a few from Sysinternals are just must-have. They are all freeware.

## 33. File & Folder Unlocker.

UFFunlock is used when you get the nasty "Access denied" errors and you know that no process should be using the resource. FFunlock is freeware.

## 34. Hijackthis.

Hijackthis is necessary just to make sure you don't have nasty programs on your computer. Hijackthis is freeware.

## 35. ImgBurn.

ImgBurn is the new Nero. It burns your CDs and DVDs. ImgBurn is freeware.

## 36. IsoBuster.

IsoBuster knows all the CD/DVD image formats. IsoBuster is shareware.

## 37. Virtual CloneDrive.

Virtual CloneDrive allows you to mount ISOs right as windows drives. Virtual CloneDrive is freeware.

## 38. SQLyog.

As crappy as SQLyog is, it's somehow the most usable MySQL front-end. 10 years ago I used mysql-gui-console but the project got discontinued and no one has written a usable MySQL front-end ever since. If I'm ever in a mood, I'll write one that doesn't suck. SQLyog is shareware but I wouldn't pay a cent for it.

## 39. pgAdmin.

I applaud to pgAdmin dev team for making the best GUI front-end for PostgreSQL. Folks who write MySQL front-ends should learn from these guys. They know what a GUI tool should do. pgAdmin is freeware.

## 40. XML Notepad.

A notepad.exe for XML files. Allows you to edit XML smartly without destroying its structure accidentally. XML Notepad is freeware.

## 41. Hex Workshop.

Hex Workshop is a really great and easy to use Hex editor. It's shareware.

## 42. Vim.

For everything else.

## What tools do you use?

This was a list of tools that I can't live without. What tools did I miss?

**30**Comments February 03, 2010

# Perl One-Liners Explained, Part V: Text conversion and substitution

This is the fifth part of a nine-part article on **Perl one-liners**. In this part I will create various one-liners for **text conversion and substitution**. See part one for introduction of the series.

Perl one-liners is my attempt to create "**perl1line.txt**" that is similar to "awk1line.txt" and "sed1line.txt" that have been so popular among Awk and Sed programmers.

The article on Perl one-liners will consist of nine parts:

- Part I: File spacing.
- Part II: Line numbering.
- Part III: Calculations.
- Part IV: String creation and array creation.
**Part V: Text conversion and substitution (this part).**- Part VI: Selective printing and deleting of certain lines.
- Part VII: Handy regular expressions.
- Part VIII: Release of perl1line.txt.
- Part IX: Release of Perl One-Liners e-book.

After I'm done with explaining the one-liners, I'll release an ebook. Subscribe to my blog to know when that happens!

**Awesome news:** I have written an e-book based on this article series. Check it out:

Alright then, here are today's one-liners:

## Text conversion and substitution

**62. ROT13 a string.**

'y/A-Za-z/N-ZA-Mn-za-m/'

This one-liner uses the `y`

operator (also known as `tr`

operator) to do ROT13. Operators `y`

and `tr`

do string transliteration. Given `y/SEARCH/REPLACE/`

, the operator transliterates all occurrences of the characters found in `SEARCH`

list with the corresponding (position-wise) characters in `REPLACE`

list.

In this one-liner `A-Za-z`

creates the following list of characters:

ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz

And `N-ZA-Mn-za-m`

creates this list:

NOPQRSTUVWXYZABCDEFGHIJKLMnopqrstuvwxyzabcdefghijklm

If you look closely you'll notice that the second list is actually the first list offset by 13 characters. Now the `y`

operator translates each character in the first list to a character in the second list, thus performing the ROT13 operation.

If you wish to ROT13 the whole file then do this:

perl -lpe 'y/A-Za-z/N-ZA-Mn-za-m/' file

The `-p`

argument puts each of `file`

's line in the `$_`

variable, the `y`

does ROT13, and `-p`

prints the `$_`

out. The `-l`

appends a newline to the output.

Note: remember that applying ROT13 twice produces the same string, i.e., `ROT13(ROT13(string)) == string`

.

**63. Base64 encode a string.**

perl -MMIME::Base64 -e 'print encode_base64("string")'

This one-liner uses the MIME::Base64 module that is in the core (no need to install it, it comes with Perl). This module exports the `encode_base64`

function that takes a string and returns base64 encoded version of it.

To base64 encode the whole file do the following:

perl -MMIME::Base64 -0777 -ne 'print encode_base64($_)' file

Here the `-0777`

argument together with `-n`

causes Perl to slurp the whole file into the `$_`

variable. Then the file gets base64 encoded and printed out, just like the string example above.

If we didn't slurp the file and encoded it line-by-line we'd get a mess.

**64. Base64 decode a string.**

perl -MMIME::Base64 -le 'print decode_base64("base64string")'

The `MIME::Base64`

module also exports `decode_base64`

function that takes a base64-encoded string and decodes it.

The whole file can be similarly decoded by:

perl -MMIME::Base64 -ne 'print decode_base64($_)' file

There is no need to slurp the whole file into `$_`

because each line of a base64 encoded file is exactly 76 characters and decodes nicely.

**65. URL-escape a string.**

perl -MURI::Escape -le 'print uri_escape($string)'

You'll need to install the URI::Escape module as it doesn't come with Perl. The module exports two functions - `uri_escape`

and `uri_unescape`

. The first one does URL-escaping (sometimes also referred to as URL encoding), and the other does URL-unescaping (URL decoding).

**66. URL-unescape a string.**

perl -MURI::Escape -le 'print uri_unescape($string)'

This one-liner uses the `uri_unescape`

function from `URI::Escape`

module to do URL-unescaping.

**67. HTML-encode a string.**

perl -MHTML::Entities -le 'print encode_entities($string)'

This one-liner uses the `encode_entities`

function from HTML::Entities module. This function encodes HTML entities. For example, `<`

and `>`

get turned into `<`

and `>`

.

**68. HTML-decode a string.**

perl -MHTML::Entities -le 'print decode_entities($string)'

This one-liner uses the `decode_entities`

function from `HTML::Entities`

module.

**69. Convert all text to uppercase.**

perl -nle 'print uc'

This one-liner uses the `uc`

function, which by default operates on the `$_`

variable and returns an uppercase version of it.

Another way to do the same is to use `-p`

command line option that enables automatic printing of `$_`

variable and modify it in-place:

perl -ple '$_=uc'

The same can also be also achieved by applying the `\U`

escape sequence to string interpolation:

perl -nle 'print "\U$_"'

It causes anything after it (or until the first occurrence of `\E`

) to be upper-cased.

**70. Convert all text to lowercase.**

perl -nle 'print lc'

This one-liner is very similar to the previous. Here the `lc`

function is used that converts the contents of `$_`

to lowercase.

Or, using escape sequence `\L`

and string interpolation:

perl -nle 'print "\L$_"'

Here `\L`

causes everything after it (until the first occurrence of `\E`

) to be lower-cased.

**71. Uppercase only the first word of each line.**

perl -nle 'print ucfirst lc'

The one-liner first applies the `lc`

function to the input that makes it lower case and then uses the `ucfirst`

function that upper-cases only the first character.

It can also be done via escape codes and string interpolation:

perl -nle 'print "\u\L$_"'

First the `\L`

lower-cases the whole line, then `\u`

upper-cases the first character.

**72. Invert the letter case.**

perl -ple 'y/A-Za-z/a-zA-Z/'

This one-liner does transliterates capital letters `A-Z`

to lowercase letters `a-z`

, and lowercase letters to uppercase letters, thus switching the case.

**73. Camel case each line.**

perl -ple 's/(\w+)/\u$1/g'

This is a lousy Camel Casing one-liner. It takes each word and upper-cases the first letter of it. It fails on possessive forms like "friend's car". It turns them into "Friend'S Car".

An improvement is:

s/(?<!['])(\w+)/\u\1/g

Which checks if the character before the word is not single quote `'`

. But I am sure it still fails on some more exotic examples.

**74. Strip leading whitespace (spaces, tabs) from the beginning of each line.**

perl -ple 's/^[ \t]+//'

This one-liner deletes all whitespace from the beginning of each line. It uses the substitution operator `s`

. Given `s/REGEX/REPLACE/`

it replaces the matched `REGEX`

by the `REPLACE`

string. In this case the `REGEX`

is `^[ \t]+`

, which means "match one or more space or tab at the beginning of the string" and `REPLACE`

is nothing, meaning, replace the matched part with empty string.

The regex class `[ \t]`

can actually be replaced by `\s+`

that matches any whitespace (including tabs and spaces):

perl -ple 's/^\s+//'

**75. Strip trailing whitespace (space, tabs) from the end of each line.**

perl -ple 's/[ \t]+$//'

This one-liner deletes all whitespace from the end of each line.

Here the `REGEX`

of the `s`

operator says "match one or more space or tab at the end of the string." The `REPLACE`

part is empty again, which means to erase the matched whitespace.

**76. Strip whitespace from the beginning and end of each line.**

perl -ple 's/^[ \t]+|[ \t]+$//g'

This one-liner combines the previous two. Notice that it specifies the global `/g`

flag to the `s`

operator. It's necessary because we want it to delete whitespace at the beginning AND end of the string. If we didn't specify it, it would only delete whitespace at the beginning (assuming it exists) and not at the end.

**77. Convert UNIX newlines to DOS/Windows newlines.**

perl -pe 's|\n|\r\n|'

This one-liner substitutes the Unix newline `\n`

LF with Windows newline `\r\n`

CRLF on each line. Remember that the `s`

operator can use anything for delimiters. In this one-liner it uses vertical pipes to delimit `REGEX`

from `REPLACE`

to improve readibility.

**78. Convert DOS/Windows newlines to UNIX newlines.**

perl -pe 's|\r\n|\n|'

This one-liner does the opposite of the previous one. It takes Windows newlines CRLF and converts them to Unix newlines LF.

**79. Convert UNIX newlines to Mac newlines.**

perl -pe 's|\n|\r|'

Apple Macintoshes used to use `\r`

CR as newlines. This one-liner converts UNIX's `\n`

to Mac's `\r`

.

**80. Substitute (find and replace) "foo" with "bar" on each line.**

perl -pe 's/foo/bar/'

This one-liner uses the `s/REGEX/REPLACE/`

command to substitute "foo" with "bar" on each line.

To replace all "foos" with "bars", add the global `/g`

flag:

perl -pe 's/foo/bar/g'

**81. Substitute (find and replace) "foo" with "bar" on lines that match "baz". **

perl -pe '/baz/ && s/foo/bar/'

This one-liner is equivalent to:

while (defined($line = <>)) { if ($line =~ /baz/) { $line =~ s/foo/bar/ } }

It puts each line in variable `$line`

, then checks if line matches "baz", and if it does, it replaces "foo" with "bar" in it.

## Perl one-liners explained e-book

I've now written the "Perl One-Liners Explained" e-book based on this article series. I went through all the one-liners, improved explanations, fixed mistakes and typos, added a bunch of new one-liners, added an introduction to Perl one-liners and a new chapter on Perl's special variables. Please take a look:

## Have Fun!

Have fun with these one-liners for now. The next part is going to be about selective printing and deleting of certain lines.

**Can you think of other text conversion and substitution procedures that I did not include here?**