I have been working on the new catonmat.net website for quite some time now and I have gathered a very long list ideas of what a modern website should have. They include ideas from psychology, search engine optimization, social media, programming, and the best web design practices.

Since I started to use github last week, all the new catonmat code will be there. I already created a catonmat repository and I am going to be pushing code there daily.

I love to share my ideas, so here are the first 50 of them. I am going to share more later in the upcoming article series "**Designing the new catonmat.net together with me.**" You should subscribe to my blog here, if you are interested in this topic and haven't subscribed yet.

The ideas are in no particular order. If some of them seem fuzzy or unclear, please ask me to clarify in the comments.

## 1. 301 Redirect Table.

The idea is to maintain a table that will be checked against the request URLs. If the request URL is in the table, it gets redirected to a new destination URL with the 301 HTTP header, which forwards the link juice from one URL to another. This is necessary because some forums break URLs in multiple lines and the resulting clickable URL is 404. If I didn't do a 301 to the new location, this link would be lost. Anyone who clicked it, would end up on the 404 error page. Another example is if someone links to a mistyped URL. Anyone who clicks it ends up being unsatisfied, at 404 page. With a 301 table I can quickly fix these problems. Here is a concrete example: Someone links to www.catonmat.net/artikle when they wanted to link to www.catonmat/article. I'd simply insert an entry to 301 redirect /artikle to /article and everyone's happy.

## 2. 404 Error Log.

I absolutely have to know what pages trigger 404 errors so that I can fix them ASAP with the 301 redirect table. There may be many false alarms, therefore I have to add filters to the 404 error log to ignore some spammy patterns.

## 3. A Great 404 Page.

If someone ends up on a non-existing page, log it as described in #2 and then explain to the visitor why the page he visited was not found. Suggest him or her or it to view 5 latest post titles, or 5 most popular posts, or most popular downloads, or something else (still have to think about it when I implement it).

## 4. A Separate Trackback/Pingback/Linkback/Refback Page With Moderation.

Tracebacks are usually displayed among comments. This is very messy. There is no place for them among comments. The idea is to have them on a separate page like www.catonmat.net/post-title/trackbacks. Also add bulk moderation as spammers love to exploit trackback. (Perhaps drop trackbacks altogether as they are of very little value.)

## 5. A Page With Last Comments.

Currently I am displaying only the 10 last comments on the right sidebar. This is insufficient. I want to see all the comments (like www.reddit.com/comments) on a single page, so that even if I am away from computer for several days, I can easily navigate through them and reply/hide/edit/delete in-place. Perhaps add a RSS feed for comments page.

## 6. Comment Statistics.

I need to tell my blog readers who are the most active people on my website. This will stimulate people to be more active. Therefore I should add a comment statistics with most popular commentators, link to their pages without nofollow to give them link juice. Also add statistics of the most commented articles.

## 7. Add Incoming Search Term Statistics On All Pages.

When people come from Google or Yahoo, save the query they used and display it on the page. This way I will always know what terms were searched for on each of the pages, without having to write complex queries.

## 8. Add Download Statistics.

Currently I have sloppy download statistics. I want nice graphs and want to see the most popular downloads by day, month, etc. This should be written as a statistics framework as traffic statistics, article statistics, delicious statistics and download statistics will all have more or less same graphs.

## 9. Public Statistics Section For del.icious.com Bookmarks.

I want to see how often my posts get bookmarked on del.icious.com. I also want nice graphs for them by day, week, month, etc. I also want a list of people who bookmarked my posts the most often, and tags they used. Make this public and also very modular so other people can reuse the code and put it on their sites. Reward the most frequent bookmarkers with links to their sites.

## 10. Insert Beautiful Images To Give Rest To Eyes.

Insert some beautiful landscape images after serveral paragraphs of text to give rest to eyes and give the reader positive emotions.

An image from ginnerobot's photostream on Flickr.

## 11. Public Traffic Statistics.

I want the super statistics for my website. And make them public. I want to see which is the most popular article today, which was yesterday. I want nice traffic graphs and trends.

## 12. Public Twitter Statistics.

Find who's tweeting about catonmat and put all these tweets on a separate page. Find who's the most active catonmat tweeter and make this person stand out. Link to twitter profiles.

## 13. Integrate GitHub Directly In Catonmat

Integrate GitHub with status updates, friend count, directly in catonmat. Would have to write some kind of a scraper or use their API, if it's usable for this purpose.

## 14. Implement Threaded Comments.

Currently I have linear comments, which are no good. To find whom someone replied to I have to scroll through the comments. This doesn't make any sense. I have to add threaded comments. This will also engage users is more conversations.

## 15. Have Individual Links To Comments Like On Reddit.

I don't like links to comments in a form www.catonmat.net/article/#comment-55. I want them to be in form www.catonmat.net/article/comment-55 so that each comment (or thread of comments) is on its own page. This way, if linked to a comment, person will precisely have that comment loaded. Browsers can misbehave in case of anchor links like #comment-55 and I don't like that.

## 16. Lightweight Syntax For Comments.

There are just a few things comments need:

- Quote someone.
- Emphesize part of a comment in bold or italic.
- Link to some page.
- Share code fragment (auto syntax highlight it).

That's it. Nothing else is necessary, no stupid HTML comments.

## 17. Filter Comments By Language.

My website is getting a lot of spam from Russia with comments in Russian. I am sure no one would write comments in Russian on my site. Add a filter to leave just the English comments. All other are spam.

## 18. Add Gravatar Icons To Comments.

Gravatar is at www.gravatar.com. It's a map from emails to jpegs of user icons. This will make people stand out. Also display the gravatar user icons on comment statistics (point above). Having gravatars will emotionally associates you with commentators and the next time you see a gravatar you can predict the nature of the comment (depending if you had positive emotions or negative before).

## 19. Add More "Contact Me" Options Near "About me" on The Right Sidebar.

You want to get yourself out. If you don't do it, no one will come after you. Add links to Facebook, Linked In, Twitter, Plurk, GitHub, FriendFeed, perhaps some other sites. Show the email as an image and add a link to my IRC channel #catonmat on irc.freenode.net. Initially show only Twitter, Facebook and GitHub. Add an arrow down clickable image. After clicked displays all other contact options.

## 20. Snippets Page.

I have been writing and collecting various programming snippets. I want to have them in a central database on my site. Instead of putting them on some foreign service like GitHub's Gist or Pastebin of some kind, I want to keep them on my website in my database so that I can easily modify them in a single place and integrate within posts.

An image from digital cat's photostream on Flickr.

## 21. Add Revision Control For All The Pages.

Currently if I edit a page, the previous page is lost and I can't see the changes. It's crucial to keep the changes as sometimes I need to get something from a year ago. I have to add revision control like wikipedia does. The URL scheme could be www.catonmat.net/article/revisions - displays all available revisions, and www.catonmat.net/article/revisions/r1/r2 displays changes between r1 and r2, but I have to think about it a bit more.

## 22. Create Tiny URLs For Articles On My Own Site

I don't want to depend on some service that may go down. Make short urls like http://catonmat.net/abc, where abc is [a-zA-Z0-9] this will give me 238328 URLs, more than enough. I could even go for something shorter.

## 23. Optimize Catonmat Load Speed To Maximum.

Use the page-speed FireBug plugin to optimize site loading speed. Page-speed is at http://code.google.com/speed/page-speed/. Things to be optimized include minified javascript, minified html, gzip compressed content, maximizing caching, use asynchronous js loading of google analytics and others.

## 24. Make The Posts More Available

Currently the posts are only available as HTML documents. I should try to convert them to PDF and put them to Scribd. I have to think about consequences as Scribd may show up on search engines at a higher ranking position than catonmat iself, which would have drastic impact on the traffic. Saving to PDF has a benefit that it's a single file. If saved as HTML, the browser creates a folder with tons of images. Can't be shared easily and is clutter. Perhaps offer a PDF download for all articles.

## 25. Make Posts Printer Friendly.

Create a nice CSS template for printing articles. At the end of the article include URLs to all the mentioned resources. Add an option to choose whether to print comments or not. URL structure could be www.catonmat.net/article/print

## 26. Add A "Share this" Widget.

Add "Share this" widget and perhaps "Reddit this", "Digg this", "Stumble this", etc., buttons. This should be based on referrer as I don't want to show "Reddit this" to a Digg visitor as there is a holy war between Reddit and Digg. Also add "Tweet this" button somewhere.

## 27. Utilize The New Google Feature Of Displaying Named Anchors In Search Results.

See this post: http://googleblog.blogspot.com/2009/09/jump-to-information-you-want-right-from.html Some of my posts utilize this (10 Awk Tips, Tricks and Pitfalls), but I need to utilize it more.

## 28. Highlight The Python Code As In SQLalchemy Documentation.

I like how the code is highlighted in SQLalchemy documentation. I am not sure if they are using Pygments or not, but I'll try to make mine exactly the same. Example: http://www.sqlalchemy.org/docs/05/ormtutorial.html

## 29. Create Pagination For Posts/Categories/Tags As In Flickr.com

I like the style of Flickr's pagination. Got to implement the same on catonmat. Example: http://www.flickr.com/photos/frijole/

~~30. Have Pages Open In A New Window By Default.~~

I feel that opening links in a new window would keep visitors on the website longer. I haven't tested but I will A/B this. Update: This is not a good idea, won't implement it.

An image from paraflyer's photostream on Flickr.

## 31. Investigate What Do Various <a rel="..."> Do.

There are a bunch of different relations like rel="bookmark", rel="prev", rel="next". This could improve the website navigation greatly. More info here: http://www.w3schools.com/TAGS/att_a_rel.asp

## 32. Perhaps Remove The Article Date Altogether.

I have noticed myself that if I search for something and I find an article from 2004, I want to look for something fresher. Got to A/B test this and see how long do people stay on the site.

## 33. Add An IP Ban List.

Sometimes spammers use the same IPs. Rather than `iptables -I INPUT --src IP -j REJECT` them, just block them at application level, 404 all pages, or redirect them elsewhere.

## 34. Automatically Translate All Pages To All Languages Via Google Translate

Sometimes people search for something in their own language and can't find it. Perhaps they don't know English term and therefore can't find what they wanted. If I automatically translate all pages to all languages, people would end up on my website and find what they were looking for. The URL scheme for this could be: www.catonmat.net/xx/..., where xx is two letter international language code.

## 35. Try Out Dustin Curtis' Advice On Best Performing Link Texts.

Dustin Curtis did an experiment where he tried various link texts to invite readers to follow him on Twitter. The first version was "I'm on Twitter", this got 4.70% click through rate. The last version was "You Should follow me on Twitter <here>". This had 12.81% CTR, which is a massive improvement.

## 36. Add "Ask Peteris Krumins" Section Where People Can Ask Me Various Questions.

People ask me a lot of questions over email. I could answer all the questions on my website instead of email. This way everyone could always find all my answers.

## 37. Add A "FAQ" section.

I get asked the same questions over and over again over email. For example,

- What do I have to know for Google interview?
- What books do I read?
- How to learn C++/C/Python/Algorithms?
- etc.

Instead of sending the answer over email, I could send these people to FAQ page where they could find my latest answers (as they change over time).

## 38. Add Knowledge Database.

This idea has the highest priority. A knowledge database is a section on my website where I can write everything that I learned each day. This should be accompanied with a desktop application that has a hotkey that instantly brings up input dialog and I can type what I just learned. I had a database like this in 2002-2004 and my knowledge literally went exponential. I wrote out key facts that I learned each day and could easily locate as necessary.

## 39. Add A Miniblog For Quick Articles.

Sometimes I have some cool idea or quick hack that I want to share, but as I am used to writing large and well thought out articles, I can't post the quick hacks and my thoughts don't get shared. A miniblog would allow me to share even the smallest thoughts that I have.

## 40. Add More Programming Quotes.

I love various smart programming quotes. I should make them more accessible, make them searchable by author/text.

An image from paraflyer's photostream on Flickr.

## 41. Integrate LaTeX In My Posts.

As I am sometimes writing about maths, I need to integrate LaTeX directly in my posts. I should not forget to do SEO on images it generates - instead of having some ridiculous <img src="latex-generator?q=$\begin{bmatrix}1&2\\3&4\end{bmatrix}$>" I should have it generate an image "<img src="matrix.jpg" title="Matrix">". This way people will be able to find my posts via image search if they search for some mathematical terms like "Matrix".

## 42. Try Out How The Articles Look With Text-Align: Justify.

I currently have the default left-aligned text. Books and journals have it justified. Not sure how it would look on my website. Have to try it out. A/B. I read somewhere that this may feel confusing to dyslexic users.

## 43. Have In-Line Code Snippets And Variables Stand Out From Rest Of The Text.

A nice example of this is Github's blog. They have gray background for constant-width things. See this: http://github.com/blog/530-how-we-made-github-fast

## 44. Have <a> Links Change Background Color On Hover.

I love how mattt.me has done it: http://mattt.me/articles/ I want this.

## 45. Have Only One Category Per Post But Multiple Tags.

A post should have only one single category. The category must strictly define the main theme of the article. A post can have multiple tags. Tags define topics discussed in the post.

## 46. Add Crazyegg Tracking For The First Month.

Must add crazyegg to track how the users navigate the site, where they click and what they visit. Optimize based on the results. http://crazyegg.com/

## 47. Add A Job Board.

As my site is getting more popular and popular among programmers, it may be a good idea to add a job board. Joel Spolsky made a million $ in a year with job boards. As the popularity of my site increases, I might make a few dollars out of it as well.

## 48. Use Statcounter and Google Analytics

This is obvious. Statcounter is for real-time data. The free version is limited to statistics of last 500 visits. Google Analytics is for keeping long term statistics with a day of delay in updates. Load them asynchronously: http://googlecode.blogspot.com/2009/12/google-analytics-launches-asynchronous.html

## 49. Form Input Fields And Text Fields Should Change Border Color On Focus.

Users should know what field they are focused to without trying to find the cursor.

## 50. Mandatory Alt Attributes For Images, Title Attributes For Links.

The rationale of this feature is that if images don't get loaded or if blind people are listening to the content of my posts via text-to-speech engine, they should know what the image displays. Alt tags also help the search engines to classify images. The same goes for title attributes for links.

## 51. Optimize Meta Description For Categories And Tags

Category and tag pages usually have meta description as "Posts in <category>". This is unsatisfactory. I want a description of category and if it's missing or is too short, I want it to have some post titles in it, to make it unique. The same for tags, make meta tags "Posts in <tag>: post title1, post title2, ..." not exceeding 20 words or so.

An image from dsevilla's photostream on Flickr.

The next post in this series will be about Python libraries that I use for the new catonmat and the structure of the site. You can also follow the development on GitHub (I started importing code only yesterday so there is not much yet).

Hey everyone, I just pushed 30 of my projects to GitHub. I realized that all the projects were scattered across my blog and there was no central repository. So I took time to organize them, write documentation, and uploaded them to GitHub.

I did all of these projects for fun and to learn better programming. You can't become a great programmer if you don't program a lot. The more you program, the more language idioms and constructs you'll learn. You'll learn common patterns that occur frequently in programming and it will greatly improve your problem solving skills.

These were all relatively small projects and I think I am ready to move to the next level. I have several larger ideas in mind that need to be turned into code. I will post the updates to catonmat.

If you find any of my projects interesting, clone and start hacking. You can also follow my profile at github. :-)

Here they all are. Enjoy!

## 1. Busy Beaver

Busy beaver is a computer science problem to finding the smallest Turing Machine that outputs the most data and eventually halts. This project is an implementation of a Turing Machine in Python and C++ that runs the busy beavers. It also comes with Turing Machine's tape visualization tool written in Perl.

## 2. Feedburner Graph Generator

Current Feedburner statistics graphs do not look nice. I wrote this Perl program to generate the nice graphs they had in 2008.

## 3. CodingHorror Keyword Analyzer

This is a Perl program that parses public statcounter data for codinghorror.com blog and stores the search keywords in an SQLite database.

- CodingHorror keyword analyzer on Github.
- Article about analyzing Coding Horror keywords on catonmat.

## 4. Adns

This is a tiny Python module that does asynchronous DNS resolution with adns library. My benchmarks were 20,000 resolutions per minute on a 512kbit/s connection.

## 5. Winamp Music Reporter Plugin

This is a Winamp plugin that reports to digitalpoint forums the tracks you are listening to. Written in C and uses Winamp SDK. The code can be modified to make it report to Facebook or Twitter, or anywhere you wish.

## 6. Bithacks.h

Bithacks.h is a C header file that implements various bit operations.

## 7. Set Operations in Unix Shell

This is an implementation of 14 set operations by using only Unix utilities such as sort, uniq, diff, comm, cat, head, tail, awk, and others.

## 8. Hacker Top

Hacker top is a Unix top-like program written in Python for monitoring Hacker News from the console.

## 9. Reddit Top

Reddit top is a Unix top-like program written in Python for monitoring Reddit from the console.

## 10. GNU Awk YouTube Downloader

This is a program written in GNU Awk that downloads YouTube videos. It works well but please don't take it too seriously. It's a proof of concept code, written to see what can be done with GNU Awk's networking interface.

## 11. VBScript YouTube Downloader

This is a program written in VBScript that downloads YouTube videos. I wrote it because when I was a child, I did a lot of programming in Visual Basic and I wanted to remember what it was like.

## 12. Perl One-Liner YouTube Downloader

This is a Perl one-liner that downloads YouTube videos. I wrote it because I love Perl golf.

- YouTube downloader one-liner in Perl on GitHub.
- An article about creating the Perl one-liner that downloads YouTube videos on catonmat.

## 13. YouTube Video Uploader

This is a YouTube video uploader that works without any APIs. It just simulates what a browser would do and takes all the steps to post the video and set the video info. Written in Perl.

## 14. Plurk Translation Tool

This is a GreaseMonkey script that translates Plurks to English.

Plurk is like Twitter but more fun and organized. Come be my friend on Plurk, my profile name is pkrumins. Written in JavaScript.

- Plurk translation tool on Github.
- Plurk translation tool description on Plurk's Open Source website.

## 15. Command Line Plurker

Plurk from your command line. Written in Perl.

## 16. Find Plurks on Google

This program searches for plurks that were indexed by Google. It outputs URLs to indexed pages. It's written in Python and uses my xgoogle library.

## 17. Delete Plurks

This is a GreaseMonkey script that adds a "delete" button on individual plurk pages. This way you can delete your or other people plurks (if it's your thread) directly from the plurk page. Written in JavaScript.

## 18. Xgoogle

A Python library to Google services - Google Search, Google Sets, Google Translate and Sponsored Links.

- Xgoogle on GitHub.
- An article about Google search module on catonmat.
- An article about Google sets module on catonmat.
- An article about Google translate module on catonmat.
- An article about Google sponsored links module on catonmat.

## 19. Reddit Media

The old Reddit Media website that I created in 2007. No longer maintained. It was written in Perl.

## 20. Reddit River

The old Reddit River website for mobile devices. No longer maintained. It was written in Python.

## 21. Digpicz

This is the digpicz.com website that I created back in 2007. It got massive attention back then because Digg didn't have picture section then. It was written in Perl.

## 22. Picurls

Picurls.com is a picture aggregator much like popurls.com but for pics. Currently down for maintenance, will be soon up again. Written in PHP.

- Picurls on GitHub.
- An article about creating Picurls.com website part I.
- An article about creating Picurls.com website part II.

## 23. Bash Vi Editing Mode Cheat Sheet

Bash has two input modes - emacs and vi. This is vi input/editing mode keyboard shortcut cheat sheet.

## 24. Bash Emacs Editing Mode Cheat Sheet

Bash has two input modes - emacs and vi. This is emacs input/editing mode keyboard shortcut cheat sheet.

## 25. Bash History Cheat Sheet

This is the bash history cheat sheet. It summarizes everything there is to know about working efficiently with command line history in bash.

## 26. Screen Cheat Sheet

This is the screen terminal emulator cheat sheet. It lists the default keyboard shortcuts for working with screen.

## 27. Perl Special Variable Cheat Sheet

This is Perl predefined variable cheat sheet. It lists all the variables from `perldoc perlvar` with a concise description and some example usages. I created it when I was mastering Perl. I enjoy Perl golf and I wanted to know all of the variables.

- Perl special variable cheat sheet on GitHub.
- An article about Perl predefined variable cheat sheet on catonmat.

## 28. Perl pack/unpack and printf/sprintf Cheat Sheet

This is Perl pack/unpack/printf/sprintf cheat sheet. The pack/unpack cheat sheet is on page one, and it lists the pack/unpack template parameters and what they do. The printf/sprintf cheat sheet is on page two, and it lists the printf/sprintf format specifiers and format attributes.

I created this when I was mastering what could be done with pack/unpack. I added printf/sprintf as I could never remember all the format specifiers.

- Perl pack, unpack, printf, sprintf cheat sheet on GitHub.
- An article about Perl pack, unpack, printf, sprintf cheat sheet on catonmat.

## 29. Awk Cheat Sheet

AWK programming language cheat sheet.

## 30. Sed Cheat Sheet

This is sed cheat sheet. Sed is the Unix stream editor. If you don't know it, you don't know Unix.

## 31. Ed Cheat Sheet

This is ed cheat sheet. Ed is The Unix Text Editor.

One day when I was learning sed, I got interested if it originated from ed, which got me interested in ed itself. I find that cheat sheets are a great way to learn new topics and therefore I created this cheat sheet.

## 32. The New catonmat.net Website

I just started pushing code to the new catonmat.net repository. It's going to be a state of the art personal website from now on.

I have around 100 ideas for it, and the next big article series on catonmat is going to be "**Designing the new catonmat.net together with me.**" You should subscribe to my blog here if you are interested!

- The New catonmat.net on GitHub.
- Articles about designing the new catonmat coming soon.

If you have any questions, don't hesitate to ask in the comments!

**5**Comments December 03, 2009

# MIT Linear Algebra, Lecture 2: Elimination with Matrices

This is the second post in an article series about MIT's course "Linear Algebra". In this post I will review lecture two on solving systems of linear equations by **elimination** and **back-substitution**. The other topics in the lecture are **elimination matrices** (also known as **elementary matrices**) and **permutation matrices**.

The first post covered the geometry of linear equations.

One of my blog readers, Seyed M. Mottaghinejad, had also watched this course and sent me his lecture notes. They are awesome. Grab them here: lecture notes by Seyed M. Mottaghinejad (includes .pdf, .tex and his document class).

Okay, here is the second lecture.

## Lecture 2: Elimination with Matrices

**Elimination** is the way every software package solves equations. If the elimination succeeds it gets the answer. If the matrix A in A**x**=**b** is a "good" matrix (we'll see what a good matrix is later) then the elimination will work and we'll get the answer in an efficient way. It's also always good to ask how can it fail. We'll see in this lecture how elimination decides if the matrix A is good or bad. After the elimination there is a step called **back-substitution** to complete the answer.

Okay, here is a system of equations. Three equations in three unknowns.

Remember from lecture one, that every such system can be written in the matrix form A**x**=**b**, where A is the matrix of coefficients, **x** is a column vector of unknowns and **b** is the column vector of solutions (the right hand side). Therefore the matrix form of this example is the following:

For the elimination process we need the matrix A and the column vector b. The idea is very simple, first we write them down in the augmented matrix form A|b:

Next we subtract rows from one another in such a way that the final result is an upper triangular matrix (a matrix with all the elements below the diagonal being zero).

So the first step is to subtract the first row multiplied by 3 from the second row. This gives us the following matrix:

The next step is to subtract the second row multiplied by 2 from the third row. This is the final step and produces an upper triangular matrix that we needed:

Now let's write down the equations that resulted from the elimination:

Working from the bottom up we can immediately find the solutions z, y, and x. From the last equation, z = -10/5 = -2. Now we put z in the middle equation and solve for y. 2y = 6 + 2z = 6 + 2(-2) = 6 - 4 = 2 => y = 1. And finally, we can substitute y and z in the first equation and solve for x. x = 2 - 2y - z = 2 - 2(1) - (-2) = 2.

We have found the solution, it's (x=2, y=1, z=-2). The process we used to find it is called the **back-substitution**.

The elimination would fail if taking a multiple of one row and adding to the next would produce a zero on the diagonal (and there would be no other row to try to exchange the failing row with).

The lecture continues with figuring out how to do the elimination by using matrices. In the first lecture we learned that **a matrix times a column vector gave us a combination of the columns of the matrix**. Similarly, **a row times a matrix gives us a combination of the rows of the matrix**.

Let's look at our first step of elimination again. It was to subtract 3 times the first row from the second row. This can be expressed as matrix multiplication (forget the column b for a while):

Let's call the matrix on the right E as elimination matrix (or elementary matrix), and give it subscript E_{21} for making a zero in the resulting matrix at row 2, column 1.

The next step was twice the second row minus the third row:

The matrix on the right is again an elimination matrix. Let's call it E_{32} for giving a zero at row 3, column 2.

But notice that these two operations can be combined:

And we can write E_{32}(E_{21}A) = U. Now remember that matrix operations are associative, therefore we can change the parenthesis (E_{32}E_{21})A = U. If we multiply (E_{32}E_{21}) we get a single matrix E that we will call the **elimination matrix**. What we have done is expressed the whole elimination process in matrix language!

Next, the lecture continues takes a step back and looks at **permutation matrices**. The question asked is "what matrix would exchange two rows of a matrix?" and "what matrix would exchange two columns of a matrix?"

Watch the lecture to find the answer to these questions!

Direct link: http://www.youtube.com/watch?v=QVKj3LADCnA

Topics covered in lecture two:

- [00:25] Main topic for today: elimination.
- [02:35] A system with three equations and three unknowns:
- [03:30] Elimination process. Taking matrix A to U.
- [08:35] Three pivots of matrix U.
- [10:15] Relation of pivots to determinant of a matrix.
- [10:40] How can elimination fail?
- [14:40] Back substitution. Solution (x=2, y=1, z=-2).
- [19:45] Elimination with matrices.
- [21:10] Matrix times a column vector is a linear combination of columns the matrix.
- [22:15] A row vector times a matrix is a linear combination of rows of the matrix.
- [23:40] Matrix x column = column.
- [24:10] Row x matrix = row.
- [24:20] Elimination matrix for subtracting three times row one from row two.
- [26:55] The identity matrix.
- [30:00] Elimination matrix for subtracting two times row two from row three.
- [32:40] E
_{32}E_{21}A = U. - [37:20] Permutation matrices.
- [37:30] How to exchange rows of a 2x2 matrix?
- [37:55] Permutation matrix P to exchange rows of a 2x2 matrix.
- [38:40] How to exchange columns of a 2x2 matrix?
- [39:40] Permutation matrix P to exchange columns of a 2x2 matrix.
- [42:00] Commutative law does not hold for matrices.
- [44:25] Introduction to inverse matrices.
- [47:10] E
^{-1}E = I.

Here are my notes of lecture two:

Have fun with this lecture! The next post is going to be either on lectures three and four together or just lecture three. Lecture three will touch a bit more on matrix multiplication and then dive into the inverse matrices. Lecture four will cover A=LU matrix decomposition (also called factorization).

PS. This course is taught from Introduction to Linear Algebra textbook. Get it here:

There is a mysterious underground Perl organization that hardly anyone has heard of. Their primary mission is to discover new secret Perl operators but they are not allowed to talk about them. I managed to infiltrate this organization and steal some of their secrets. Here are 8 secret Perl operators that I retrieved from their mysterious hoard:

- The Spaceship Operator
- The Eskimo Greeting Operator
- The Goatse Operator
- The Turtle Operator
- The Inchworm Operator
- The Inchworm-On-A-Stick Operator
- The Spacestation Operator
- The Venus Operator

(to be honest, only the first is a real Perl operator, all others are syntactic tricks.)

## The Spaceship Operator <=>

<=> is the spaceship operator. Most commonly it's used to sort a list of numbers. Here is an example:

my @numbers = (-59, 99, 87, 1900, 42, 1, -999, 30000, 0); my @sorted = sort { $a <=> $b } @numbers; print "@sorted\n"; # output: -999 -59 0 1 42 87 99 1900 30000

If you don't specify a block with the spaceship operator to sort() function, it will treat the numbers as strings and sort them asciibetically:

my @numbers = (-59, 99, 87, 1900, 42, 1, -999, 30000, 0); my @sorted = sort @numbers; print "@sorted\n"; # output: -59 -999 0 1 1900 30000 42 87 99

In general the spaceship operator is defined as following:

- $a <=> $b is -1 if $a < $b.
- $a <=> $b is 0 if $a == $b.
- $a <=> $b is 1 if $a > $b.
- $a <=> $b is undef if $a and $b are NaN.

## The Eskimo Greeting Operator }{

The Eskimo greeting operator can be most frequently met in Perl one-liners.

For example, this one-liner uses the Eskimo greeting to emulate `wc -l` command and prints the number of lines in a file:

perl -lne '}{ print $.' file

Here the Eskimo greets the print function. To understand what happens here, you have to know what the -n command line option does. It causes Perl to assume the following loop around your program:

while (<>) { ... }

Where `...` contains the code specified by the -e command line option. If the code specified is `}{ ...` then it causes the while loop to be closed with no actions to be done and only the `...` part gets executed.

Therefore the one-liner above is equivalent to:

while (<>) { } { print $. }

This just prints the special variable $. which is the number of input lines processed.

This can be extended further and we can have Eskimo greet code on both sides:

perl -lne 'code1 }{ code2'

Code1 gets executed within the loop and code2 after the loop is done:

while (<>) { code1 } { code2 }

If you are interested in the topic of Perl one-liners, see the first part of my article "Perl One-Liners Explained".

## The Goatse Operator =()=

The Goatse operator, as nasty as it may sound, doesn't do any nasty things. Instead it does a wonderful thing and causes an expression on the right to be evaluated in array context.

Here is an example,

my $str = "5 foo 6 bar 7 baz"; my $count =()= $str =~ /\d/g; print $count;

This program prints 3 - the number of digits in $str. How does it do it? Let's deparse the 2nd line:

(my $count = (() = ($str =~ /\d/g)));

What happens here is that the expression ($str =~ /\d/g) gets assigned to the empty list (). Assigning to a list forces the list context. The whole (() = ($str =~ /\d/g)) thing gets evaluated in list context, but then it gets assigned to a scalar which causes it to get evaluated again in scalar context. So what we have is a list assignment in scalar context. The key thing to remember is that a list assignment in scalar context returns the number of elements on the right-hand side of the list assignment. In this example the right-hand side of the list assignment is ($str =~ /\d/g). This matches globally (/g flag) and finds 3 digits in $str. Therefore the result is 3.

## The Turtle Operator "@{[]}"

I couldn't find the name of this operator therefore I decided to name it the turtle operator, because it looks a bit like a turtle, @ being the head, and {[]} being the shell.

This operator is useful for interpolating an array inside a string.

Compare these two examples:

print "these people @{[get_names()]} get promoted"

and

print "these people ", join " ",get_names(), " get promoted"

Clearly, the first example wins for code clarity.

More precisely, writing

print "@{[something]}"

is exactly the same as writing

print join $", something

## The Inchworm Operator ~~

The inchworm operator can be used to force scalar context.

Here is an example with localtime() function. In scalar context localtime() returns human readable time, but in list context it returns a 9-tuple with various date elements.

$ perl -le 'print ~~localtime' Mon Nov 30 09:06:13 2009

Here localtime was evaluated in scalar context, even though it was called within print that forces list context. It returned human readable date and time.

$ perl -le 'print localtime' 579301010913330

Here localtime returned a list of 9 elements and print function just printed them one after another. To really see that it's a list of 9 elements, let's use the turtle operator:

$ perl -le 'print "@{[localtime]}"' 5 13 9 30 10 109 1 333 0

## The Inchworm-On-A-Stick Operator ~-

For numbers greater than 0, this operator decrements them by one. Example:

my $x = 5; print ~-$x; # prints 4

It works because ~-$x parses to (~(-$x)), which on a two-complement machine is effectively the same as $x-1.

## The Spacestation Operator -+-

The spacestation operator turns a string starting with positive number into a number. Here are some examples:

print -+-"4zy" # prints 4 print -+-'3.99' # prints 3.99 print -+-'2e5' # prints 200000

## The Venus Operator 0+

It's named the Venus operator because the astronomical symbol for the planet Venus looks similar.

It does the same as the spacestation operator, it numifies a string, but it binds less tightly than spacestation. An example:

print 0+"4zy" # prints 4

Share the secrets!

**7**Comments November 24, 2009

# MIT Linear Algebra, Lecture 1: The Geometry of Linear Equations

This is going to be my summary of the freely available^{*} Linear Algebra course from MIT. I watched the lectures of this course in the summer of 2008. This was not the first time I learned linear algebra. I had already had two terms of linear algebra when I studied physics back in 2004. But it was not enough for a curious mind like mine. I wanted to see how it was taught at the world's best university.

The rationale of why I am posting these mathematics lectures on my programming blog is because I believe that if you want to be a great programmer and work on the most exciting and world changing problems, then you have to know linear algebra. Larry and Sergey wouldn't have created Google if they didn't know linear algebra. Take a look at this publication if you don't believe me "Linear Algebra Behind Google." Linear algebra has also tens and hundreds of other computational applications, to name a few, data coding and compression, pattern recognition, machine learning, image processing and computer simulations.

The course contains 35 lectures. Each lecture is 40 minutes long, but you can speed them up and watch one in 20 mins. The course is taught by no other than Gilbert Strang. He's the world's leading expert in linear algebra and its applications and has helped the development of Matlab mathematics software. The course is based on working out a lot of examples, there are almost no theorems or proofs.

The textbook used in this course is Introduction to Linear Algebra by Gilbert Strang:

I'll write the summary in the same style as I did my summary of MIT Introduction to Algorithms. I'll split up the whole course in 30 or so posts, each post will contain one or more lectures together with my comments, my scanned notes, embedded video lectures and a timeline of the topics in the lecture. But, not to flood my blog with just mathematics, I will write one or two programming posts in between. You should subscribe to my posts through RSS here.

***** The whole course is available at MIT's Open Course Ware: Course 18.06, Linear Algebra.

I'll review the first lecture today.

## Lecture 1: The Geometry of Linear Equations

The first lecture starts with Gilbert Strang stating the **fundamental problem of linear algebra**, which is to solve systems of linear equations. He proceeds with an example.

The example is a system of two equations in two unknowns:

There are three ways to look at this system. The first is to look at it a row at a time, the second is to look a column at a time, and the third is use the matrix form.

If we look at this equation a row at a time, we have two independent equations 2x - y = 0 and -x + 2y = 3. These are both line equations. If we plot them we get the row picture:

The row picture shows the two lines meeting at a single point (x=1, y=2). That's the solution of the system of equations. If the lines didn't intersect, there would have been no solution.

Now let's look at the columns. The column at the x's is (2, -1), the column at y's is (-1, 2) and the column at right hand side is (0, 3). We can write it down as following:

This is a linear combination of columns. What this tells us is that we have to combine the right amount of vector (2, -1) and vector (-1, 2) to produce the vector (0, 3). We can plot the vectors in the column picture:

If we take one green x vector and two blue y vectors (in gray), we get the red vector. Therefore the solution is again (x=1, y=2).

The third way to look at this system entirely through matrices and use the matrix form of the equations. The matrix form in general is the following: A**x** = **b** where A is the coefficient matrix, **x** is the unknown vector and **b** is the right hand side vector.

How to solve the equations written in matrix form will be discussed in the next lectures. But I can tell you beforehand that the method is called Gauss elimination with back substitution.

For this two equations, two unknowns system, the matrix equation Ax=b looks like this:

The next example in the lecture is a system of three equations in three unknowns:

We can no longer plot it in two dimensions because there are three unknowns. This is going to be a 3D plot. Since the equations are linear in unknowns x, y, z, we are going to get three planes intersecting at a single point (if there is a solution).

Here is the row picture of 3 equations in 3 unknowns:

The red is the 2x - y = 0 plane. The green is the -x + 2y - z = -1 plane, and the blue is the -3y + 4z = 4 plane.

Notice how difficult it is to spot the point of intersection? Almost impossible! And all this of going one dimension higher. Imagine what happens if we go 4 or higher dimensions. (The intersection is at (x=0, y=0, z=1) and I marked it with a small white dot.)

The column picture is almost as difficult to understand as the row picture. Here it is for this system of 3 equations in 3 unknowns:

The first column (2, -1, 0) is red, the second column (-1, 2, -3) is green, the fourth column (0, -1, 4) is blue, and the result (0, -1, 4) is gray.

Again, it's pretty hard to visualize how to manipulate these vectors to produce the solution vector (0, -1, 4). But we are lucky in this particular example. Notice that if we take none of red vector, none of green vector and one of blue vector, we get the gray vector! That is, we didn't even need red and green vectors!

This is all still tricky, and gets much more complicated if we go to more equations with more unknowns. Therefore we need better methods for solving systems of equations than drawing plane or column pictures.

The lecture ends with several questions:

- Can A
**x**=**b**be solved for any**b**? - When do the linear combination of columns fill the whole space?,
- What's the method to solve 9 equations with 9 unknowns?

The examples I analyzed here are also carefully explained in the lecture, you're welcome to watch it:

Direct link: http://www.youtube.com/watch?v=ZK3O402wf1c

Topics covered in lecture one:

- [00:20] Information on textbook and course website.
- [01:05] Fundamental problem of linear algebra: solve systems of linear equations.
- [01:15] Nice case: n equations, n unknowns.
- [02:20] Solving 2 equations with 2 unknowns
- [02:54] Coefficient matrix.
- [03:35] Matrix form of the equations. Ax=b.
- [04:20] Row picture of the equations - lines.
- [08:05] Solution (x=1, y=2) from the row picture.
- [08:40] Column picture of the equations - 2 dimensional vectors.
- [09:50] Linear combination of columns.
- [12:00] Solution from the column picture x=1, y=2.
- [12:05] Plotting the columns to produce the solution vector.
- [15:40] Solving 3 equations with 3 unknowns
- [16:46] Matrix form for this 3x3 equation.
- [17:30] Row picture - planes.
- [22:00] Column picture - 3 dim vectors.
- [24:00] Solution (x=0, y=0, z=1) from the column picture by noticing that z vector is equal to b vector.
- [28:10] Can Ax=b be solved for every b?
- [28:50] Do the linear combinations of columns fill the 3d space?
- [32:30] What if there are 9 equations and 9 unknowns?
- [36:00] How to multiply a matrix by a vector? Two ways.
- [36:40] Ax is a linear combination of columns of A.

Here are my notes of lecture one. Sorry about the handwriting. It seems that I hadn't written much at that time and the handwriting had gotten really bad. But it gets better with each new lecture. At lecture 5 and 6 it will be as good as it gets.

Have fun with this lecture! The next post is going to be about a systematic way to find a solution to a system of equations called elimination.

PS. This course is taught from Introduction to Linear Algebra textbook. Get it here: