youtube video downloader in vbscriptWhen I was a small kid I liked RAD (Rapid Application Development) in Microsoft® Visual Basic® programming language. I mastered the language entirely from MSDN's samples and it was such a joy to create GUI applications in a few hours or days and they just worked! Now it has been a few years years since I have not touched Visual Basic and I have become really rusty in this language.

If you have been reading my blog, you might have noticed that I have been creating these YouTube video downloaders in various programming languages. I have no plans to create a downloader in Visual Basic itself, because I am creating a GUI downloader in C and C++ already but I would like to use a derived language (a subset of Visual Basic language), VBScript, to create the downloader.

That would refresh my knowledge of Visual Basic a little and also let me learn a little more about programming Windows Script Host (WSH).

Here is a quote from a really good introduction to what WSH is:

The first time people encounter Windows Script Host, they often express some confusion. What exactly is WSH? Is it a language, like VBScript or JScript? No; although WSH enables you to run programs written in these languages, it is not a language itself. WSH is a script host. A script host is a program that provides an environment in which users can execute scripts in a variety of languages, languages that use a variety of object models to perform tasks.

You are probably already familiar with other script hosts. Microsoft® Internet Explorer, for example, enables users to execute scripts that use the Dynamic HTML object model. Shell programs (such as C Shell, Bourne Shell and Korn Shell) enable you to write scripts that use an object model capable of manipulating the file system. Even the command prompt can be thought of as a scripting environment because it can run scripts written in the "batch file" language.

WSH is an unusual script host in that it was designed to be general-purpose. Unlike most of the scripting tools mentioned above, WSH imposes restrictions on neither the language used to write scripts nor the object models used by scripts.

WSH is ideal for non-interactive scripting needs such as logon scripting and administrative scripting but we will use it to download YouTube videos as well.

The key advantage of this script is that it will run on any Windows operating system newer than Windows 98. If you have Windows 98 or an older Windows, follow this link and install the latest version of WSH (unfortunately version 5.1 (5.6 is the latest))!

Let's define the interface of the script.

The script can either be run in WScript environment or CScript environment. WScript is the system default on Windows 2000. Characteristics of this environment are that output appears in a pop-up window, and script execution waits until the user clears the pop-up window by pressing OK button. The other environment is the CScript environment. The primary difference between the CScript and WScript environments is that the CScript environment directs display information to the command window, which has a side effect of letting a script run to completion without pausing for each message sent to the UI. I want this script to be usable from both of the environments.

Here is what I am thinking. By default I want this script to be accepting the video URL to download as the first command line argument (followed by more videos as next arguments). If the first argument is not provided and it is run in WScript environment I want the script to pop up an InputBox dialog asking for the URL of the video to download. Otherwise, if it is run in CScript environment, I want it to quit.

How do we find in which environment the script is being run? I had no idea, so I turned to Google Groups and searched for "cscript wscript detect" and found this post which explained how to do it with this snippet of code:

If "CSCRIPT.EXE" = UCase(Right(WScript.Fullname, 11)) Then
    WScript.Echo "The program was run by CScript!"
End If 

The next thing we need to figure out is how to get the command line arguments of a running script.

Turns out that each WSH script has a WScript object available without the script needing to bind to the object. Command-line arguments are stored in the WshArguments collection, which you access through the Arguments property of the WScript object.

Here is a diagram of WScript object's methods, properties and WShArguments expanded:

wsh wscript object's hierarcy

The easiest way to loop over all arguments is shown in this snippet:

Set objArgs = WScript.Arguments
For I = 0 to objArgs.Count - 1
   WScript.Echo objArgs(I)

Now just instead of "WScript.Echo objArgs(I)" we call "DownloadVideo objArgs(I)", where DownloadVideo is our procedure for downloading YouTube videos.

In one of the previous articles, downloading youtube videos with awk programming language, I explained how the embedded YouTube flash video player retrieves the video file itself. Please see that article if you are interested in how I figured it out.

Last two things we have to figure out before we have a running script is how to talk over HTTP with VBScript and how to save the incoming binary data to a file.

Both VBScript and WScript object provide CreateObject function (but there is a difference between them) which allows binding to COM objects.

There is a "Microsoft.XmlHttp" COM Object which is shipped with Internet Explorer 5.0 and later. This object is what actually provides the well known AJAX interface used in Web2.0 applications - the XMLHttpRequest interface. Here at MSDN is the documentation of this interface with all the methods and properties it provides.

When we get the video, we will be dealing with binary data. Microsoft Scripting Runtime provides us with FileSystemObject (FSO) but unfortunately it is not suitable for writing binary files.

There is a way to write binary files with FSO but it is so slow that I dropped this solution. It took more than 10 minutes to write 1MB of data!

' Given a FileName and Data, saves Data to file named FileName
Sub SaveData(FileName, Data)
    Dim Fso: Set Fso = CreateObject("Scripting.FileSystemObject")
    Dim TextStream: Set TextStream = Fso.CreateTextFile(FileName, True)

    WScript.Echo LenB(Data)
    TextStream.Write BinaryToString(Data)
End Sub

' Given Binary data, converts it to a string
Function BinaryToString(Binary)
  Dim I, S
  For I = 1 To LenB(Binary)
    S = S & Chr(AscB(MidB(Binary, I, 1)))
  BinaryToString = S
End Function

A much better way to write binary data to a file with VBScript is using ADODO.Stream Object which deals with binary data out of the box. Look at SaveVideo function in my final program to see how it is used to write binary data to file.

Script Usage

Before the script can be used, you have to tell your computer to trust domain. If you do not do this, executing the script will lead you to the following error:

ytdown.vbs(73, 5) msxml3.dll: Access is denied. domain can be trusted by adding it to trusted sites security zone in Internet Explorer.
Launch your Internet Explorer browser and head to Tools -> Internet Options, then follow the steps illustrated in this image:

internet explorer trusted sites zone security

Once you have trusted domain, you can start downloading the videos.

One of the ways is to double click the ytdown.vbs icon which will launch the script in WScript environment and an input dialog will appear asking for a video to download:

launch ytdown.vbs by double clicking the icon

After you press "OK", the downloader will save the video to a file with the title of the video and .flv extension in the same directory!

The other way to download a video is to call it via command line in CScript environment:

C:\ytdown>cscript ytdown.vbs ""
Microsoft (R) Windows Script Host Version 5.6
Copyright (C) Microsoft Corporation 1996-2001. All rights reserved.

Downloading video '30 fps, 30 frames (1 second)'

Happy downloading!

The downloaded .flv file can be converted to a better format like DivX or .avi.
Read about converting a video to a better format in this article.

Here is the final script:

' Peteris Krumins (
'  -  good coders code, great reuse
' 2007.08.03 v1.0 - initial release
' 2007.10.21 v1.1 - youtube changed the way it displays vids
' 2008.03.01 v1.2 - youtube changed the way it displays vids
' 2009.12.02 v1.3 - youtube changed the way it displays vids
Option Explicit

Dim WscriptMode

' Detect if we are running in WScript or CScript
If UCase(Right(WScript.Fullname, 11)) = "WSCRIPT.EXE" Then
    WScriptMode = True
    WScriptMode = False
End If 

Dim Args: Set Args = WScript.Arguments

If Args.Count = 0 And WScriptMode Then
    ' If running in WScript and no command line args are provided
    ' ask the user for a URL to the YouTube video
    Dim Url: Url = InputBox("Enter a YouTube video URL to download" & vbCrLf & _
                   "For example,", _
                   "YouTube Downloader,")
    If Len(Url) = 0 Then: WScript.Quit 1
    DownloadVideo Url
ElseIf Args.Count = 0 And Not WScriptMode Then
    ' If running in CScript and no command line args are provided
    ' show the usage and quit
    WScript.Echo "Usage: " & WScript.ScriptName & " <video url 1> [video url 2] ..."
    WScript.Quit 1
    ' Download all videos
    Dim I

    For I = 0 to args.Count - 1
        DownloadVideo args(I)
End If

' Downloads a YouTube video and saves it to a file
Sub DownloadVideo(Url)
    Dim Http, VideoTitle, VideoName, Req

    Set Http = CreateObject("Microsoft.XmlHttp") "GET", Url, False

    If Http.status <> 200 Then
        WScript.Echo "Failed getting video page at: " & Url & vbCrLf & _
                     "Error: " & Http.statusText
        Exit Sub
    End If

    Dim VideoId: VideoId = ExtractMatch(Url, "v=([A-Za-z0-9-_]+)")
    If Len(VideoID) = 0 Then
        WScript.Echo "Could not extract video ID from " & Url
        Exit Sub
    End If

    VideoTitle = GetVideoTitle(Http.responseText)
    If Len(VideoTitle) = 0 Then
        WScript.Echo "Failed extracting video title from video at URL: " & Url & vbCrLf & _
                     "Will use the video ID '" & VideoID & "' for the filename."
        VideoName = VideoID
        VideoName = VideoTitle
    End If

    Dim FmtMap: FmtMap = GetFmtMap(Http.responseText)
    If Len(FmtMap) = 0 Then
        WScript.Echo "Could not extract fmt_url_map from the video page."
        Exit Sub
    End If

    Dim VideoURL: VideoURL = Find_Video_5(FmtMap)
    If Len(VideoURL) = 0 Then
        WScript.Echo "Could not extract fmt_url_map from the video page."
        Exit Sub
    End If

    If WScriptMode = False Then: WScript.Echo "Downloading video '" & VideoName & "'" "GET", VideoURL, False

    If Http.status <> 200 Then
        WScript.Echo "Failed getting the flv video: " & Url & vbCrLf & _
                     "Error: " & Http.statusText
        Exit Sub
    End If

    Dim SaneFilename
    SaneFilename = MkFileName(VideoName)

    SaveVideo SaneFilename, Http.ResponseBody
    WScript.Echo "Done downloading video. Saved to " & SaneFilename & "."
End Sub

' Given fmt_url_map, url-escapes it, and finds the video url for video
' with id 5, which is the regular quality flv video.
Function Find_Video_5(FmtMap)
    FmtMap = Unescape(FmtMap)
    Find_Video_5 = ExtractMatch(FmtMap, ",?5\|([^,]+)")
End Function

' Given YouTube Html page, extract the fmt_url_map parameter that contains
' the URL to the .flv video
Function GetFmtMap(Html)
    GetFmtMap = ExtractMatch(Html, """fmt_url_map"": ""([^""]+)""")
End Function

' Given YouTube Html page, the function extracts the title from <title> tag
Function GetVideoTitle(Html)
    ' get rid of all tabs
    Html = Replace(Html, Chr(9), "")

    ' get rid of all newlines (vbscript regex engine doesn't like them)
    Html = Replace(Html, vbCrLf, "")
    Html = Replace(Html, vbLf, "")
    Html = Replace(Html, vbCr, "")

    GetVideoTitle = ExtractMatch(Html, "<title>YouTube ?- ?([^<]+)<")
End Function

' Given the Title of a video, function creates a usable filename for a video by
' sanitizing it - stripping parenthesis, changing non alphanumeric characters
' to _ and adding .flv extension
Function MkFileName(Title)
    Title = Replace(Title, "(", "")
    Title = Replace(Title, ")", "")

    Dim Regex
    Set Regex = New RegExp
    With Regex
        .Pattern = "[^A-Za-z0-9-_]"
        .Global = True
    End With

    Title = Regex.Replace(Title, "_")
    MkFileName = Title & ".flv"
End Function

' Given Text and a regular expression Pattern, the function extracts
' the first submatch
Function ExtractMatch(Text, Pattern)
    Dim Regex, Matches

    Set Regex = New RegExp
    Regex.Pattern = Pattern

    Set Matches = Regex.Execute(Text)
    If Matches.Count = 0 Then
        ExtractMatch = ""
        Exit Function
    End If

    ExtractMatch = Matches(0).SubMatches(0)
End Function

' Function saves Data to FileName
Function SaveVideo(FileName, Data)
  Const adTypeBinary = 1
  Const adSaveCreateOverWrite = 2
  Dim Stream: Set Stream = CreateObject("ADODB.Stream")
  Stream.Type = adTypeBinary
  Stream.Write Data
  Stream.SaveToFile FileName, adSaveCreateOverWrite
End Function

' ==========================================================================
' The following code saves binary data to file using FileSystemObject
' It is so slow that even on a 3.2Ghz computer saving 1 MB takes 10 minutes!
' Don't use it! I put it here just to illustrate the wrong solution!
' ==========================================================================

' Given a Filename and Data, the function saves Data to File
'Sub SaveVideo(File, Data)
'    Dim Fso: Set Fso = CreateObject("Scripting.FileSystemObject")
'    Dim TextStream: Set TextStream = Fso.CreateTextFile(File, True)
'    WScript.Echo LenB(Data)
'    TextStream.Write BinaryToString(Data)
'End Sub

' Given Binary data, converts it to a string
'Function BinaryToString(Binary)
'  Dim I, S
'  For I = 1 To LenB(Binary)
'    S = S & Chr(AscB(MidB(Binary, I, 1)))
'  Next
'  BinaryToString = S
'End Function

' ==========================================================================
' The following is an implementation of UrlUnescape. It turned out VBScript
' has Unescape() function built in already, that does it!
'Function UrlUnescape(Str)
'    Dim Regex, Match, Matches
'    Set Regex = New RegExp
'    With Regex
'        .Pattern = "%([0-9a-f][0-9a-f])"
'        .IgnoreCase = True
'        .Global = True
'    End With
'    ' Wanted to do this, but it wasn't quite possible
'    ' UrlUnescape = Regex.Replace(Str, Chr(CInt("&H" & $0)))
'    Set Matches = Regex.Execute(Str)
'    For Each Match in Matches
'        Str = Replace(Str, Match, Chr(CInt("&H" & Match.SubMatches(0))))
'    Next
'    UrlUnescape = Str
'End Function

Download Visual Basic Script YouTube Video Downloader

Download link: vbscript youtube video downloader
Total downloads: 28813 times

If you notice any bugs in this script, have any recommendations, want to criticize my code, want to thank me, then just post a comment on this article!

ps. remember the ILOVEYOU virus? It was also written in VBScript, that alone indicates that this language is worth knowing.

This article is part of the article series "GUI Video Downloader."
<- previous article next article ->

video downloader software, vim, mingw, wxwidgetsThis is a continuation of multi-part series on how to create a cross-platform desktop video downloading application. In part I we prepared our coding environment with vim, various vim plugins, mingw and gdb. In this part we will start creating a C library for downloading just the YouTube videos with modularity in mind, so that we could create a nice desktop application capable of downloading videos from other video sharing sites like break, metacafe, dailymotion and others. Complementary to this library we will be also be creating a command line application which will use this C library. The command line application will serve us as a test tool for the library, so we know the library works ok. Once we have these two components, we will build a GUI around the library and create the desktop app.

Writing the Library

I have chosen a name libvd for the library which stands for video download library.

The library will be written in C programming language because:

  • it is the number one cross platform language,
  • I haven't programming in C for a while, so it will be a great refresher,
  • the GUI framework which I will be using is written in C++ which will have no problems with using C code
  • the application will be small and will require no unnecessary library/toolkit dependencies

Downloading a video means working with HTTP protocol. Let's use "good coders code, great reuse" approach and seek for a C library which would ease the usage of HTTP protocol.

Having worked on a few projects in C programming language involving HTTP protocol, I know an excellent library. It's called libcurl. I even implemented cookie interface for this library which was missing two years ago.

What is libcurl? Quoting the author of libcurl: "libcurl is a free and easy-to-use client-side URL transfer library, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, FILE and LDAP. libcurl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, Kerberos4), file transfer resume, http proxy tunneling and more!

libcurl is highly portable, it builds and works identically on numerous platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HPUX, IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Amiga, OS/2, BeOs, Mac OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS and more..."

Woah! Much more than we need! And notice how cross platform it is! Exactly what we are looking for!

Let's download the libcurl library. Since I am developing the program on a fresh Windows XP virtual machine, I chose to download version of the library. It really does not matter what operating system I am developing this program on, I could have as well chosen Linux, then I would have downloaded the library ending with extension .gz or .bz2.

download libcurl for windows

Compiling libcurl

Once we have downloaded libcurl, let's unpack it and look for instructions on how to compile it with MinGW on a Windows machine. I chose c:\vd as the root directory for the project on the windows machine because I am afraid of those whitespace characters windows has in path. I unpacked the libcurl library to c:\vd\curl.

At this point we have no idea how to build the libcurl on Windows, right? So let's look in the docs subdirectory (C:\vd\curl\docs) for a clue. Usually the instructions on how to build a library on a particular platform can be found in files like README.platform or INSTALL.platform or just INSTALL. Indeed, libcurl has a file README.win32. Let's read it.

libcurl's win32 readme file

Oops, it says to read README first, we look in the same docs subdirectory but there is no README! We look in the parent directory, whew, yeah there is a README. But it is just 1KB in size and says nothing...

The only reasonable file left is INSTALL, let's look at it. Oh yeah, it's 29K in size! Must be it!

Indeed, scrolling a few screens down by pressing vim's Ctrl_D, we find instructions on how to build libcurl on win32 with MinGW compiler!

libcurl's win32 install file with mingw

Libcurl also provides support for compressed HTTP content with zlib, and secure connections over HTTPS with OpenSSL. The INSTALL file mentions what to do to get support for these thingies. We do not need them, so we ignore them.

Don't rush yet compiling the library! We also want our application to be small, not just portable. Libcurl has support for so many protocols but our project need just the HTTP protocol!

Looking a little further in the INSTALL file, we find the following instruction:

   Disabling Specific Protocols in Win32 builds

   The configure utility, unfortunately, is not available for the Windows
   environment, therefore, you cannot use the various disable-protocol
   options of the configure utility on this platform.

   However, you can use the following defines to disable specific

   HTTP_ONLY             disables all protocols except HTTP

   If you want to set any of these defines you have the following

   - Modify lib/setup.h

We really want just the HTTP protocol, so let's follow this instruction and modify setup.h file in lib directory (file C:\vd\curl\lib\setup.h) and add the following line right after the comments:

#define HTTP_ONLY

The file should now look like this:

modifying libcurl to make it support HTTP only!

Okay, now we are ready to compile libcurl!

The INSTALL file says that to compile the libcurl library we first need to run 'mingw32.bat' and then 'make mingw32'.


   Run the 'mingw32.bat' file to get the proper environment variables set,
   then run 'make mingw32' in the root dir. Use 'make mingw32-ssl' to build
   curl SSL enabled.

mingw32.bat? I don't have such file in my mingw32 distribution! Now what? Let's be real developers and compile the library ourselves then. Let's make sure the MinGW's bin directory (in my case C:\MinGW\bin) is in our PATH environment variable and let's run the second command - 'make mingw32'.


C:\vd\curl>make mingw32
'make' is not recognized as an internal or external command,
operable program or batch file.

There is no program named 'make' in MinGW's distribution, there is only mingw32-make.exe, let's run it:

C:\vd\curl>c:\MinGW\bin\mingw32-make.exe mingw32
c:/MinGW/bin/mingw32-make -C lib -f Makefile.m32 ZLIB=1
mingw32-make[1]: Entering directory `C:/vd/curl/lib'
gcc -I. -I../include -I"../../zlib-1.2.3" -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -DHAVE_LIBZ -DHAVE_ZLIB_H -c file.c
gcc -I. -I../include -I"../../zlib-1.2.3" -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -DHAVE_LIBZ -DHAVE_ZLIB_H -c timeval.c
gcc -I. -I../include -I"../../zlib-1.2.3" -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -DHAVE_LIBZ -DHAVE_ZLIB_H -c base64.c
In file included from base64.c:43:
urldata.h:92:59: zlib.h: No such file or directory
In file included from base64.c:43:

Huh? I never wanted my libcurl to be built with zlib support. Let's get rid of it by editing the Makefile (C:\vd\curl\Makefile) directly. Search for mingw32 target (string "mingw32:") and you'll find:

	$(MAKE) -C lib -f Makefile.m32 ZLIB=1
	$(MAKE) -C src -f Makefile.m32 ZLIB=1

We really don't want to build the curl executable, so let's get rid of the last line and also get rid of "ZLIB=1" variable definition.
The lines should look now like this:

	$(MAKE) -C lib -f Makefile.m32

Now let's run the mingw32-make again:

C:\vd\curl>c:\MinGW\bin\mingw32-make.exe mingw32
c:/MinGW/bin/mingw32-make -C lib -f Makefile.m32
mingw32-make[1]: Entering directory `C:/vd/curl/lib'
gcc -I. -I../include -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -c base64.c
gcc -I. -I../include -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -c hostip.c
rm -f libcurl.a
process_begin: CreateProcess(NULL, rm -f libcurl.a, ...) failed.
make (e=2): The system cannot find the file specified.
mingw32-make[1]: *** [libcurl.a] Error 2
mingw32-make[1]: Leaving directory `C:/vd/curl/lib'
mingw32-make: *** [mingw32] Error 2

Argh! There is no 'rm' on Windows, at least not with MinGW development tools package. Let's modify the Makefile.m32 file (C:\vd\curl\lib\Makefile.m32) and change 'rm' to Windows 'del' command.

Find this line in Makefile.m32:

RM = rm -f

and change it to

RM = del /F

Perfect! Running the mingw32-make.exe again finishes with no errors! Lookin in the C:\vd\curl\lib directory we find the static library libcurl.a and the dynamic library libcurl.dll. Had we compiled it on Linux, the dynamic library would have been named

Testing libcurl

I wrote the following test program which outputs the website of to the standard output if libcurl works correctly:

#include <stdio.h>
#include <stdlib.h>
#include "curl/curl.h"

int main(void) {
    CURLcode ret;
    CURL *curl = curl_easy_init();
    if (curl == NULL) {
        fprintf(stderr, "Failed creating CURL easy handle!\n");

    /* let's get because I love google */
    ret = curl_easy_setopt(curl, CURLOPT_URL, "");
    if (ret != CURLE_OK) {
        fprintf(stderr, "Failed getting %s\n",

    ret = curl_easy_perform(curl);
    if (ret != 0) {
        fprintf(stderr, "Failed getting %s\n",

    return 0;

I placed this program in C:\vd\tests\curltest directory and named the source file testcurl.c. Our curl library is located in C:\vd\curl\lib and the curl header files in C:\vd\curl\include. Assuming out currenly working directory is C:\vd\tests\curltest, the following command line will compile the C source file to an executable:

gcc testcurl.c -o testcurl -DCURL_STATICLIB -I..\..\curl\include -L..\..\curl\lib -lcurl -lwsock32 -lwinmm -Wall

It will create testcurl.exe executable which can be run. Let's try running it:

<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">

Yeah! It works! Now we are ready to write the libvd (video download) library which will use the libcurl for communicating over HTTP protocol.

I'll do this in Part III of this series. Until then, have fun compiling libcurl!

I have always had historic interest of various UNIX tools and the UNIX itself. One day when I was learning sed, the stream editor, (sed cheat sheet) I got interested how much the most powerful text editing tools on UNIX - 'ed', 'ex', 'sed' and 'vim' (or just 'vi') had in common and what they borrowed from one another.

I was not perfectly sure if 'sed' has originated from 'ed', so I Google searched for 'sed history' which turned up some interesting results, one of them being "The original users manual for sed, by Lee E. McMahon, from the 7th edition UNIX Manual (1978)".

Quoting the manual "Sed is a lineal descendant of the UNIX editor, ed." Great! I knew that 'sed' was non-interactive text editor whereas 'ed' was an interactive one. Also I knew that 'ex' was improved 'ed' and that 'sed' was a non-interactive 'ed' and that 'ex' had become 'vi' and was used in 'vi' command line mode. To boost my vi/vim skills and get a touch on how text editing was done with just 'ed', and satisfy my historical curiosity I set myself a goal to learn it.

As I mentioned in awk and sed cheat sheet posts, the best method for me to learn a new tool is to have all commands in front of me, so when I am doing experimentation I can quickly find the command and also see all the other commands at the same time, blowing them in subconsciousness.

Since 'ed' is line-oriented text editor, it is important to understand the line addressing, this cheat sheet summarizes the line addressing.

Most of 'ed' command are single letter in length. This cheat sheet summarizes all the commands, showing the supported address ranges and a detailed description of each command.

From my experience, once I had completed this cheat sheet and had it in front of me, I picked 'ed' up in 30 minutes. And then spent a few more hours experimenting and trying various constructs.

Have fun learning ed!

Download Ed UNIX Text Editor's Cheat-Sheet

Plain Text (.txt):
Download link: ed text editor cheat sheet (.txt)
Downloaded: 31071 times

Download link: ed text editor cheat sheet (.pdf)
Downloaded: 49781 times

Microsoft Word 2000 format (.doc):
Download link: ed text editor cheat sheet (.doc)
Downloaded: 10504 times

upload youtube perlWe have been downloading and converting YouTube videos but have not looked at how to upload them.

I will teach you how to upload YouTube videos via a command line. To find out how a video is being uploaded we will need just the Firefox browser!

Here is a typical scenario when you would want to automatically upload your videos to YouTube. Suppose you were using some other video sharing site and had already uploaded like 100 of your videos there. To get more popular on the net you'd also want to get your videos on YouTube, right? Doing it manually is a boring and tedious job, you want a program to do it for you while you sit back and relax.

Finding the YouTube Upload Form's Elements

Log into your account and go to "My Account" menu in the upper right. Then press the "Upload New Video" button.

how to upload youtube videos

Now let's use Firefox's "Page Info" tool which is located under "Tools" menu.

firefox's page info built in tool

When the "Page Info" tool pops up, select the "Forms" tab. The tool will list all the HTML forms on the page. The one named "theForm" with action's url "" is the upload form! When a user uploads the video, it gets submitted. Our program will submit this form for us.

youtube upload video page's forms, fields, values

Now that we have found all the fields we need to submit, we can write the tool itself. I'll use my favorite programming language - Perl, again. It is perfect for this job because it has extremely good packages for working with HTTP protocol.
I will probably write another post just about the Perl programming language - how to get it working on Windows Operating System. For now, if you are running Windows, then you can download ActiveState's ActivePerl which is Perl's port to win.

Writing the Program

As I said, we will be creating the tool in Perl programming language. In the previous post about YouTube I used the WWW::Mechanize package. I can tell you in advance that it will not work this time because there will be some unexpected surprises, such as hidden form elements being created by Javascript when the form is submitted. Since WWW::Mechanize stores all the forms internally, it will not be able to submit this form with these dynamic form elements. That's why we will use the base class of WWW::Mechanize - the LWP::UserAgent.

LWP::UserAgent package can be installed with the following command:

perl -MCPAN -e 'install LWP::UserAgent'

Once you have it installed, we can start writing the program.

Step 1: Logging in to YouTube

Before we can upload a video, we need to login to YouTube. Let's use the same procedure as before to find out what form do we need to submit to login. Let's go to this url:

and look at all the forms on this page. One of them is named "loginForm" with four fields. We will need to submit this form and save the cookies. Luckily LWP::UserAgent can do it for us and we will not have to parse HTTP headers or anything like that.

YouTube's login form fields, types and their values

We see that there is no action URL provided for the loginForm, that means that the form should be POSTed to the same URL it was located on, in our case:

We also see that the form has 4 elements which should be submitted:

  • 'current_form' with value 'loginForm'
  • 'username' with our youtube login/username
  • 'password' with our password, and
  • 'action_login' with value 'Log In'

How would we submit this form with LWP::UserAgent? Easy - it has a member function $ua->post( $url, \%form, $header_field_name => $value, ... ), where %form is a hash of key, value pairs of form elements and $header_field_name => $value is a pair of HTTP headers we would like to add to the request.

Given an object of LWP::UserAgent named $ua, code which would login us to YouTube is the following:

        current_form => 'loginForm',
        username     => 'youtube_login',
        password     => 'youtube_password',
        action_login => 'Log In'

See how easy it was? I love it.

The only problem is how to detect if we have really logged in. I noticed that when we log in, the upper right menu has a 'Log Out' button. So if we logged in succesfully, we should find this button.

Step 2: Uploading the Video

Now, once we have logged in, let's upload the video.

Here is the image with all the fields we have to submit to upload a video, once more:

youtube upload video page's forms, fields, values

There are 17 fields totally:

  • "field_myvideo_title" - title of the video
  • "field_myvideo_descr" - description of the video
  • "field_myvideo_keywords" - comma separated tags
  • "field_myvideo_categories" - video category
  • "language" - language of the video, i'll leave it "EN" for English
  • "action_upload" - a string "Upload a video..."
  • "session_token" - we will have to extract the session token for this upload
  • "allow_embeddings" - should the video be allowed to be embedded on other sites, i'll leave the default "Yes"
  • "allow_responses" - should we allow video responses, i'll leave the default "Yes"
  • "allow_comments" - should we allow comments on the video, i'll leave the default "Yes"
  • "allow_ratings" - should we allow users to rate the vide, i'll leave the default "Yes"
  • "location" - latitude and longitude of the location video was shot, i'll leave it empty
  • "field_date_mon" - month the video was shot, i'll use default 0 (unknown)
  • "field_date_day" - day the video was shot, i'll use 0 (unknown)
  • "field_date_yr" - year the video was shot, i'll use 0 (unknown)
  • "field_privacy" - privacy of video (public or private), i'll use the default - "public"
  • "ignore_broadcast_settings" - no idea what this is, i'll use the default value 0

The trickiest part here is to extract the "session_token" which is some kind of unique id.
Going through the HTML source of upload page:

We find a javascript function called "dynamic_append_session_token" which dynamically creates a hidden form element containing the session ID:

finding the session id of youtube upload form

We have to extract this session ID from the line in red. It can be done with the following regular expression:

token_elem\.setAttribute\('value', '(.+?)'\);

Update 2008.03.12: This is no longer true! A session cookie is represented by a variable now:

var gXSRF_token = 'OTxXiSg8O-oQFB-PWTSaDv7oX2V8MTIwNTQ1NzYxNQ==';

When we have done this, we can submit the form. For this form we will need to add a "Content-Type: form-data" header to the request because the upload form has explicitly stated that its "enctype" is "multipart/form-data".

As it turns out the actual video upload process is a two step process. This was the first step which set the video information. After we submit this form, we get redirected to a page which contains the upload file selection form. It might seem that that is the only element in this form but not so. Actually this 2nd step form has a few new fields and it also has all the fields from 1st step as hidden elements. Here are the modified and added fields in the 2nd step:

  • "contact" - no idea what this field is for
  • "field_command" - always "myvideo_submit"
  • "field_uploadfile" - our video file we want to upload
  • "field_private_share_entities" - no idea
  • "action_upload" - submit button's caption "Upload Video"
  • "addresser" - another mysterious unique id

Here is what page info tool shows us about 2nd step's form elements:

youtube upload video 2nd step

Notice that the form's action URL changes, we have to extract it from the html source. Someone might suggest not to use regexes for parsing HTML but it is perfectly ok for this tiny project.

The trickiest thing here is telling LWP::UserAgent that 'field_uploadfile' is a form element of 'file' type. If we look in the LWP::UserAgent's documentation we find that the three argument post() subroutine uses "POST() function from HTTP::Request::Common to build the request." Further looking in the HTTP::Request::Common documentation we find that to specify a file to be uploaded the field has to be an array ref looking like this: [ $file, $filename, Header => Value... ].

That's it. When we submit this form the video should have been uploaded. Now we just need to detect if the upload was successful. I noticed that when the video has uploaded successfully, YouTube thanks me for uploading. It looks like this:

youtube video upload complete

So checking if a video uploaded successfully is reduced to checking for the thank you message.

The program can be used from command line and takes a few mandatory arguments:

Usage: perl -l [login] -p [password] -f <video file> -c <category> -t <title> -d <description> -x <comma, separated, tags>

The login (-l) and password (-p) arguments are not mandatory because not to expose your password on a publically shared machine (by looking at process list) they can be specified in the source file by changing YT_LOGIN and YT_PASS constants.

Possible categories (for -c switch):
2    - Autos & Vehicles
23   - Comedy
24   - Entertainment
1    - Film & Animation
20   - Gadgets & Games
26   - Howto & DIY
10   - Music
25   - News & Politics
22   - People & Blogs
15   - Pets & Animals
17   - Sports
19   - Travel & Places

Here is the final code:

# Version 1.0: 2007.07.30
# Version 1.1: 2007.10.12: youtube changed html
# Version 1.2: 2008.03.13: youtube changed html
# Version 1.3: 2009.12.02: youtube changed html and now logins with google login
# Peter Krumins (
#  --  good coders code, great reuse

use constant VERSION => "1.3";

use strict;
use warnings;

use LWP::UserAgent;
use HTML::Entities 'decode_entities';
use Getopt::Std;

# debug?
use constant DEBUG => 0;

# set these values for default -l (login) and -p (pass) values
use constant YT_LOGIN => "";
use constant YT_PASS  => "";

# video categories
my %cats = (
    2   =>  'Autos & Vehicles',
    23  =>  'Comedy',
    27  =>  'Education',
    24  =>  'Entertainment',
    1   =>  'Film & Animation',
    20  =>  'Gaming',
    26  =>  'Howto & Style',
    10  =>  'Music',
    25  =>  'News & Politics',
    29  =>  'Nonprofits & Activism',
    22  =>  'People & Blogs',
    15  =>  'Pets & Animals',
    28  =>  'Science & Technology',
    17  =>  'Sports',
    19  =>  'Travel & Places'

# various urls
my $login_url      = '';
my $login_post_url = '';
my $login_cont_url = '';
my $upload_url     = '';
my $upload_video_url1 = '';
my $upload_video_url2 = '';
my $upload_video_url3 = '';
my $upload_video_url4 = '';
my $upload_video_set_info = '';

unless (@ARGV) {
    exit 1;

my %opts;
getopts('l:p:f:c:t:d:x:', \%opts);

# if -l or -p are not given, try to use YT_LOGIN and YT_PASS constants
unless (defined $opts{l}) {
    unless (length YT_LOGIN) {
 print "Youtube username/login as neither defined nor passed as an argument\n";
        print "Use -l switch to specify the username\n";
        print "Example: -l joe_random\n";
        exit 1;
    else {
        $opts{l} = YT_LOGIN;

unless (defined $opts{p}) {
    unless (length YT_PASS) {
        print "Password was neither defined nor passed as an argument\n";
        print "Use -p switch to specify the password\n";
        print "Example: -p secretPass\n";
        exit 1;
    else {
        $opts{p} = YT_PASS;

unless (defined $opts{f} && length $opts{f}) {
    print "No video file was specified\n";
    print "Use -f switch to specify the video file\n";
    print 'Example: -f "C:\Program Files\movie.avi"', "\n";
    print 'Example: -f "/home/pkrumins/"', "\n";
    exit 1;

unless (-r $opts{f}) {
    print "Video file is not readable or does not exist\n";
    print "Check the permissions and the path to the file\n";
    exit 1;

unless (defined $opts{c} && length $opts{c}) {
    print "No video category was specified\n";
    print "Use -c switch to set the category of the video\n";
    print "Example: -c 20, would set category to Gadgets & Games\n\n";
    exit 1;

unless (defined $cats{$opts{c}}) {
    print "Category '$opts{c}' does not exist\n\n";
    exit 1;

unless (defined $opts{t} && length $opts{t}) {
    print "No video title was specified\n";
    print "Use -t switch to set the title of the video\n";
    print 'Example: -t "Super Cool Video Title"', "\n";
    exit 1;

unless (defined $opts{d} && length $opts{d}) {
    print "No video description was specified\n";
    print "Use -d switch to set the description of the video\n";
    print 'Example: -d "The coolest video description"', "\n";
    exit 1;

unless (defined $opts{x} && length $opts{x}) {
    print "No tags were specified\n";
    print "Use -x switch to set the tags\n";
    print 'Example: -x "foo, bar, baz, hacker, purl"', "\n";
    exit 1;

# tags should be at least two chars, can't be just numbers
my @tags = split /,\s+/, $opts{x};
my @filtered_tags = grep { length > 2 && !/^\d+$/ } @tags;
unless (@filtered_tags) {
    print "Tags must at least two chars in length and must not be numeric!\n";
    print "For example, 'foo', 'bar', 'yo29' are valid tags, but ";
    print "'22222', 'hi', 'b9' are invalid\n";
    exit 1;

$opts{x} = join ', ', @filtered_tags;
# create the user agent, have it store cookies and
# pretend to be a cool windows firefox browser
my $ua = LWP::UserAgent->new(
    cookie_jar => {},
    agent => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: '.
             'Gecko/20070515 Firefox/'

if (DEBUG) {
    $ua->add_handler("request_send",  sub { print "Request: \n"; shift->dump; print "\n"; return });
    $ua->add_handler("response_done", sub { print "Response: \n"; shift->dump; print "---\n"; return });

# let the user agent follow redirects after a POST request
push @{$ua->requests_redirectable}, 'POST';

print "Logging in to YouTube...\n";

print "Uploading the video ($opts{t})...\n";

print "Done!\n";

sub login {
    # go to login page to get redirected to google sign in page
    my $res = $ua->get($login_url);
    unless ($res->is_success) {
        die "Failed going to YouTube's login URL: ", $res->status_line;

    # extract GALX id
    my $GALX = extract_field($res->content, qr/name="GALX".+?value="([^"]+)"/s);
    unless ($GALX) {
        die "Failed logging in. Unable to extract GALX identifier from Google's login page.";

    # submit the login form
    $res = $ua->post($login_post_url,
            ltmpl    => 'sso',
            continue => '',
            service  => 'youtube',
            uilel    => 3,
            hl       => 'en_US',
            GALX     => $GALX,
            Email    => $opts{l},
            Passwd   => $opts{p},
            rmShown  => 1,
            signIn   => 'Sign in',
            asts     => ''
    unless ($res->is_success) {
        die "Failed logging in: failed submitting login form: ", 

    # Get the meta http-equiv="refresh" url
    my $next_url = extract_field($res->content, qr/http-equiv="refresh" content="0; url='(.+?)'/);
    unless ($next_url) {
        die "Failed logging in. Getting next url from http-equiv failed.";
    $next_url = decode_entities($next_url);

    $res = $ua->get($next_url);
    unless ($res->is_success) {
        die "Failed logging in. Extraction of next_url failed: ",

    $res = $ua->get($login_cont_url);
    unless ($res->is_success) {
        die "Failed logging in. Navigation to failed: ", 

    # We have no good way to check if we really logged in.
    # I found that when you have logged in the upper right menu changes
    # and you have access to 'Sign Out' option.
    # We check for this string to see if we have logged in.
    unless ($res->content =~ />Sign Out</) {
        die "Failed logging in: username/password incorrect";

sub upload {
    # upload is actually a multistep process
    # get upload page to extract some gibberish info
    my $resp = $ua->get($upload_url);
    unless ($resp->is_success) {
        die "Failed getting $upload_url: ", $resp->status_line;

    my $SI = extract_field($resp->content, qr/"sessionKey": "([^"]+)"/);
    unless ($SI) {
        die "Failed extracting sessionKey. YouTube might have redesigned!";

    my $UK = extract_field($resp->content, qr/"uploadKey": "([^"]+)"/);
    unless ($UK) {
        die "Failed extracting uploadKey. YouTube might have redesigned!";

    my $session_token = extract_field($resp->content, qr/"session_token" value="([^"]+)"/);
    unless ($session_token) {
        die "Failed extracting session_token. YouTube might have redesigned!";

    prepare_upload_urls($SI, $UK);

    # tell the server that we are up for uploads
    $resp = $ua->get($upload_video_url1);
    unless ($resp->is_success) {
        die "Failed getting upload_video_url1: ", $resp->status_line;

    # now lets post some more gibberish data
my $post_data_gibberish =<<EOL;
{"protocolVersion":"0.7","createSessionRequest":{"fields":[{"external":{"name":"file","filename":"$opts{f}","formPost":{}}},{"inlined":{"name":"return_address","content":"","contentType":"text/plain"}},{"inlined":{"name":"upload_key","content":"$UK","contentType":"text/plain"}},{"inlined":{"name":"action_postvideo","content":"1","contentType":"text/plain"}},{"inlined":{"name":"live_thumbnail_id","content":"$SI.0","contentType":"text/plain"}},{"inlined":{"name":"parent_video_id","content":"","contentType":"text/plain"}},{"inlined":{"name":"allow_offweb","content":"True","contentType":"text/plain"}},{"inlined":{"name":"uploader_type","content":"Web_HTML","contentType":"text/plain"}}]},"clientId":"scotty html form"}

    $resp = $ua->post($upload_video_url2, Content => $post_data_gibberish);
    unless ($resp->is_success) {
        die "Failed posting gibberish to upload_video_url2: ", $resp->status_line

    # extract the file upload url
    my $file_upload_url = extract_field($resp->content, qr/"url":"([^"]+)"/);
    unless ($file_upload_url) {
        die "Failed extracting file upload url. YouTube might have redesigned!";

    # now lets tell the server that we are starting to upload
    $resp = $ua->get($upload_video_url3);
    unless ($resp->is_success) {
        die "Failed getting upload_video_url3: ", $resp->status_line;

    # now lets post the video
    $resp = $ua->post($file_upload_url, 
            Filedata => [ $opts{f} ]
        "Content_Type" => "form-data"

    unless ($resp->is_success) {
        die "Failed uploading the file: ", $resp->status_line;

    my $video_id = extract_field($resp->content, qr/"video_id":"([^"]+)"/);

    # tell the server that we are done
    $resp = $ua->get($upload_video_url4);
    unless ($resp->is_success) {
        die "Failed getting upload_video_url4: ", $resp->status_line;

    # finally set the video info
    $resp = $ua->post($upload_video_set_info,
            session_token     => $session_token,
            action_edit_video => 1,
            updated_flag      => 0,
            video_id          => $video_id,
            title             => $opts{t},
            description       => $opts{d},
            keywords          => $opts{x},
            category          => $opts{c},
            privacy           => 'public'

    unless ($resp->is_success) {
        die "Failed setting video info (but it uploaded ok!): ", $resp->status_line;

    if ($resp->content =~ /"errors":\s*\[(.+?)\]/) {
        die "The video uploaded OK, but there were errors setting the video info:\n", $1;

sub prepare_upload_urls {
    my ($SI, $UK) = @_;
    for ($upload_video_url1, $upload_video_url3, $upload_video_url4)

sub extract_field {
    my ($content, $rx) = @_;
    if ($content =~ /$rx/) {
        return $1

    print "Usage: $0 ",
          "-l [login] ", 
          "-p [password] ",
          "-f <video file> ",
          "-c <category> ",
          "-t <title> ",
          "-d <description> ",
          "-x <comma, separated, tags>\n\n";

sub print_cats {
    print "Possible categories (for -c switch):\n";
    printf "%-4s - %s\n", $_, $cats{$_} foreach (sort {
        $cats{$a} cmp $cats{$b}
    } keys %cats);

    print "Version: v", VERSION, "\n";

sub preamble {
    print "YouTube video uploader by Peter Krumins (peter\\n";
    print "  --  good coders code, great reuse\n";
    print "\n"


Download link: youtube video uploader (perl script)
Total downloads: 18752 times

After you download the script ytup.perl, rename it back to I could not get the webserver to stop interpreting .pl extension as a CGI script, so I renamed it to ytup.perl.

Thanks for reading the tutorial! If you notice any inaccuracies, bugs or just have something to say, comment on this article!

This article is part of the article series "GUI Video Downloader."
<- previous article next article ->

video downloader software, vim, mingw, wxwidgetsIn my previous posts I showed you how to download YouTube videos and how to convert them to a better format. Wouldn't it be great to use the skills learned and create a small, cool desktop application that downloads YouTube videos (and later extend it to more video sites) and converts them to whatever format you want? I bet it would!

Let's get hands on it!

This will be a multi-part article on how to create this application. As the title of my blog "good coders code, great reuse" suggests, I will reuse as many software components as possible. I will use a free compiler, free gui framework and any free libraries I can find useful. Also I will try to take as many screenshots as possible to make this tutorial newbie friendly. For this reason I just installed a fresh copy of Windows XP on my VMware virtual machine (for those of you who don't know what virtual machine is, here is a quote from Intro to Virtualization: "Virtualization allows multiple virtual machines, with heterogeneous operating systems to run in isolation, side-by-side on the same physical machine.") which I will be developing the software from scratch on.

Since I am using the "good coders code, great reuse" principle, naturally I want parts of my code to be reusable. That's why I will first create a neat video downloading library in C which I will reuse later in the test command line application and the final GUI application which will probably be written in C++ using some cross-platform GUI framework/toolkit.

Part I: Preparation of the Tools

Since I am doing this project from scratch on a naked WinXP machine, the first thing I personally need need is a good browser. I will be searching for tools, libraries and will read documentation a lot, so I need a good browser. My choice - FireFox.

Next I will need a text editor to write the code in. My choice for this project - Vim. Since I want to create this project from scratch, I will need some essential plugins for vim, otherwise editing the source files would take me forever. From all the plugins I usually use in my vim configuration, for this project I'll use: a to quickly switch between .c and .h source files, bufexplorer to quickly navigate around buffers, bufpos to quickly switch between buffers, c-support to quickly write C/C++ code (it has key mapping cheat-sheet in the plugin's archive), surround to quickly change/add/insert quotes, parens etc, and taglist to navigate around source files. To use the last plugin (taglist) we also need Exuberant Ctags tool. Installation process of all these programs and plugins is straight forward.

I installed vim to "C:\program files\vim" and ctags to "C:\ctags56".

To compile the project we will also need a descent compiler. I will choose MinGW which is "a collection of freely available and freely distributable Windows specific header files and import libraries combined with GNU toolsets that allow one to produce native Windows programs that do not rely on any 3rd-party C runtime DLLs." Getting to run MinGW is not as easy as the previous tools so if you are not sure just download the automatic installer and it will do everything for you. The automatic installer can be downloaded from 'Current' package version release of MinGW at SourceForge.

I installed mingw using automated installer to "C:\mingw".

We still need a debugger. Going quickly through MinGW's files at SourceForge I couldn't find the GDB package. So I joined #mingw on FreeNode IRC network and in a few minutes someone helped me by suggesting that it was located in snapshot release. Thank you!

[17:15:49] * Now talking in #mingw
[17:16:07] <pkrumins> hi, what debugger could I use with mingw? I don't see gdb coming with any of the packages, did i miss it?
[17:20:30] <Vinky> pkrumins, snapshot->gdb

IRC is a powerful resource to get help at. Learn to use it.

Next we need a cross platform GUI toolkit. I am pretty familiar with wxWidgets and have used it in a few projects, so I will use for this project as well. I will not compile it at the moment because first I want to create a library and then the command line tool and only then the GUI app.

We are now ready to create the video downloading library. Before we do it, let's test if our development environment works.
Let's write the hello world application, compile it and run it. Click, click...

#include <stdio.h>

int main(void) {
    (void)printf("hello world\n");
    return 0;

We are using vim with c-support plugin, so compiling the hello world from vim is done (by looking up in the c-support plugin's keyboard mapping cheat sheet) by typing \rc and the program can be run by typing \rr. Quick and neato.

video downloader programming environment screenshot - vim, gcc, hello world

This is end of part one. In part two I will create the library and the command line test program. Until next time!