upload youtube perlWe have been downloading and converting YouTube videos but have not looked at how to upload them.

I will teach you how to upload YouTube videos via a command line. To find out how a video is being uploaded we will need just Firefox browser (
)!

Here is a typical scenario when you would want to automatically upload your videos to YouTube. Suppose you were using some other video sharing site and had already uploaded like 100 of your videos there. To get more popular on the net you’d also want to get your videos on YouTube, right? Doing it manually is a boring and tedious job, you want a program to do it for you while you sit back and relax.

Finding the YouTube Upload Form’s Elements

Log into your account and go to “My Account” menu in the upper right. Then press the “Upload New Video” button.

how to upload youtube videos

Now let’s use Firefox’ “Page Info” tool which is located under “Tools” menu.

firefox’s page info built in tool

When the “Page Info” tool pops up, select the “Forms” tab. The tool will list all the HTML forms on the page. The one named “theForm” with action’s url “http://www.youtube.com/my_videos_upload” is the upload form! When a user uploads the video, it gets submitted. Our program will submit this form for us.

youtube upload video page’s forms, fields, values

Now that we have found all the fields we need to submit, we can write the tool itself. I’ll use my favorite programming language - Perl, again. It is perfect for this job because it has extremely good packages for working with HTTP protocol.
I will probably write another post just about the Perl programming language - how to get it working on Windows Operating System. For now, if you are running Windows, then you can download ActiveState’s ActivePerl which is Perl’s port to win.

Writing the Program

As I said, we will be creating the tool in Perl programming language. In the previous post about YouTube I used the WWW::Mechanize package. I can tell you in advance that it will not work this time because there will be some unexpected surprises, such as hidden form elements being created by Javascript when the form is submitted. Since WWW::Mechanize stores all the forms internally, it will not be able to submit this form with these dynamic form elements. That’s why we will use the base class of WWW::Mechanize - the LWP::UserAgent.

LWP::UserAgent package can be installed with the following command:

perl -MCPAN -e 'install LWP::UserAgent'

Once you have it installed, we can start writing the program.

Step 1: Logging in to YouTube

Before we can upload a video, we need to login to YouTube. Let’s use the same procedure as before to find out what form do we need to submit to login. Let’s go to this url:

http://www.youtube.com/login

and look at all the forms on this page. One of them is named “loginForm” with four fields. We will need to submit this form and save the cookies. Luckily LWP::UserAgent can do it for us and we will not have to parse HTTP headers or anything like that.

YouTube’s login form fields, types and their values

We see that there is no action URL provided for the loginForm, that means that the form should be POSTed to the same URL it was located on, in our case:

http://www.youtube.com/login

We also see that the form has 4 elements which should be submitted:

  • ‘current_form’ with value ‘loginForm’
  • ‘username’ with our youtube login/username
  • ‘password’ with our password, and
  • ‘action_login’ with value ‘Log In’

How would we submit this form with LWP::UserAgent? Easy - it has a member function $ua->post( $url, \%form, $header_field_name => $value, … ), where %form is a hash of key, value pairs of form elements and $header_field_name => $value is a pair of HTTP headers we would like to add to the request.

Given an object of LWP::UserAgent named $ua, code which would login us to YouTube is the following:


$ua->post('http://www.youtube.com/login',
    {
        current_form => 'loginForm',
        username     => 'youtube_login',
        password     => 'youtube_password',
        action_login => 'Log In'
    },
);

See how easy it was? I love it.

The only problem is how to detect if we have really logged in. I noticed that when we log in, the upper right menu has a ‘Log Out’ button. So if we logged in succesfully, we should find this button.

Step 2: Uploading the Video

Now, once we have logged in, let’s upload the video.

Here is the image with all the fields we have to submit to upload a video, once more:

youtube upload video page’s forms, fields, values

There are 17 fields totally:

  • “field_myvideo_title” - title of the video
  • “field_myvideo_descr” - description of the video
  • “field_myvideo_keywords” - comma separated tags
  • “field_myvideo_categories” - video category
  • “language” - language of the video, i’ll leave it “EN” for English
  • “action_upload” - a string “Upload a video…”
  • “session_token” - we will have to extract the session token for this upload
  • “allow_embeddings” - should the video be allowed to be embedded on other sites, i’ll leave the default “Yes”
  • “allow_responses” - should we allow video responses, i’ll leave the default “Yes”
  • “allow_comments” - should we allow comments on the video, i’ll leave the default “Yes”
  • “allow_ratings” - should we allow users to rate the vide, i’ll leave the default “Yes”
  • “location” - latitude and longitude of the location video was shot, i’ll leave it empty
  • “field_date_mon” - month the video was shot, i’ll use default 0 (unknown)
  • “field_date_day” - day the video was shot, i’ll use 0 (unknown)
  • “field_date_yr” - year the video was shot, i’ll use 0 (unknown)
  • “field_privacy” - privacy of video (public or private), i’ll use the default - “public”
  • “ignore_broadcast_settings” - no idea what this is, i’ll use the default value 0

The trickiest part here is to extract the “session_token” which is some kind of unique id.
Going through the HTML source of upload page:

http://www.youtube.com/my_videos_upload

We find a javascript function called “dynamic_append_session_token” which dynamically creates a hidden form element containing the session ID:

finding the session id of youtube upload form

We have to extract this session ID from the line in red. It can be done with the following regular expression:


token_elem\.setAttribute\('value', '(.+?)'\);

Update 2008.03.12: This is no longer true! A session cookie is represented by a variable now:

var gXSRF_token = 'OTxXiSg8O-oQFB-PWTSaDv7oX2V8MTIwNTQ1NzYxNQ==';

When we have done this, we can submit the form. For this form we will need to add a “Content-Type: form-data” header to the request because the upload form has explicitly stated that its “enctype” is “multipart/form-data”.

As it turns out the actual video upload process is a two step process. This was the first step which set the video information. After we submit this form, we get redirected to a page which contains the upload file selection form. It might seem that that is the only element in this form but not so. Actually this 2nd step form has a few new fields and it also has all the fields from 1st step as hidden elements. Here are the modified and added fields in the 2nd step:

  • “contact” - no idea what this field is for
  • “field_command” - always “myvideo_submit”
  • “field_uploadfile” - our video file we want to upload
  • “field_private_share_entities” - no idea
  • “action_upload” - submit button’s caption “Upload Video”
  • “addresser” - another mysterious unique id

Here is what page info tool shows us about 2nd step’s form elements:

youtube upload video 2nd step

Notice that the form’s action URL changes, we have to extract it from the html source. Someone might suggest not to use regexes for parsing HTML but it is perfectly ok for this tiny project.

The trickiest thing here is telling LWP::UserAgent that ‘field_uploadfile’ is a form element of ‘file’ type. If we look in the LWP::UserAgent’s documentation we find that the three argument post() subroutine uses “POST() function from HTTP::Request::Common to build the request.” Further looking in the HTTP::Request::Common documentation we find that to specify a file to be uploaded the field has to be an array ref looking like this: [ $file, $filename, Header => Value… ].

That’s it. When we submit this form the video should have been uploaded. Now we just need to detect if the upload was successful. I noticed that when the video has uploaded successfully, YouTube thanks me for uploading. It looks like this:

youtube video upload complete

So checking if a video uploaded successfully is reduced to checking for the thank you message.

The program can be used from command line and takes a few mandatory arguments:

Usage: perl ytup.pl -l [login] -p [password] -f <video file> -c <category> -t <title> -d <description> -x <comma, separated, tags>

The login (-l) and password (-p) arguments are not mandatory because not to expose your password on a publically shared machine (by looking at process list) they can be specified in the source file by changing YT_LOGIN and YT_PASS constants.

Possible categories (for -c switch):
2    - Autos & Vehicles
23   - Comedy
24   - Entertainment
1    - Film & Animation
20   - Gadgets & Games
26   - Howto & DIY
10   - Music
25   - News & Politics
22   - People & Blogs
15   - Pets & Animals
17   - Sports
19   - Travel & Places

Here is the final code:


#!/usr/bin/perl
#
# Version 1.0: 2007.07.30
# Version 1.1: 2007.10.12: youtube changed html
# Version 1.2: 2008.03.13: youtube changed html
#
# Peteris Krumins (peter@catonmat.net)
# http://www.catonmat.net  -  good coders code, great reuse
#

use constant VERSION => "1.1";

use strict;
use warnings;
use LWP::UserAgent;
use Getopt::Std;

$Getopt::Std::STANDARD_HELP_VERSION = 1;

# set these values for default -l (login) and -p (pass) values
#
use constant YT_LOGIN => "";
use constant YT_PASS  => "";

# video categories
#
my %cats = (
    2   =>  'Autos & Vehicles',
    23  =>  'Comedy',
    24  =>  'Entertainment',
    1   =>  'Film & Animation',
    20  =>  'Gadgets & Games',
    26  =>  'Howto & DIY',
    10  =>  'Music',
    25  =>  'News & Politics',
    22  =>  'People & Blogs',
    15  =>  'Pets & Animals',
    17  =>  'Sports',
    19  =>  'Travel & Places'
);

# various urls
my $login_url  = 'http://www.youtube.com/login';
my $upload_url = 'http://www.youtube.com/my_videos_upload';

unless (@ARGV) {
    HELP_MESSAGE();
    exit 1;
}

my %opts;
getopts('l:p:f:c:t:d:x:', \%opts);

# if -l or -p are not given, try to use YT_LOGIN and YT_PASS constants
unless (defined $opts{l}) {
    unless (length YT_LOGIN) {
        preamble();
 print "Youtube username/login as neither defined nor passed as an argument\n";
        print "Use -l switch to specify the username\n";
        print "Example: -l joe_random\n";
        exit 1;
    }
    else {
        $opts{l} = YT_LOGIN;
    }
}

unless (defined $opts{p}) {
    unless (length YT_PASS) {
        preamble();
        print "Password was neither defined nor passed as an argument\n";
        print "Use -p switch to specify the password\n";
        print "Example: -p secretPass\n";
        exit 1;
    }
    else {
        $opts{p} = YT_PASS;
    }
}

unless (defined $opts{f} && length $opts{f}) {
    preamble();
    print "No video file was specified\n";
    print "Use -f switch to specify the video file\n";
    print 'Example: -f "C:\Program Files\movie.avi"', "\n";
    print 'Example: -f "/home/pkrumins/super.cool.video.wmv"', "\n";
    exit 1;
}

unless (-r $opts{f}) {
    preamble();
    print "Video file is not readable or does not exist\n";
    print "Check the permissions and the path to the file\n";
    exit 1;
}

unless (defined $opts{c} && length $opts{c}) {
    preamble();
    print "No video category was specified\n";
    print "Use -c switch to set the category of the video\n";
    print "Example: -c 20, would set category to Gadgets & Games\n\n";
    print_cats();
    exit 1;
}

unless (defined $cats{$opts{c}}) {
    preamble();
    print "Category '$opts{c}' does not exist\n\n";
    print_cats();
    exit 1;
}

unless (defined $opts{t} && length $opts{t}) {
    preamble();
    print "No video title was specified\n";
    print "Use -t switch to set the title of the video\n";
    print 'Example: -t "Super Cool Video Title"', "\n";
    exit 1;
}

unless (defined $opts{d} && length $opts{d}) {
    preamble();
    print "No video description was specified\n";
    print "Use -d switch to set the description of the video\n";
    print 'Example: -d "The coolest video description"', "\n";
    exit 1;
}

unless (defined $opts{x} && length $opts{x}) {
    preamble();
    print "No tags were specified\n";
    print "Use -x switch to set the tags\n";
    print 'Example: -x "foo, bar, baz, hacker, purl"', "\n";
    exit 1;
}

# tags should be at least two chars, can't be just numbers
my @tags = split /,\s+/, $opts{x};
my @filtered_tags = grep { length > 2 && !/^\d+$/ } @tags;
unless (@filtered_tags) {
    preamble();
    print "Tags must at least two chars in length and must not be numeric!\n";
    print "For example, 'foo', 'bar', 'yo29' are valid tags, but ";
    print "'22222', 'hi', 'b9' are invalid\n";
    exit 1;
} 

$opts{x} = join ', ', @filtered_tags;
# create the user agent, have it store cookies and
# pretend to be a cool windows firefox browser
my $ua = LWP::UserAgent->new(
    cookie_jar => {},
    agent => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.4) '.
             'Gecko/20070515 Firefox/2.0.0.4'
);

# let the user agent follow redirects after a POST request
push @{$ua->requests_redirectable}, 'POST';

print "Logging in to YouTube...\n";
login();

print "Uploading the video ($opts{t})...\n";
upload();

print "Done!\n";

sub login {
    # submit the login form
    my $res = $ua->post($login_url,
        {
            current_form => 'loginForm',
            username     => $opts{l},
            password     => $opts{p},
            action_login => 'Log In'
        }
    );
    unless ($res->is_success) {
        die "Failed logging in: failed submitting login form: ",
            $res->status_line;
    }

    # We have no good way to check if we really logged in.
    # I found that when you have logged in the upper right menu changes
    # and you have access to 'Log Out' option.
    # We check for this string to see if we have logged in.
    unless ($res->content =~ /Log Out/) {
        die "Failed logging in: username/password incorrect";
    }
}

sub upload {
    # upload is actually a two step process, first we set the video info,
    # and then we post the actual video file
    #

    my $resp = $ua->get($upload_url);
    unless ($resp->is_success) {
        die "Failed getting $upload_url: ", $resp->status_line;
    }

    my $session_token = extract_session_token($resp->content);
    unless (defined $session_token) {
        die "Failed extracting session token, YouTube might have redesigned!";
    }

    # let's prepare the video field hash which we will need in both steps
    my %vid_fields = (
        field_myvideo_title       => $opts{t},
        field_myvideo_descr       => $opts{d},
        field_myvideo_keywords    => $opts{x},
        field_myvideo_categories  => $opts{c},
        language                  => "EN",
        action_upload             => "Upload a video...",
        allow_embeddings          => "Yes",
        allow_responses           => "Yes",
        allow_comments            => "Yes",
        allow_ratings             => "Yes",
        location                  => "",
        field_date_mon            => 0,
        field_date_day            => 0,
        field_date_yr             => 0,
        field_privacy             => "public",
        ignore_broadcast_settings => 0,
        session_token             => $session_token
    );

    $resp = upload_step_one(\%vid_fields);

    # now add additional video fields, the new session token,
    # the addresser field (no idea what it is), and some more fields
    $vid_fields{session_token} = extract_session_token($resp->content);
    unless (defined $vid_fields{session_token}) {
        die "Failed extracting session token for upload step two\n",
            "YouTube might have redesigned :(";
    }

    if ($resp->content =~ m{name="addresser" value="(.+?)">}) {
        $vid_fields{addresser} = $1;
    }
    else {
        die "Failed extracting 'addresser' id for upload step two\n",
            "YouTube might have redesigned :(";
    }
    $vid_fields{contact}                      = "";
    $vid_fields{field_command}                = "myvideo_submit";
    $vid_fields{field_uploadfile}             = [ $opts{f} ];
    $vid_fields{field_private_share_entities} = "";
    $vid_fields{action_upload}                = "Upload Video";

    # the upload form's action URL at step 2 changes, we need to extract it
    my $action_url;
    if ($resp->content =~ m{enctype="multipart/form-data" action="(.+?)"}) {
        $action_url = $1;
    }
    else {
        die "Failed extracting action URL, YouTube might have redesigned!";
    }

    $resp = upload_step_two($action_url, \%vid_fields);

    # After the video has been uploaded, youtube thanks the user
    # for uploading the vid. Let's test for this thanks message
    # to see if the upload was successful
    unless ($resp->content =~ /Upload Complete/) {
        die "Upload might have failed, no 'thanks for upload' message ",
            "was found!.\n",
            "Or YouTube might have redesigned!";
    }
}

sub extract_session_token {
    my $content = shift;

    #if ($content =~ m{token = "(.+?)"}) {
    #    return $1;
    #}
    if ($content =~ m{_token = '(.+?)'}) {
        return $1;
    }

    return;
}

sub upload_step_one {
    my $vid_fields = shift;
    my $resp = $ua->post($upload_url, $vid_fields,
        "Content_Type" => "form-data");

    unless ($resp->is_success) {
        die "First upload step failed: ", $resp->status_line;
    }

    return $resp;
}

sub upload_step_two {
    my ($url, $vid_fields) = @_;
    my $resp = $ua->post($url, $vid_fields,
        "Content_Type" => "form-data");

    unless ($resp->is_success) {
        die "Second upload step failed: ", $resp->status_line;
    }

    return $resp;
}

sub HELP_MESSAGE {
    preamble();
    print "Usage: $0 ",
          "-l [login] ",
          "-p [password] ",
          "-f <video file> ",
          "-c <category> ",
          "-t <title> ",
          "-d <description> ",
          "-x <comma, separated, tags>\n\n";
    print_cats();
}

sub print_cats {
    print "Possible categories (for -c switch):\n";
    printf "%-4s - %s\n", $_, $cats{$_} foreach (sort {
        $cats{$a} cmp $cats{$b}
    } keys %cats);
}

sub VERSION_MESSAGE {
    preamble();
    print "Version: v", VERSION, "\n";
}

sub preamble {
    print "YouTube video uploader by Peteris Krumins (peter\@catonmat.net)\n";
    print "http://www.catonmat.net  -  good coders code, great reuse\n";
    print "\n"
}
Download

Download link: youtube video uploader (perl script)
Total downloads: 1205 times

After you download the script ytup.perl, rename it back to ytup.pl. I could not get the webserver to stop interpreting .pl extension as a CGI script, so I renamed it to ytup.perl.

Thanks for reading the tutorial! If you notice any inaccuracies, bugs or just have something to say, comment on this article!