I was implementing password authentication for VNC in node.js the other day and faced a problem where it would just never successfully authenticate. I checked my implementation several times and it seemed fine. Then I tried to implement it in Python just to see if I was missing something obvious. It also didn't work. Then I tried doing the authentication through openssl and it was also giving a result I didn't expect. Finally I found some old C code and wrote node-des, which worked. I still haven't figured out why it didn't work in node, python and openssl. Perhaps my readers can help me understand what was going on!

Here is a quick overview of how VNC password authentication works. On the VNC server you set the authentication password. When a VNC client connects, the VNC server generates random 16 byte garbage called challenge and sends it to the client. The client DES encrypts the challenge with the password as the key. The result is called the response and is also 16 bytes in size. The client then sends it to the VNC server. The VNC server then also DES encrypts challenge with the password and compares it with the response. If challenge matches response, the session gets authenticated.

Let's say the password set on the VNC server is browsers, and the challenge that the VNC server sends to the client is the following 16 bytes:

0c fd 6b 8c 5c 04 b3 e5 01 40 b9 de 33 9e 0d db

Then DES encrypting these 16 bytes with the password browsers should produce the following response:

44 ee fc 48 83 bb 95 f1 c0 82 c3 e2 98 c3 6a 4a

I sniffed this data from a real VNC client - VNC server communication and I use this as a test case for node-des as it's the correct result. However now let's look at what happens if I encrypt the challenge with node's crypto module, with Python and with openssl.

First node's crypto module. I wrote the following program to DES encrypt the 16 challenge challenge bytes with the password browsers.

var crypto = require('crypto');
var challenge = Buffer([
   0x0c, 0xfd, 0x6b, 0x8c,
   0x5c, 0x04, 0xb3, 0xe5,
   0x01, 0x40, 0xb9, 0xde,
   0x33, 0x9e, 0x0d, 0xdb
]);

var response = crypto
  .createCipher('des', 'browsers')
  .update(challenge);

console.log(Buffer(response));

When I run it, it produces 27 byte long output and it's not what I'd expect:

<Buffer c3 a9 72 c2 8b c3 a2 10 56 c2 8f c3 94 12 c2 a3 c3 a3 33 c2 ba c3 9e c2 a7 c2 96>

So node.js doesn't produce the expected 16 byte result. In fact I tried all the possible variations of the des algorithm that openssl (which node.js bases its crypto module on) supports. Perhaps I chose the wrong des implementation as there are many?

var crypto = require('crypto');
var challenge = Buffer([
   0x0c, 0xfd, 0x6b, 0x8c,
   0x5c, 0x04, 0xb3, 0xe5,
   0x01, 0x40, 0xb9, 0xde,
   0x33, 0x9e, 0x0d, 0xdb
]);

var algos = ("des des-cbc des-cfb des-ecb des-ede " +
            "des-ede-cbc des-ede-cfb des-ede-ofb " +
            "des-ede3 des-ede3-cbc des-ede3-cfb " +
            "des-ede3-ofb des-ofb des3 desx").split(/\s+/);

algos.forEach(function (algo) {
    console.log(algo);
    try { 
        var response = crypto
          .createCipher(algo, 'browsers')
          .update(challenge)

        console.log(Buffer(response));
    }
    catch (x) {
        console.log("Crypto doesn't support:", algo);
    }
});

It produced the following output and not a single result was what I'd expect:

des
<Buffer c3 a9 72 c2 8b c3 a2 10 56 c2 8f c3 94 12 c2 a3 c3 a3 33 c2 ba c3 9e c2 a7 c2 96>
des-cbc
<Buffer c3 a9 72 c2 8b c3 a2 10 56 c2 8f c3 94 12 c2 a3 c3 a3 33 c2 ba c3 9e c2 a7 c2 96>
des-cfb
<Buffer 24 43 c2 99 11 c2 8c c3 ac c3 b2 42 c3 b1 77 c3 a2 c2 96 05 c3 b3 c3 94 02>
des-ecb
<Buffer c2 aa 09 c3 98 c3 92 c2 8c 69 68 c2 97 c2 88 09 c2 ae c3 89 4f 3f c3 9f c3 83>
des-ede
<Buffer c2 84 c2 86 3f c3 ba 2b c2 ba 4b c2 bd 50 05 c3 a8 c3 9d 37 66 15 51>
des-ede-cbc
<Buffer 7d 60 c2 8d 4e 27 c3 b2 0f 6f 0e 48 54 60 38 2e c2 b3 7e>
des-ede-cfb
<Buffer 43 4d c2 ac 19 c3 92 c2 81 c2 a2 3f c2 9a c2 be c2 8a c2 b3 72 44 37 34>
des-ede-ofb
<Buffer 43 4d c2 ac 19 c3 92 c2 81 c2 a2 3f 49 c3 a4 04 24 2c 77 05 40>
des-ede3
<Buffer c2 a4 c2 84 c2 b2 2e 45 c2 90 c2 af c2 9c 21 5b c2 bd c3 aa c2 84 c3 9b 4c 76>
des-ede3-cbc
<Buffer c3 81 5c c2 b9 28 c2 be c3 b6 74 c3 89 c3 89 c2 8c 62 c2 ab 3a c2 95 6c 3f>
des-ede3-cfb
<Buffer 3f c3 8d 22 c3 8e 5d 01 29 c2 8b c2 85 c2 b5 11 75 6f c2 b5 5b c3 9c>
des-ede3-ofb
<Buffer 3f c3 8d 22 c3 8e 5d 01 29 c2 8b c2 8c 64 51 c2 8e c2 a7 2f c2 a8 c2 97>
des-ofb
<Buffer 24 43 c2 99 11 c2 8c c3 ac c3 b2 42 09 68 1b c3 bd 2d 79 c2 ba c3 a1>
des3
<Buffer c3 81 5c c2 b9 28 c2 be c3 b6 74 c3 89 c3 89 c2 8c 62 c2 ab 3a c2 95 6c 3f>
desx
<Buffer 4c 45 c2 bb c2 bd 78 c2 bd c3 b5 c2 b9 c3 96 c3 b7 c3 b0 6d c3 b9 c3 8d c3 a9 2d>

I had no idea what was going on at this point, so I decided to try Python and see if I can get the result I expect through Python's DES algorithm. Worst case I could spawn Python each time I need to authenticate at VNC. I used the pyDes module for my experiment and installed it to virtualenv as following:

$ virtualenv pydes
$ cd pydes
$ . ./bin/activate
$ easy_install pydes

Then I wrote the following test program:

from pyDes import des

challenge = "\x0c\xfd\x6b\x8c" \
            "\x5c\x04\xb3\xe5" \
            "\x01\x40\xb9\xde" \
            "\x33\x9e\x0d\xdb";

response = des("browsers").encrypt(challenge);

print ["0x%x" % ord(b) for b in response]

The output was 16 bytes but not what I'd expect:

['0xd0', '0x6f', '0x0', '0x1a', '0x73', '0xb8', '0x24', '0xc6', '0xf1', '0x72', '0x81', '0x6e', '0x6d', '0x59', '0xc8', '0xa7']

Then I thought, perhaps the endianness was incorrect, so I reversed the words in the challenge:

from pyDes import des

challenge = "\xfd\x0c\x8c\x6b" \
            "\x04\x5c\xe5\xb3" \
            "\x40\x01\xde\xb9" \
            "\x9e\x33\xdb\x0d";

response = des("browsers").encrypt(challenge);

print ["0x%x" % ord(b) for b in response]

But no, this was also giving some other result that wouldn't work with VNC:

['0x2e', '0xe8', '0xec', '0x7c', '0xde', '0xf7', '0x36', '0x56', '0xf0', '0x60', '0xbe', '0x6f', '0xb', '0xf8', '0x9b', '0x76']

Being totally clueless, I decided to use the plain command line openssl tool just to see if node.js or Python were messing up the binary data or something like that. As I had sniffed the challenge from the VNC session, I put it in a file challenge and ran the openssl tool as following:

$ openssl des -pass pass:browsers -in challenge -e -nosalt | hexdump
0000000 72e9 e28b 5610 d48f a312 33e3 deba 96a7
0000010 d253 826d 9b86 3620                    

The output was again different, 24 bytes in size and it didn't match the response.

At this point I had had enough with all these different results and I started searching the web for the simplest possible DES implementation as I wasn't up to implementing DES myself. I found some C code that was public domain and wrote node-des that produces the results that I expect and that VNC server accepts.

var des = require('../build/Release/des.node');
var challenge = Buffer([
   0x0c, 0xfd, 0x6b, 0x8c,
   0x5c, 0x04, 0xb3, 0xe5,
   0x01, 0x40, 0xb9, 0xde,
   0x33, 0x9e, 0x0d, 0xdb
]);

var response = des.encrypt('browsers', challenge);

console.log(response);

Output:

$ node des.js
<SlowBuffer 44 ee fc 48 83 bb 95 f1 c0 82 c3 e2 98 c3 6a 4a>

Victory! It worked and my VNC code was functional! The question is, why did every single attempt to DES encrypt with node, python and openssl fail? Any ideas? Let's get to the bottom of this in the comments!

Update

One of my readers noticed that many of the words in node's output were of the form cx xx, so he asked me if I was sure that node wasn't converting the output to utf8? I was pretty sure it wasn't, buffers are buffers of raw bytes after all, but this looked too suspicious. I looked up in the documentation and found out that the default encoding for the buffers was utf8...

So I modified the original program and replaced console.log(Buffer(response)) with console.log(Buffer(response, 'binary')).

var crypto = require('crypto');
var challenge = Buffer([
   0x0c, 0xfd, 0x6b, 0x8c,
   0x5c, 0x04, 0xb3, 0xe5,
   0x01, 0x40, 0xb9, 0xde,
   0x33, 0x9e, 0x0d, 0xdb
]);

var response = crypto
  .createCipher('des', 'browsers')
  .update(challenge);

console.log(Buffer(response, 'binary'));

Now I do get 16 bytes in the output but it still doesn't match the sniffed response:

<Buffer e9 72 8b e2 10 56 8f d4 12 a3 e3 33 ba de a7 96>

I also modified the program that runs through all the DES algorithms by adding the 'binary' encoding to the Buffer's constructor. The output now is 16 bytes for all algorithms:

des
<Buffer e9 72 8b e2 10 56 8f d4 12 a3 e3 33 ba de a7 96>
des-cbc
<Buffer e9 72 8b e2 10 56 8f d4 12 a3 e3 33 ba de a7 96>
des-cfb
<Buffer 24 43 99 11 8c ec f2 42 f1 77 e2 96 05 f3 d4 02>
des-ecb
<Buffer aa 09 d8 d2 8c 69 68 97 88 09 ae c9 4f 3f df c3>
des-ede
<Buffer 84 86 3f fa 2b ba 4b bd 50 05 e8 dd 37 66 15 51>
des-ede-cbc
<Buffer 7d 60 8d 4e 27 f2 0f 6f 0e 48 54 60 38 2e b3 7e>
des-ede-cfb
<Buffer 43 4d ac 19 d2 81 a2 3f 9a be 8a b3 72 44 37 34>
des-ede-ofb
<Buffer 43 4d ac 19 d2 81 a2 3f 49 e4 04 24 2c 77 05 40>
des-ede3
<Buffer a4 84 b2 2e 45 90 af 9c 21 5b bd ea 84 db 4c 76>
des-ede3-cbc
<Buffer c1 5c b9 28 be f6 74 c9 c9 8c 62 ab 3a 95 6c 3f>
des-ede3-cfb
<Buffer 3f cd 22 ce 5d 01 29 8b 85 b5 11 75 6f b5 5b dc>
des-ede3-ofb
<Buffer 3f cd 22 ce 5d 01 29 8b 8c 64 51 8e a7 2f a8 97>
des-ofb
<Buffer 24 43 99 11 8c ec f2 42 09 68 1b fd 2d 79 ba e1>
des3
<Buffer c1 5c b9 28 be f6 74 c9 c9 8c 62 ab 3a 95 6c 3f>
desx
<Buffer 4c 45 bb bd 78 bd f5 b9 d6 f7 f0 6d f9 cd e9 2d>

But still no match.

Solution

Ryan (in the comments below) found out why this was happening. Turns out that the VNC authentication reverses the order of bits in every byte of the password.

I was basing my implementation on the RFB protocol's specification and it doesn't say a word about it.

I'm quoting that document:

The server sends a random 16-byte challenge. The client encrypts the challenge with DES, using a password supplied by the user as the key, and sends the resulting 16-byte response.

There is no way I could have guessed this. How often do you think of reversing bits in every byte of the password? Never.

Comments

Paul Evans Permalink
April 19, 2012, 17:00

Since DES is a reversible crypto algorithm and not a one-way hash function, you could attempt to DES decrypt all the various outputs you've obtained here, and see what they reverse back into. Perhaps that may yield some hints?

Dan Luu Permalink
April 19, 2012, 17:09

Paul, that's unlikely to work. Flipping any single bit should result in an output that looks random and completely unrelated. That isn't always perfectly true in practice, which is how differential cryptanalysis is possible, but, for all practical purposes, DES is resistant to differential cryptanalysis

Paul Evans Permalink
April 19, 2012, 18:08

Hrm? This is nothing to do with cryptanalysis. We already have the ciphertext and the key. We simply want to look at what the cleartext comes out as. E.g. it could be that the input that was processed had a linefeed on the end, or some whitespace, or any of a number of simple implementation bugs at the encoding stage.

April 19, 2012, 18:17

See the comment below by Ryan. He links to an article that explains what is really going on!

April 19, 2012, 17:13

from the source code you're using in it, and comments from other people on the web it looks like VNC uses "almost DES" they're making DES drop the MSB instead of the LSB like normal since the password is normally ASCII (when it was created anyway) then it'll drop the 7th bit that's always a 0 instead of one that's part of the password. Not doing this would seem to allow you to shift one character to the right or left in the alphabet without consequence (depending on the LSB of course) which is not a good thing for passwords.

http://www.vidarholen.net/contents/junk/vnc.html has a little more about it

April 19, 2012, 18:01

That must be it! I'm using exactly the d3des.c implementation.

April 19, 2012, 20:31

Some crypto/digest APIs provide an additional function to complete the digest cycle. For example, Java's API uses both an update() method to add bytes to the digester and a digest() method to wrap up the computation, flush things, pad, etc.

http://docs.oracle.com/javase/7/docs/api/java/security/MessageDigest.html#digest()

It looks like the node.js crypto module has a similar method:

http://nodejs.org/docs/v0.3.1/api/crypto.html#cipher.final

Perhaps this is the issue.

April 19, 2012, 23:16

Good point. However that's not the case. Turns out node's Buffers have utf8 as default encoding. So all the data was messed up.

I just wrote an update about this. See the "Update" section in this post.

Michael Black Permalink
April 20, 2012, 10:56

All those who use DES should be aware of what you just found out. From the spec:
http://www.itl.nist.gov/fipspubs/fip46-2.htm
"Blocks are composed of bits numbered from left to right, i.e., the left most bit of a block is bit one."

So...implementers must process from LSB to MSB of each byte...otherwise you won't match other DES implementations (as you found out).

If they don't mention what order they expect the data bytes in then that's poor documentation on their part (as you found out again).

Darko Permalink
April 20, 2012, 18:24

Hello. I can't save yor page as mht or html download just stops can you check it??

Darko

April 20, 2012, 22:32

Works for me. Which browser are you using? I'm using Chrome 17.

April 20, 2012, 21:16

Coincidentally, I was confronted by the very same problem in my own implementation, vnc-client. Luckily your module showed up on npm the next day. Thanks!

April 20, 2012, 22:31

That's awesome. :)

August 27, 2013, 03:41

Well researched I love it and the writing was done perfectly.

August 27, 2013, 07:11

Can you please write more blog entries on this subject, its a request, because after reading your blog I am highly interested in learning more.

August 30, 2013, 01:31

Thanks for that great post and I appreciate the great blog you have.

August 31, 2013, 01:00

You've given me much to think about with this intelligent post.

September 03, 2013, 07:05

You said exactly what I was thinking in this post and I wanted to say thanks.

September 19, 2013, 01:20

Very cool entry and I so satisfied that I have found this useful information. I will definitely bookmark your site and visit this website again in the future. Thanks a lot for posting this valuable info for us. King regards!

September 19, 2013, 02:55

Most of us are thankful to be working, and accept whatever the company gives us.

September 19, 2013, 02:59

I am interested in the article. Thank so much. I hope to see more useful news from you.

September 20, 2013, 01:49

I admire the work that you do there. Must be a challenge to do a VIP blog each week. But very appreciated,

September 21, 2013, 01:03

I positively enjoying each little bit of it and I have you bookmarked to check out new stuff you web log post

September 21, 2013, 01:45

I am interested in the article. Thank so much. I hope to see more useful news from you.

September 24, 2013, 02:50

Very nice post. I simply stumbled upon your weblog and wished to mention that I have truly enjoyed surfing around your blog posts. In any case I’ll be subscribing to your rss feed and I am hoping you write again very soon!

September 26, 2013, 02:50

Excellent site, keep up the good work my colleagues would love this.

September 26, 2013, 03:32

Thanks for sharing excellent informations. Your website is very cool.

October 07, 2013, 09:22

This kind of information is rare in the internet. Very helpful post. I have been looking for it about a week

October 10, 2013, 01:45

Thanks a lot and keep sharing the information. Keep updating the information for all of us.

October 28, 2013, 03:07

I want to extend my greatest appreciation with the article you post me really love to read it

October 30, 2013, 02:45

I think I will become a great follower.Just want to say your article is striking. The clarity in your post is simply striking and i can take for granted you are an expert on this subject.

November 02, 2013, 02:31

I am glad for commenting to let you understand what a awesome discovery our girl gained checking yuor web blog. She discovered such a lot of issues, including what it's like to possess an awesome helping mindset to have a number of people effortlessly gain knowledge of selected problematic topics.

November 11, 2013, 00:21

Very cool entry and I so satisfied that I have found this useful information. I will definitely bookmark your site and visit this website again in the future. Thanks a lot for posting this valuable info for us. King regards!

November 25, 2013, 01:08

I visited your site & after visiting i found that it is very informational for everyone you have done really a great job thank you

November 28, 2013, 03:28

I think I will become a great follower.Just want to say your article is striking. The clarity in your post is simply striking and i can take for granted you are an expert on this subject.

December 19, 2013, 08:41

Thanks so much with this fantastic new web site. I’m very fired up to show it to anyone. It makes me so satisfied your vast understanding and wisdom have a new channel for trying into the world.

December 24, 2013, 07:20

Thank for your writing. I think this give more information which is very popular nowadays.

January 04, 2014, 07:44

Bloggers usually bring fresh and new ideas down to the internet and I believe that you have done this perfectly without any hesitation.

Very excellent content. Alot of great ideals for me to start immediately. Its hearing blogs like this that inspires me. Thanks!

March 03, 2014, 03:10

I will be looking for futher updates. I have bookma rked your page and look fwd to more post

April 04, 2014, 16:01

thank i live in viet nam

April 04, 2014, 16:02

i live in viet nam thanks

April 22, 2014, 01:40

cach lam tinh bang ty suong nhat, cach lam chuyen ay

August 16, 2014, 05:48

InternetJoeProHome | InternetJoePro

August 16, 2014, 13:02

Được thành lập ngày 31/01/1997, Công ty Cổ phần Viễn thông FPT (FPT Telecom) khởi đầu với tên gọi Trung tâm Dịch vụ Trực tuyến

August 28, 2014, 03:08

The web improvement generally requires the producing a web site it includes content improvement, internet design, scripting of client and server side scripting e-commerce among other people. All these are equally important for the success of the website and the company’s good results...

September 13, 2014, 02:36

Cảm ơn bạn đã dành thời gian để đăng thông tin có giá trị như vậy. Tôi có một vài suy nghĩ cảm hứng tốt đẹp sau khi đọc nó. Tôi thích nó rất nhiều.

October 15, 2014, 02:42

Interesting point of view and I agree with most of what you say.You've made some valid points and I think you're spot on.

Diana Permalink
December 17, 2014, 16:43

This article is quite helpful and informative too. I enjoyed a lot. Thanks for sharing such a great article.

Beautiful Christmas Quotes for your friends and family...
christmas quotes
Christmas Messages for Whatsapp

Best Christmas Greetings for your friends and family...
christmas greeting
christmas wishes

What to Write in Christmas Card....... Check out best Christmas Greeting Card Words
What to Write in Christmas Card

Get Beautiful and Unique Christmas Wallpapers for free
free christmas wallpaper
christmas tree decorating ideas

Thanks for sharing such a great article.

Leave a new comment

(why do I need your e-mail?)

(Your twitter name, if you have one. (I'm @pkrumins, btw.))

Type the word "unix_315": (just to make sure you're a human)

Please preview the comment before submitting to make sure it's OK.

Advertisements