Wireshark-users: [Wireshark-users] SSL decrypter pre master secret handling

From: Wayne Blacklock <wayneblacklock@xxxxxxxxx>
Date: Thu, 16 Aug 2012 14:20:32 -0400
Hello everyone,

I'm currently in the middle of debugging a painful SSL issue. With the same client server combination we sometimes suffer SSL handshaking errors and other times we don't. When things go wrong the server (Active Directory) closes the connection without sending a response. The only clue in the AD logs is an error 960 and 20. Which I believe corresponds to a bad_record_mac.

The cipher is TLS_RSA_WITH_AES_128_CBC_SHA

What I have noticed, is that when everything is OK, Wireshark can decrypt (using the servers private key) the SSL handshake no problem, note this line from the output:

ssl_decrypt_pre_master_secret:
RSA_private_decrypt
pcry_private_decrypt: stripping 207 bytes, decr_len 255
decrypted_unstrip_pre_master[255]:
pre master secret[48]:

When everything isn't ok, the SSL decryption fails:

ssl_decrypt_pre_master_secret:RSA_private_decrypt
pcry_private_decrypt: stripping 146 bytes, decr_len 255
decrypted_unstrip_pre_master[255]:
<snip>
ssl_decrypt_pre_master_secret wrong pre_master_secret length (109, expected 48)
dissect_ssl3_handshake can't decrypt pre master secret

Now I am no SSL expert, but I don't think this is right. What I hope to understand and the reason for this post, is how exactly the byte stripping works... where does the SSL decrypter  get 146 from and could this explain our SSL issues? If so, what exactly is going wrong? I've had a bit of a look through the TLS spec but it isn't clear to me and so I would appreciate any and all advice.

Thanks,

Wayne.