[Chris Chen <chchen@xxxxxxx>]

So here's my dilemma--

I've got Windows 7 clients that are having a hell of a time doing  
SSL/TLSv1 Handshakes with SMTP and IMAP servers that live behind a  
Linux Virtual Server.

All other clients work great. The Windows 7 clients report timeouts  
connecting to IMAPS and SMTPS. So, I ran a trace, and...

Here's a simple tshark dump:

  1   0.000000 WIN7_IP -> LINUX_IP TCP 49168 > urd [SYN] Seq=0  
Win=8192 Len=0 MSS=1380 WS=2 8192 52
  2   0.000010 LINUX_IP -> WIN7_IP TCP urd > 49168 [SYN, ACK] Seq=0  
Ack=1 Win=5840 Len=0 MSS=1460 WS=2 5840 52
  3   0.000680 WIN7_IP -> LINUX_IP TCP 49168 > urd [ACK] Seq=1 Ack=1  
Win=66240 Len=0 66240 40
  4   0.000990 WIN7_IP -> LINUX_IP SSL Client Hello 66240 209
  5   0.001005 LINUX_IP -> WIN7_IP TCP urd > 49168 [ACK] Seq=1  
Ack=170 Win=6912 Len=0 6912 40
  6   0.009298 LINUX_IP -> WIN7_IP TLSv1 Server Hello,  6912 1420
  7   0.009313 LINUX_IP -> WIN7_IP TLSv1 Certificate, Server Key  
Exchange, Server Hello Done 6912 1356
  8   0.010187 WIN7_IP -> LINUX_IP TCP 49168 > urd [ACK] Seq=170  
Ack=2697 Win=66240 Len=0 66240 40
  9   0.024244 WIN7_IP -> LINUX_IP TLSv1 Client Key Exchange, Change  
Cipher Spec, Encrypted Handshake Message 66240 174
 10   0.025339 LINUX_IP -> WIN7_IP TLSv1 Change Cipher Spec,  
Encrypted Handshake Message 7984 99
 11   0.225653 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change  
Cipher Spec, Encrypted Handshake Message 7984 99
 12   0.226324 WIN7_IP -> LINUX_IP TCP 49168 > urd [ACK] Seq=304  
Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401 66180 52
 13   0.627556 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change  
Cipher Spec, Encrypted Handshake Message 7984 99
 14   0.628474 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#1] 49168 > urd  
[ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401  
66180 52
 15   1.431361 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change  
Cipher Spec, Encrypted Handshake Message 7984 99
 16   1.432062 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#2] 49168 > urd  
[ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401  
66180 52
 17   3.039985 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change  
Cipher Spec, Encrypted Handshake Message 7984 99
 18   3.040671 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#3] 49168 > urd  
[ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401  
66180 52
 19   6.255200 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change  
Cipher Spec, Encrypted Handshake Message 7984 99
 20   6.255869 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#4] 49168 > urd  
[ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401  
66180 52
 21  12.686644 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change  
Cipher Spec, Encrypted Handshake Message 7984 99
 22  12.687330 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#5] 49168 > urd  
[ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401  
66180 52
 23  25.548536 LINUX_IP -> WIN7_IP TLSv1 [TCP Retransmission] Change  
Cipher Spec, Encrypted Handshake Message 7984 99
 24  25.549373 WIN7_IP -> LINUX_IP TCP [TCP Dup ACK 12#6] 49168 > urd  
[ACK] Seq=304 Ack=2756 Win=66180 Len=0 SLE=1597629342 SRE=1597629401  
66180 52
 25  29.280023 WIN7_IP -> LINUX_IP TLSv1 Encrypted Alert 66180 77
 26  29.280040 LINUX_IP -> WIN7_IP TLSv1 Application Data 7984 157
 27  29.280109 WIN7_IP -> LINUX_IP TCP 49168 > urd [FIN, ACK] Seq=341  
Ack=2756 Win=66180 Len=0 66180 40
 28  29.280147 LINUX_IP -> WIN7_IP TCP urd > 49168 [FIN, ACK]  
Seq=2873 Ack=342 Win=7984 Len=0 7984 40
 29  29.280582 WIN7_IP -> LINUX_IP TCP 49168 > urd [RST, ACK] Seq=342  
Ack=2873 Win=0 Len=0 0 40

So it looks like the handshake goes great up until line 9. The Linux  
host retransmits, then the ACK arrives, but the two hosts keep doing  
this dance until someone gives up (looks like the Windows 7 host, who  
sends the disconnect).

The crazy thing is I got openssl for windows installed and ran the  
s_client tool to see what was going on--it made it through the entire  
handshake, but
stalled. Here's the sample output from a successful run:

---
No client certificate CA names sent
---
SSL handshake has read 2756 bytes and written 247 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 1024 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID:  
21534131F1D5D5A7CCEE2D20A6CA23BA023D93B2F85F07ABF23C45666DDC63D6
    Session-ID-ctx:
    Master-Key:  
AEF4CF4E7F85DA24345902E54B7815E5235876EDE334614AA99A04C86BD4F361699A7C3BEB0338E80162EFB617F186CA
    Key-Arg   : None
    Krb5 Principal: None
    PSK identity: None
    PSK identity hint: None
    Start Time: 1275719887
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
220 mailhost.domain blah blah blah (This is the MTA's greeting line)

However, when the retransmit nonsense happens, there is not 220  
line--the output stops at ---.

The crazy thing is, when I press enter, an additional packet is sent,  
and the connection resumes, and the 220 line appears.

Has anyone seen this behavior before?

cc

--
Chris Chen <chchen@xxxxxxx>
UNIX Systems Administrator
Office of Information Technologies
Portland State University