Windows XP/2003 and SSL in today's reality (in example with MDaemon)

Time goes by, and Windows XP and Windows Server 2003 is not getting any younger. At the same time, security standards are developing independently of these legacy operating systems. Using them in your infrastructure, you not only put your entire it system under attack, but sometimes just lose functionality.

Here is example when MDaemon cannot establish SSL connection with other mail servers or clients (both while sending and receiving mail)

If you are using MDaemon with Windows Server 2003, you may experience problems with delivering or receiving mail from other mail servers.

Here are some logs, which can indicate the problem:

Wed 2015-09-16 10:30:43: ----------
Wed 2015-09-16 10:30:48: [9741:1] Session 9741; child 1
Wed 2015-09-16 10:30:48: [9741:1] Accepting SMTP connection from [xxx.xxx.xxx.xxx:14564] to [xx.xx.xx.xx:25]
Wed 2015-09-16 10:30:48: [9741:1] --> 220 mail.receiver-company.com ESMTP MDaemon 12.5.3; Wed, 16 Sep 2015 10:30:48 +0300
Wed 2015-09-16 10:30:48: [9741:1] <-- EHLO mail.sender-company.com
Wed 2015-09-16 10:30:48: [9741:1] --> 250-mail.receiver-company.com Hello mail.sender-company.com, pleased to meet you
Wed 2015-09-16 10:30:48: [9741:1] --> 250-VRFY
Wed 2015-09-16 10:30:48: [9741:1] --> 250-EXPN
Wed 2015-09-16 10:30:48: [9741:1] --> 250-ETRN
Wed 2015-09-16 10:30:48: [9741:1] --> 250-AUTH LOGIN CRAM-MD5 PLAIN
Wed 2015-09-16 10:30:48: [9741:1] --> 250-8BITMIME
Wed 2015-09-16 10:30:48: [9741:1] --> 250-STARTTLS
Wed 2015-09-16 10:30:48: [9741:1] --> 250 SIZE
Wed 2015-09-16 10:30:48: [9741:1] <-- STARTTLS
Wed 2015-09-16 10:30:48: [9741:1] --> 220 Begin TLS negotiation
Wed 2015-09-16 10:30:48: [9741:1] * SSL error 0x80090308 The token supplied to the function is invalid
Wed 2015-09-16 10:30:48: [9741:1] SMTP session terminated (Bytes in/out: 555/283)
Wed 2015-09-16 10:30:48: ----------

Thu 2015-09-17 08:57:21: ----------
Thu 2015-09-17 08:57:43: [3120:2] Session 3120; child 2
Thu 2015-09-17 08:57:43: [3120:2] Accepting SMTP connection from [xxx.xxx.xxx.xxx:52568] to [xx.xx.xx.xx:465]
Thu 2015-09-17 08:57:43: [3120:2] * SSL error 0 The operation completed successfully.
Thu 2015-09-17 08:57:43: [3120:2] SMTP session terminated (Bytes in/out: 221/2534)
Thu 2015-09-17 08:57:43: ----------

Also you can encounter errors such "SSL negotiation failed, error code 0x80090326".

Solution

The solution is to install the updates into the system, which adds support for AES Cipher in Windows:

• TLS_RSA_WITH_AES_128_CBC_SHA AES128-SHA
• TLS_RSA_WITH_AES_256_CBC_SHA AES256-SHA

Unfortunately, there is update for Windows Server 2003 only, and there is no update for Windows XP. You will have to accept this, as support for XP has already been completed, and there will be no updates anymore.

In some cases, Windows XP Service Pack 3 can help you or (in the case of 64-bit systems) KB968730.