Error Attempting a TLS 1.2 Connection to an IBM z/OS Telnet Server

  • Document ID:7022064
  • Creation Date:16-Jul-2014
  • Modified Date:02-Mar-2018
    • Micro Focus Products:
      Extra!
      Reflection (legacy 14.1)
      Reflection Desktop
      Reflection for the Web

Environment

Reflection 2014
Reflection Pro 2014
Reflection for IBM 2014
Reflection for IBM version 14.x
EXTRA! X-treme version 9.3
Reflection for the Web 2014 (All Editions)

Situation

When attempting a SSL 3.0, TLS 1.0, or TLS 1.2 connection to an IBM z/OS telnet server, the connection may fail to be established.

The error that displays depends on the product:

Reflection 2014 error:

Host: <servername>
An error occurred in communications.
Reflection SSL/TLS could not establish an encrypted connection.

Reflection 14.x error:

An error occurred in communications -Reflection SSL/TLS could not establish an encrypted connection.

EXTRA! X-treme 9.3 error:

Messages display in the Status Application Audit Log:

TLSStartSecurity returned error 9702 <SSL/TLS handshake failed.>
Secure connection was requested but not granted by server <servername> on port <serverport>
Socket failed to connect.
Last socket error = 0. No error

Reflection for the Web 2014 error:

TLS alert Unexpected Message encountered

Or:

Connection to Host Failed

Resolution

We recommend installing and configuring Reflection 2014 R1 SP1 or higher to resolve this issue, though you can instead follow the Alternative Solution using z/OS Telnet Server below.

Disable "Implicit" Option in Reflection 2014

Reflection 2014, beginning in R1 SP1, supports the Host sending the DO STARTTLS command and will respond properly if the “Implicit SSL/TLS Connection†box in the Security Properties dialog of the Reflection 3270 session is unchecked. For more information about the R1 SP1 release, see KB 7021432.

Alternative Solution using z/OS Telnet Server

Note: To make a SSL 3.0, TLS 1.0 or TLS 1.2 connection using the Reflection or EXTRA! products, you must modify the IBM Telnet Server configuration. Below are some suggested settings for the IBM Telnet Server; however, we recommend that you contact IBM to configure your Telnet Server correctly for your environment.

Change the CONNTYPE to SECURE on the IBM z/OS Telnet Server. This will configure the server to wait for a Client Hello from the client.

For detailed information about the CONNTYPE parameters, see http://pic.dhe.ibm.com/infocenter/zos/v1r13/index.jsp?topic=%2Fcom.ibm.zos.r13.halz001%2Fcontst.htm.

Cause

The connection to an IBM z/OS telnet server fails with an error because the telnet server is attempting to start the connection with the DO STARTTLS command. 

For an IBM z/OS telnet server, the TCP port definition is probably configured for CONNTYPE NEGTSECURE. With the CONNTYPE set to NEGTSECURE, the telnet server sends a DO STARTTLS command. If the telnet server receives anything other than what it expects, it will drop the TCP/IP connection to the client PC.

When SSL/TLS is enabled in the Attachmate 3270 emulation software, it is in secure mode and initiates the SSL or TLS connection by sending a Client Hello message. The telnet server sends DO STARTTLS at the same time the client software sends the Client Hello. Since this behavior is not what the IBM z/VM telnet server expects, it closes the TCP/IP connection to the client PC with a TCP FIN command.

Additional Information

At the time of this technical note's original publication, Attachmate 3270 emulation software did not support the STARTTLS option from the Host. Although the IBM z/VM Telnet Server 6.4 and later and the IBM Telnet Server in z/OS V1R2 support this command, the RFC at http://tools.ietf.org/html/draft-altman-telnet-starttls-02 where the STARTTLS option was proposed has expired without approval of the IETF, thus the command is not an industry accepted standard.

Legacy KB ID

This document was originally published as Attachmate Technical Note 2723.