iChain does not work with SSL certificates containing UTF-8 encoding

  • 3369938
  • 04-Oct-2006
  • 16-Mar-2012

Environment

Novell iChain 2.3
Secure Exchange
3rd party SSL Certificate
SSL Certicificate issued by Thawte CA

Situation

iChain abends when trying to apply a 3rd party SSL certificate to an accelerator service.
Unable to connect to the Secure Exchange port of an accelerator using 3rd party SSL certificate.
Abend: EIP in NILE.NLM at code start +0000C844h
How to decode and check a certificate for UTF-8 encoding
How to decode and check a certificate for UTF8 encoding

Resolution

It is currently known that SSL Certificates issued by Thawte after March 2005 use UTF8STRING for the commonName field. As such, these certificates do not currently work with iChain. With iChain builds prior to 2.3.282 the system will abend when attempting to apply the certificate to an accelerator. In builds 2.3.282 and later the box will not abend, but the certificate will still not be used. The SSL handshake will fail when a user attempts to browse to the site.

Update: It has been reported Thawte no longer uses UTF-8 encoding for all SSL certificates. Specifically, wildcard certificates were able to be obtained without UTF-8 encoding as of 1 Oct. 2006. Novell Technical Support has not verified this with Thawte, and neither recommends or endorses the use of any particular CA for 3rd party certificates. This update is for informational purposes only.

At this time the only option available is to obtain a different SSL certificate from a vendor which does not use UTF8 encoding for any of the fields in the certificate. Novell is working to resolve this issue, but no time frame for resolution is currently available.

Additional Information

commonName field of the SSL certificate uses UTF-8 encoding. (UTF8STRING)
SubjectName or IssuerName field of the SSL certificate uses UTF-8 encoding. (UTF8STRING)
iChain 2.3 does not properly handle certificates with fields in UTF-8 format.

Using various tools it is possible to confirm whether or not a certificate contains UTF-8 encoded fields. The following example shows how this can be done with OpenSSL. (OpenSSL is available for a variety of Operating Systems including Linux and Windows. For additional information please see the OpenSSL website athttp://www.openssl.org/)

Export the certificate in DER format:

  • If the certificate is in PFX (PKCS12) format it will first need to be converted into PEM (Base64) format:
    • openssl pkcs12 -in certificate.pfx -out certificate.pem -nokeys -clcerts
  • Once the certificate is in PEM format it can be converted to DER format:
    • openssl x509 -incertificate.pem -out certificate.der -outform DER

With the certificate in DER format it is possible to view the ASN.1 information. The following command will display the information in human readable format:

openssl asn1parse -inform DER -in certificate.der

There will be many lines of output detailing each of the fields of the certificate. Simply look for "UTF8STRING" to see if that encoding type has been used for fields within the certificate. Recent Thawte certificates are known to use this encoding for the commonName field, producing something similar to the following in the output:

402:d=5 hl=2 l= 3 prim: OBJECT :commonName
407:d=5 hl=2 l= 16 prim: UTF8STRING

The above process can also be accomplished using Internet Explorer and the Windows certificate handling routines.

  1. Open the server certificate by double clicking on the file.
  2. Select the Details tab, then click on the "Copy to File..." button.
  3. Click "Next" to begin the Certificate Export Wizard.
  4. Verify that "DER encoded binary X.509 (.CER)" is selected, then click the "Next" button.
  5. Enter a filename for this exported certificate (such as c:\certificate.cer), then click the "Next" button.
  6. Click "Finish" to export the file, then the "OK" button to exit the Wizard.

After the file is in DER format it is possible to view the certificate details using the GUIdumpASN utility. (http://www.geminisecurity.com/guidumpasn.html)

  1. Install, then launch the utility.
  2. Select "File" then "Open" (Ctrl-O) from the program menu.
  3. Browse to, then select the exported certificate from the steps above, then click the "Open" button.
  4. The certificate details are displayed.

Output similar to the following will be displayed in the utility:

402 06 3: OBJECT IDENTIFIER commonName (2 5 4 3)
407 0C 16: UTF8String '*.example.com'


Excerpt from a Debug Log when iChain abends on this condition:
Server ICS_SERVER halted Friday, April 22, 2005 12:08:24.601 pm
Abend 1 on P00: Server-5.60.03: Page Fault Processor Exception (Error code 00000000)

Registers:
CS = 0008 DS = 0010 ES = 0010 FS = 0010 GS = 0010 SS = 0010
EAX = 00000000 EBX = 00000000 ECX = CAAF683C EDX = 00000040
ESI = 00000000 EDI = 0000FC00 EBP = C80B8324 ESP = CAAF68E8
EIP = C84BC844 FLAGS = 00010246
C84BC844 8B4008 MOV EAX,[EAX+08]=?
EIP in NILE.NLM at code start +0000C844h
Access Location: 0x00000008

The violation occurred while processing the following instruction:
C84BC844 8B4008 MOV EAX,[EAX+08]
C84BC847 50 PUSH EAX
C84BC848 52 PUSH EDX
C84BC849 8B4C2454 MOV ECX,[ESP+54]
C84BC84D 55 PUSH EBP
C84BC84E 8901 MOV [ECX],EAX
C84BC850 E8EBB86A0F CALL NWUTIL.NLM|NWUtilMemCopy
C84BC855 83C40C ADD ESP,0000000C
C84BC858 8B5C2420 MOV EBX,[ESP+20]
C84BC85C 53 PUSH EBX

Running process: Server 12 Process
Thread Owned by NLM: SERVER.NLM
Stack pointer: CAAF636C
OS Stack limit: CAAEF040
Scheduling priority: 67371008
Wait state: 3030070 Yielded CPU
Stack: --00000000 ?
--CAAF68FC ?
--00000000 ?
--00000000 ?
--00000000 ?
-C850F1F0 (NILE.NLM|keyFileAttrName+0)
--00000000 ?
--00000001 ?
--68060006 ?
--12842004 ?
--68060002 ?
--CAAF695C ?
--C80B8324 ?
C84B89F6 (NILE.NLM|cmsGetKeyFile+26)
--68060002 ?
--CAAF695C ?
-C850F1F0 (NILE.NLM|keyFileAttrName+0)
--C80B8324 ?
--0000FC00 ?
--CAAF6B7C ?
--68060002 ?
--68060002 ?
--C80B8324 ?
C84CEBA3 (NILE.NLM|SASSDK_crypt_init+2D3)
--68060002 ?
--CAAF695C ?
--C80B8324 ?
--0000FC00 ?
--CAAF6B7C ?
--0049005C ?
--00530043 ?
--0054005F ?
--00450052 ?
--005C0045 ?
00630069 (SERVER.NLM|EvaluateAgingPageType+5D)
--005C0073 ?
00690077 (SERVER.NLM|AddNLMToAddressSpace+7)
0064006C (SERVER.NLM|UpdateVMInfoCol12+1B78)
--002D005F ?
--0049005F ?
--00530043 ?
--0053005F ?
--00520045 ?
--00450056 ?
--00000052 ?
--00000000 ?
--00000008 ?
--0000000D ?
--0000000D ?
--00000001 ?
--00000001 ?
--000000A8 ?
--D3617054 ?
--00000005 ?
--20000203 ?
--00000003 ?
--00000003 ?
--00000008 ?
--0000000A ?
--0000000A ?
--00000002 ?
--00000001 ?
--00000400 ?
--D361706C ?
--00000000 ?
--00000800 ?
--00000800 ?
--00000000 ?
--00000001 ?
--0000000D ?
--00000000 ?
--00000000 ?
--00000000 ?
--00000000 ?
--D36170BC ?
--00000001 ?
--200002C3 ?
--00000003 ?
--00000000 ?
--00000008 ?
--0000000D ?
--0000000D ?
--00000001 ?
--00000001 ?
--00000038 ?
--D36170C8 ?
--00000002 ?
--200002C3 ?
--00000003 ?
--00000000 ?
--00000008 ?
--0000000D ?
--0000000D ?
--00000001 ?
--00000001 ?
--000000A8 ?
--D3617054 ?
006E121F (SERVER.NLM|GetCurrentClock+193)
006E122D (SERVER.NLM|GetCurrentClock+1A1)
--CAF2D100 ?

.

Formerly known as TID# 10098516