Securing outbound SMTP communication of Vibe with GWIA.

  • 7010270
  • 07-Mar-2012
  • 27-Apr-2012


Novell GroupWise 2012
Novell GroupWise 8
Novell Vibe OnPrem 3


You want to make a secure SMTP connection for outbound mail communication of Vibe. Here we describe a scenario of how to secure Vibe on Linux with GWIA.


First make sure that your GWIA is configured with SSL part. Here are steps to secure GWIA by a certificates that are issued locally by Novell eDir CA:

1. Use the "Generate CSR Utility", GWCSRGEN.EXE in the GroupWiseSoftware Distribution Directory (\admin\utility\gwcsrgen).
In this example we will name the Keyfile yourserver.key and the CSR file, youserver.csr.Note the password down as needs to be specified in the SSL GWIA configuration.
Fill out all the required fields. For the hostname specify the DNS name that will be used for the public mail traffic. I.e. Do not use the internal DNS name or servername unless the same name is used to receive and send mails. If you are unsure about the hostname, simply use *
This process will create a KEY file and a CSR file.

2. You can use C1 to issue a certificate file for GWIA but since C1 was not further developed, it might not be compliant to use it with any later eDir version. Therefore it is better to use iManager too instead:

a) Open a web browser and type following URL to login into iManager:


Use admin or admin equivalent credentials.

b) Find Novell Certificate Server section and expand it. Click on Issue Certificate.

c) Browse to a location of the CSR file you created in step 1.

d) In a Key type section select SSL or TLS, do not select "Set the key usage extension to critical". Do not select "Enable extended key usage" section and also do not select "Set the key usage extension to critical".

e) Certificate type - select Certificate Authority, leave Path length to Unspecified. Do not select "Set basic constrains extension to critical".

f) Select a Validity period as needed.

g) Save to - use Base64 format ... and in next steps you can download the base64 certificate file.

3. Copy the yourserver.key from step 1 and base64 certificate file from step 2 into a root of GWIA directory.

4. In C1, check properties of GWIA, GroupWise | SSL Settings tab. Here navigate "Certificate file" to the base64 certificate file and "SSL key file" to yourserver.key. Do not forget to click on "Set Password" button and define a password specified in step 1.
5. Stay in C1 and configure next GWIA property - GroupWise | Network Address. Here specify SSL either Enabled or Required.

6. After you restart the GWIA and there are no complains about starting SSL, you can test SSL connection via a telnet session:
C: telnet <gwia_host> 25
S: 220 <> Ready
C: EHLO nico

Here STARTTLS is important.

Next you will need to secure your Vibe server. You can use the keytool Linux utility to generate new ".kyestore" file, new CSR and also for importing the DER file from the eDir CA. The keytool utility is located on a Linux server in /usr/java/<jdk_version>jre/bin. Please, refer to Vibe 3 online Administration documentation for detailed instructions how to generate ".keystore", CSR files. You can follow the IV Site Security section, 32.3.1 Preparing for Secure HTTP Connections. Here again, instead of C1, you can use iManager. For SelfSignedSever certificate:

a) go into Novell Certificate Server -> Configure Certificate Authority
b) click on Certificates tab
c) click on Self Signed Certificate - first validate it and then Export
d) do not select "Export private key" and make sure the export format s DER
e) click on "save the exported certificate" link to download it

Follow the rest of sections in the documentation:
Importing the Certificate Files into the Vibe Keystore
Replacing the Original Vibe Keystore File with Your Permanent Keystore File

Up to know we have secured GWIA and also Vibe. Next we are going to reconfigure Vibe to use secure SMTP connection for outbound communication:

1. Locate on the Vibe server installer-teaming.linux installation script and start it.

2. Accept a license agreement

3. Select Reconfigure settings.

4. Go thorough Advanced configuration option steps till you get on Outbound E-Mail Configuration screen. Here change protocol into "smtps" and finish through a rest of reconfiguration the steps.

5. Go into /opt/novell/teaming/apache-tomcat/conf/Catalina/localhost directory and open the ssf.xml file via nay text editor.

6. Find a section that starts with <Resource name="mail/SiteScape".
7. Add in this section new configuration parameter mail.smtp.starttls.enable="true".
8. Here find a parameter mail.transport.protocol="smtps".  This is a result of reconfiguration of the Vibe that we did earlier. Now comes important part. If you leave this mail.transport parameter set in smtps, you MUST set GWIA with SSL Required settings. If you change the transport.protocol into smtp value, then you MUST set GWIA with SSL Enabled. Using a combo Vibe`s"smtps" and GWIA`s Required configuration setting will make Vibe and GWIA using SSL protocol, using Vibe`s smtp and GWIA`s Enabled makes Vibe and GWIA using TLS protocol.

9. Save the XML file and restart Vibe.

10. Login into any Vibe account and send a mail to any existing Vibe account that has email configuration. Upon sending a mail, you shall see in GWIA log file lines similar to example bellow:

15:30:35 B3BC DMN: MSG 167161 Accepted connection: [::ffff:] ()
15:30:35 B3BC DMN: MSG 167161 SMTP upgraded to a secure connection.
15:30:35 B3BC Successful login with client/server access:
15:30:35 B3BC DMN: MSG 167161 Receiving file: /gw8/dom1/wpgate/gwia/receive/b97e05f4.000
15:30:35 B3BC DMN: MSG 167161 SMTP session ended: [::ffff:] ()
15:30:37 B3D4 MSG 167161 Processing inbound message: /gw8/dom1/wpgate/gwia/receive/b97e05f4.000
15:30:37 B3D4 MSG 167161 Sender:
15:30:37 B3D4 MSG 167161 Recipient:
15:30:37 B3D4 MSG 167161 Queuing to MTA
15:30:37 B3D4 MSG 167161 File: /gw8/dom1/wpgate/gwia/wpcsin/4/4f50e79d.3l1 Message Id: (4F50D98D.1A6:167:422) Size: 16.0 Kb
15:30:37 B45E MTP: File sent: 4f50e79d.3l1 Size: 16468

You will see a line  with "SMTP upgraded to a secure connection." only when Vibe is set with mail.transport.protocol="smtp" and GWIA with SSL setting on Enabled. This is probably the best combination you can chose as then the connection is via TLS which provides higher security than SSL.
The SSL combination: Vibe with mail.transport.protocol="smtps" and GWIA with SSL Required, will not show the line like "SMTP upgraded to a secure connection" but the connection is SSL-ized. However, using GWIA`s SSL setting on Required has one down site. Most of the SMTP hosts do not support such a configuration. If you then try to send email to such Internet host, it will disconnect and the GWIA will list "450 Host Down (" error. This would mean rather quite big limitation where you can send to / receive from ordinary Internet mails.