Upgrade error: SSLCertificateFile: file '/opt/novell/apache2/certs/myProxyCert.pem' does not exist or is empty

Access Manager 3.2 running successfully where users could access protected resources via the Access Gateway Appliance (AGA) after having authenticated at the Identity (IDP) server. Whilst upgrading to 3.2 Support Pack 1, administrator aborted the install at the final stage. When re-running the upgrade script again on the AGA, the Apache server failed to start correctly and no user request from browsers would get responded to.

When running 'netstat -patune|grep httpd', no listener was visible on the TCP 443 ports configured. The Access Manager upgrade logs showed an error updating the novell-apache2 rpm set ( SSLCertificateFile: file '/opt/novell/apache2/certs/myProxyCert.pem' does not exist or is empty), and the rpm logs showed the following specific error:

D: fini      040755  2 (   0,   0)         0 /etc/opt/novell/apache2/conf/vhosts.d
D: fini      040755  4 (   0,   0)         0 /opt/novell
D: fini      040755  7 (   0,   0)         0 /opt/novell/apache2
D: fini      040755  2 (   0,   0)         0 /opt/novell/apache2/bin
D: fini      100755  1 (   0,   0)      6920 /opt/novell/apache2/bin/novell-apache2;509ad49a
D: fini      100755  1 (   0,   0)      2220 /opt/novell/apache2/bin/novell-apache2.redhat;509ad49a
D: fini      120777  1 (   0,   0)        36 /opt/novell/apache2/cacerts;509ad49a
D: fini      120777  1 (   0,   0)        34 /opt/novell/apache2/certs;509ad49a
LZDIO:      23 reads,   188416 total bytes in 0.007687 secs
###############################################
error: unpacking of archive failed on file /opt/novell/apache2/certs: cpio: rename failed - Inappropriate ioctl for device
D: ========== +++ novell-apache-gateway-session-cache-3.2.1-12 x86_64-linux 0x0

Temporarely rename the /opt/novell/apache2/certs directory, and manually copy the proxy cert myProxyCert.pem referenced in the error into the /etc/opt/novell/apache2/certs/ directory after the novell-apache2 service has been restarted (as shown below)

The 3rd party certificate assigned to the service is added to a DIRECTORY called certs. When the SP1 upgrade is run, it seems to tries to rename it, and covert it to a symbolic link to /etc/opt/novell/apache2/conf/certs. This process appeared to fail here as noticed in the verbose output of the rpm install:

error: unpacking of archive failed on file /opt/novell/apache2/certs: cpio: rename failed - Inappropriate ioctl for device

After renaming the /opt/novell/apache2/certs directory, it was able to create the symlink to the correct location. We then copied the pem formated server cert into the new location as specified by the symlink and apache restarted successfully.

irzave09:/etc/ld.so.conf.d # cd /opt/novell/apache2/certs
irzave09:/opt/novell/apache2/certs # ls
myProxyCert.pem

irzave09:/opt/novell/apache2/certs # cd ..
irzave09:/opt/novell/apache2 # clear
irzave09:/opt/novell/apache2 # ls
bin      certs           certs;509ad0aa  certs;509ad307  certs;509ad39f  certs;509ad462  clientcerts  sbin cacerts  certs;5099c49e  certs;509ad0bd  certs;509ad339  certs;509ad43a  certs;509ad49a  libexec      share 

irzave09:/opt/novell/apache2 # ll
total 20
drwxr-xr-x 2 root root 4096 Nov  8 08:07 bin
lrwxrwxrwx 1 root root   36 Nov  8 08:07 cacerts -> /etc/opt/novell/apache2/conf/cacerts
drwxr-xr-x 2 root root 4096 Aug 14 08:53 certs
lrwxrwxrwx 1 root root   34 Nov  7 12:47 certs;5099c49e -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 07:50 certs;509ad0aa -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 07:51 certs;509ad0bd -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:00 certs;509ad307 -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:01 certs;509ad339 -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:03 certs;509ad39f -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:05 certs;509ad43a -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:06 certs;509ad462 -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:07 certs;509ad49a -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   40 Aug  6 13:39 clientcerts -> /etc/opt/novell/apache2/conf/clientcerts
drwxr-xr-x 2 root root 4096 Aug  6 13:39 libexec
drwxr-xr-x 2 root root 4096 Aug  6 13:39 sbin
drwxr-xr-x 4 root root 4096 Aug  6 13:39 share 

irzave09:/opt/novell/apache2 # mv certs certs.orig

irzave09:/opt/novell/apache2 # ll
total 20
drwxr-xr-x 2 root root 4096 Nov  8 08:07 bin
lrwxrwxrwx 1 root root   36 Nov  8 08:07 cacerts -> /etc/opt/novell/apache2/conf/cacerts
drwxr-xr-x 2 root root 4096 Aug 14 08:53 certs.orig
lrwxrwxrwx 1 root root   34 Nov  7 12:47 certs;5099c49e -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 07:50 certs;509ad0aa -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 07:51 certs;509ad0bd -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:00 certs;509ad307 -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:01 certs;509ad339 -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:03 certs;509ad39f -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:05 certs;509ad43a -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:06 certs;509ad462 -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   34 Nov  8 08:07 certs;509ad49a -> /etc/opt/novell/apache2/conf/certs
lrwxrwxrwx 1 root root   40 Aug  6 13:39 clientcerts -> /etc/opt/novell/apache2/conf/clientcerts
drwxr-xr-x 2 root root 4096 Aug  6 13:39 libexec
drwxr-xr-x 2 root root 4096 Aug  6 13:39 sbin
drwxr-xr-x 4 root root 4096 Aug  6 13:39 share

irzave09:/opt/novell/apache2 # rcnovell-apache2 start
Starting Novell Gateway Service...                                                                                                              failed

see /var/log/novell-apache2/rcnovell-apache2.out for details

irzave09:/opt/novell/apache2 # cd /root/NAM-magappliance/
irzave09:~/NAM-magappliance # rpm -Uhvvv --force /root/NAM-magappliance/mag/rpms/*apache*.rpm  >> rpminstaoll.txt 2>&1
irzave09:~/NAM-magappliance # rcnovell-apache2 start

Starting Novell Gateway Service...                                                                                                              failed

 see /var/log/novell-apache2/rcnovell-apache2.out for details

irzave09:~/NAM-magappliance # more  /var/log/novell-apache2/rcnovell-apache2.out
Syntax error on line 29 of /etc/opt/novell/apache2/conf/vhosts.d/neil.conf:
SSLCertificateFile: file '/opt/novell/apache2/certs/myProxyCert.pem' does not exist or is empty

irzave09:~/NAM-magappliance # cd /opt/novell/apache2/

irzave09:/opt/novell/apache2 # ls -alsh
total 28K
4.0K drwxr-xr-x  7 root root 4.0K Nov  8 08:25 .
4.0K drwxr-xr-x 16 root root 4.0K Oct 16 02:00 ..
4.0K drwxr-xr-x  2 root root 4.0K Nov  8 08:25 bin
   0 lrwxrwxrwx  1 root root   36 Nov  8 08:25 cacerts -> /etc/opt/novell/apache2/conf/cacerts
   0 lrwxrwxrwx  1 root root   34 Nov  8 08:25 certs -> /etc/opt/novell/apache2/conf/certs
4.0K drwxr-xr-x  2 root root 4.0K Aug 14 08:53 certs.orig  
   0 lrwxrwxrwx  1 root root   34 Nov  7 12:47 certs;5099c49e -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   34 Nov  8 07:50 certs;509ad0aa -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   34 Nov  8 07:51 certs;509ad0bd -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   34 Nov  8 08:00 certs;509ad307 -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   34 Nov  8 08:01 certs;509ad339 -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   34 Nov  8 08:03 certs;509ad39f -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   34 Nov  8 08:05 certs;509ad43a -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   34 Nov  8 08:06 certs;509ad462 -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   34 Nov  8 08:07 certs;509ad49a -> /etc/opt/novell/apache2/conf/certs
   0 lrwxrwxrwx  1 root root   40 Nov  8 08:25 clientcerts -> /etc/opt/novell/apache2/conf/clientcerts
4.0K drwxr-xr-x  2 root root 4.0K Nov  8 08:25 libexec
4.0K drwxr-xr-x  2 root root 4.0K Nov  8 08:25 sbin
4.0K drwxr-xr-x  4 root root 4.0K Sep 25 02:06 share
 

irzave09:/opt/novell/apache2 # cd certs.orig/

irzave09:/opt/novell/apache2/certs.orig # cp myProxyCert.pem /etc/opt/novell/apache2/conf/certs/

D: fini      120777  1 (   0,   0)        36 /opt/novell/apache2/cacerts;509ad49a
D: fini      120777  1 (   0,   0)        34 /opt/novell/apache2/certs;509ad49a
LZDIO:      23 reads,   188416 total bytes in 0.007687 secs
###############################################
error: unpacking of archive failed on file /opt/novell/apache2/certs: cpio: rename failed - Inappropriate ioctl for device
D: ========== +++ novell-apache-gateway-session-cache-3.2.1-12 x86_64-linux 0x0