Vpn client can not make a vpn connection to vpn server

  • 7002892
  • 02-Apr-2009
  • 26-Apr-2012


Novell BorderManager 3.9 Support Pack 2
IKE_20090302.zip applied
VPN client 3.8.16 for windows xp
VPN client 3.9.0 for windows xp
VPN client 3.9.2 for windows xp
VPN client 3.9.1 for windows vista


After apply NBM39SP2 support pack and replaced ike.nlm with the one from IKE_20090302.zip, some vpn clients can not establish a connection to vpn server. After the GUI screen flash for a while, an error is displayed:

"An error was reproted by the ike application
Either "vpn server ip" is an invalid vpn server address or the IKE is not loaded
on the vpn server. For more details please look at the ike.log"

IKE.log on the server shows this error:

3-31-2009 7:45:29 am There is NAT in between server and client
3-31-2009 7:45:29 am ***Send Main Mode message to xx.xx.xx.xx
3-31-2009 7:45:29 am I-COOKIE=454DA369795F41BA,R-COOKIE=8A85A0570CFE6BCD,MsgID=0,1stPL=KEY-PAYLOAD,state=-1721142388
3-31-2009 7:45:29 am ***Receive Main Mode message from xx.xx.xx.xx
3-31-2009 7:45:29 am I-COOKIE=454DA369795F41BA,R-COOKIE=8A85A0570CFE6BCD,MsgID=0,1stPL=ID-PAYLOAD,state=-1721142376
3-31-2009 7:45:29 am  Responder : Nat inbetween change port to 4500
3-31-2009 7:45:29 am Recieved MM ID payload type 1 protocol 0 portnum 0 length 8
3-31-2009 7:45:29 am *Received MM ID ID_IPV4_ADDR
3-31-2009 7:45:29 am Recieved notify message type 24578 from xx.xx.xx.xx
3-31-2009 7:45:29 am MM Preshare Resp : Client's Real address - 0x19DA8C0
3-31-2009 7:45:29 am C2S Preshared key is not configured
3-31-2009 7:45:29 am sending notify message type 35  to xx.xx.xx.xx
3-31-2009 7:45:29 am ***Send Unacknowledge Informational message to xx.xx.xx.xx
3-31-2009 7:45:30 am I-COOKIE=454DA369795F41BA,R-COOKIE=8A85A0570CFE6BCD,MsgID=C0CE9902,1stPL=HASH-PAYLOAD,state=-1721142328
3-31-2009 7:45:30 am Failed to create IKE-SA - Could not get  IPSEC Policy probably wrong QM ID , dst = xx.xx.xx.xx
3-31-2009 7:45:33 am IKE-SA 98FC71C0 is Deleted,I-COOKIE=454DA369,R-COOKIE=8A85A057,dst=xx.xx.xx.xx
3-31-2009 7:45:33 am The client  xx.xx.xx.xx removed from vpninf , vendoridmask = 0x80000001

ikelog.txt shows:

03-31-2009 02:42:55 PM ***Send Main Mode message to vpn server ip

03-31-2009 02:42:55 PM I-COOKIE=f0ce1a5e27f2e75a,R-COOKIE=c2a9d5d2a63d4198,MsgID=0,1stPL=KEY-PAYLOAD,state=20314732

03-31-2009 02:42:55 PM Local Address at 0 : client public ip
03-31-2009 02:42:55 PM Local Address at 1 :
03-31-2009 02:42:55 PM ***Receive Unacknowledge Informational message from vpn server ip

03-31-2009 02:42:55 PM I-COOKIE=f0ce1a5e27f2e75a,R-COOKIE=c2a9d5d2a63d4198,MsgID=720df494,1stPL=NOTIFY-PAYLOAD,state=20314832

03-31-2009 02:42:55 PM Recieved notify message type 54 from vpn server ip
03-31-2009 02:42:55 PM Error :Unknown notify message type 54 recieved from server
03-31-2009 02:42:55 PM Notify Recvd :Deleting IKE SA and related QM SAS - Peer vpn server ip
03-31-2009 02:43:00 PM IKE-SA 1666450 is Deleted,I-COOKIE=f0ce1a5e,R-COOKIE=c2a9d5d2,dst=

Vpn server has not preshared key configured and vpn client is using nmas to authenticate.


This is issue is seen with PC where more than one interface/ip address is available for Windows OS, as for example, having vmware installed. Vpn client uses one for the initial nmas authentication but it sends the other one when negotiating IKE phase I. As they do not match, IKE does not recognize the client as nmas client and check if preshared key is configured. As it is not, error is sent back to vpn client and connection terminated.

This is fix in a new  ikeapp.exe,  that fixes this issue on both vpn clients, XP and Vista.
It is included on the new vpn client 3.9.3, included on the bm39sp2_ir1 patch