Abend 1 on P00: Server-5.70.06-0: Deadlock detected waiting for spinlock currently owned by CPU 00

At irregular intervals the server was since abending with the following abend:

"Abend 1 on P00: Server-5.70.06-0: Deadlock detected waiting for spinlock currently owned by CPU 00"

The NLM being the thread owner at abend time differed per abend, like for example sometimes it was abending in DBSRV8.NLM and another time it was for example abending in LPROTECT.NLM.

When searching the stack in the Abend.log for the various Spinlock abends you can see a recurring pattern.

We always found the following below in the stack for these abends:

(LOADER.NLM|kspinlock_patch+76)
(SERVER.NLM|VM_SPIN_LOCK+A)
(SERVER.NLM|VMSpinLock+0)

In the past we have seen similar VM_SPIN_LOCK issues and coredump analysis taught us that certain memory node structures appeared to have become corrupted when they were just recently accessed by Shared Memory APIs. We found the root cause for some of those abends to be that the System Bios had incorrectly reported areas of ram back to the Operating System saying that it was available for normal memory allocation calls.

In this particular case, the server was a HP Proliant DL380-G3, with a P28 Bios dated 08/16/2002.

For this particular model the HP Minimum required Bios revision is dated 09/15/2004 as this contains critical bug fixes and we found that applying the latest firmware resolved the various spinlock abends.