Authentications to eDirectory fail with a -632 error: NMAS has no threads available

  • 7025246
  • 13-Aug-2021
  • 16-Sep-2021

Environment

eDirectory 9
ZENworks Configuration Management 2020 Authentication

Situation

Periodically users cannot login and receive a -632 error.  LDAP authentication appears to not be affected.

Ndstrace shows -669 FFFFFD63 FAILED AUTHENTICATION

There are only 5000 connections and eDirectory's thread pool is adequate.

There are multiple failed authentications and intruder lockouts from various applications each second.

Ndstrace is showing that NMAS is out of threads and unable to create a new one.
2526590720 NMAS: [2021/07/08 10:58:14.245] 24379492: Create thread request
2526590720 NMAS: [2021/07/08 10:58:14.245] 24379492: No free threads in pool. Creating a new thread.
2759264000 RSLV: [2021/07/08 10:58:14.245] Respond with local entry succeeded.
2526590720 NMAS: [2021/07/08 10:58:14.245] 24379492: ERROR: 11 Creating new thread
2526590720 NMAS: [2021/07/08 10:58:14.245] 24379492: Using thread 0x0
2526590720 NMAS: [2021/07/08 10:58:14.245] 24379492: ERROR: 11 Creating server thread
2526590720 NMAS: [2021/07/08 10:58:14.246] 24379492: Destroy NMAS Session
2526590720 NMAS: [2021/07/08 10:58:14.246] 24379492: ERROR: 11 Failed to started login refresh session

Resolution

1. This server had close to 16 NMAS authentications per second being performed on it.  By default, the NMAS policy and methods will get refreshed on each login.  The server was approaching the NMAS maximum of 16 read locks on a method.  Therefore, there is a potential that if the the "nmas refreshpolicy" happens during peak login times, it is possible to get into a deadlock-type of issue and all NMAS client logins back up behind the "nmas refresh".
For this, a /var/opt/novell/eDirectory/data/nmas.config file can be created.  The following statement indicates to refresh every 5 minutes.  This can be set up to 1100.  Following it with a carridge return is a good idea:
nmas RefreshRate 5

2. Many failed authentications from running applications were seen.  Sometimes there were multiples per second.  By default, NMAS will delay ALL authentications for 3 seconds following a failed login.  This is to prevent brute force attacks.  However, this many can again force NMAS client logins to stack up consuming all NMAS threads.  Setting the delay to 1 second can help minimize the impact until the applications or users failing to login can be rectified.  This setting can be found in the Admin guide in the Using NetIQ iMonitor section.

Both of these issues resulted in all NMAS theads being consumed and no further logins were possible.

Cause

NMAS maintains its own thread pool separate from eDirectory's and will request threads directly from the OS.