eDirectory's LDAP performance decreases when using negative filters in search.

  • 3851649
  • 23-Jan-2007
  • 26-Apr-2012

Environment

Novell NetWare 6.5 Support Pack 5
Novell eDirectory 8.7.3.9 for NetWare 6.5

Situation

The tree has 20-30 groups a few of which contain 30,000 members. Nevertheless, previous client LDAP-based applications in the environment had an excellent response time. Often member searches against groups were being returned in only a few milliseconds.
A new application that also uses LDAP for client lookups\authentications was being evaluated. This application demonstrated a search response time for member lookups ranging in the 200ms range. Though still subsecond, there is concern that the search time may significantly increase under heavy load. The server went to a 20% utilization when only one search was run.
A sample of a negative filter in a LDAP search is below:
ldapsearch - L - s base - b"cn=Portal Users,ou=groups,o=Novell"- D cn=admin,o=novell - h 10.1.2.1"(!(member=cn=user1,ou=employees,ou=people,o=Novell))"

Resolution

High group member counts carry some overhead. This can affect response time to queries and lengthens the calculation of the users security rights.
Negative filters are aptly named. A negative filter has nothing but drawbacks since LDAP is no longer performing a presence search on whether a user is a member of a group. Rather it must build a member list of all entries in that group then apply the filter to it. There is also the overhead of calculating the user object's or PUBLIC's security equivalence required to determine whether LDAP should return each entry to that user in the the search return.
Other downsides include:
- Regardless of whether appropriate indexes are in place, the indexes will not be used when a negative filter is in the search.
- Even if the fdn of the group in the search is known LDAP will have to scan all entries in that group to compile a list for the filter then apply the filter prior to the return. Due to the high number of members in these groups the search will take some time.

The only way to significantly speed up the search return time is to change the search being performed to one that simply does a lookup for the user's presence in a group's membership list. Then the built-in indexes for member and equivilant-to-me can be utilized in the search.

Incremental changes to help:

- Move as many services as possible off this box (file serving, iMgr, DirXML, GW, etc).
- Reduce the NSS cache and user space (server -uxxxxxxx) on the server if java requirements are small enough in order to free up memory. Check the max service processes to ensure the correct number of threads are being assigned to do these searches as well as any needed background processes. (The max on NetWare is 10,000 thereby giving eDirectory a maximum of 5000.)
- Assign the additional memory to eDirectory's dib cache. Please refer to eDirectory's documentation for more detail on how to improve performance.
For a starting point assuming a 1GB dib:
Static cache size: 750-1GB
Block cache percentage: 40
Ensurethe last line in the sys:\_netware\ndsdb.ini file is preallocatecache=true with a hard carridge return after it. After a week go into that server's instance of iMonitor, examine the return rate of the cache and adjust accordingly.
- Verify the server hardware is adequate and possibly load balance among several servers.