Caching in eDirectory 8 and 8.5

The caching has been significantly enhanced for new versions of eDirectory 8 and eDirectory 8.5. This information should help to assist in setting optimal performance for the memory and DS databases on each server.
Performance issues after upgrading DS from 8.59 or earlier versions. How to set the cache to perform on NetWare 5x with eDir 8 & 8.5.
Improved Caching in eDirectory 8.5 and greater

Understanding Caching for NDS 8 and NDS eDirectory 8.5

With NDS eDirectory 8.5 two types of caching have been implemented:

·        Block (or page) Caching
·        Entry Caching (sometimes referred to as record cache)

Physical block or block caching is typically implemented by database systems. NDS 8 has employed this type of caching from the introduction. Before reading a block (or page) from the database, NDS will search its block cache for the needed information.

Entry caching was introduced in NDS 8 with DS version 8.73 and above and with the initial release of NDS eDirectory 8.5. The entry cache contains logical entries in the directory as opposed to physical blocks.

Block and entry caches have a certain amount of redundancy, but each is designed to boost performance for different operations. Block cache is typically used for update operations, while entry cache is most useful for operations that tend to browse and traverse the NDS tree by reading through entries. Both cache types work to provide optimal performance for retrieval and updates to information. For example, block cache is helpful when it comes to searching indexes, whereas the entry cache helps when it comes to retrieving entries referenced from an index. By default, NDS will equally split the allocated "NDS Cache" across the block and entry caches.

Note: DS versions previous to NDS 8 version 8.73 only support block caching.

Cache settings for DS version previous to 8.73 may not be sufficient with new NDS 8.5 versions. For example, if your DS 8.73 version has performed correctly with only 50 MB of cache before implementing eDirectory 8.5, you may find that performance has decreased after the upgrade. This is because only 25 MB is now allowed for block caching as opposed to the 50 MB before upgrading. Likewise, the entry caching may be insufficient with only 25MB allocated to it.

To ensure optimal performance you should configure your system based on cache fault indicators, database size, and memory available. Do not expect to cache the entire database. Don't expect to see zero cache faults and don't expect the faults to be at zero to have optimal performance in eDirectory.

NOTE: This is a very broad, general rule of thumb. It could vary in different implementations of eDirectory. You must consider your overall environment, hardware, database size, etc. to optimally configure database caching. Understanding how the new caching works should help you assess your requirements.

If you want to maximize the amount of memory available for DS, a formula that could be used to calculate the maximum amount of memory needed is:

((SizeOfDIBSet + (SizeOfDIBSet * 4)) = MemoryForNDSDIB

Where SizeOfDIBSet equals the bytes for all files that start with NDS in the DIB directory (for NetWare this is found in the SYS:_NETWARE directory).

Now that you have an amount of memory the database may need, you can check to see if it exceeds the 60% limit by dividing MemoryForNDSDIB by the Total Server Memory and multiplying that amount by 100.

If you do exceed the limit, you may want to adjust the multiplier of 4 down to 2 (do not go below 2 on this multiplier). If you still exceed the limit, either get more memory or you experience some performance degradation.


Monitoring and Configuring Cache for NDS 8 (not NDS eDirectory 8.5)

With NDS 8 (not to be confused with eDirectory 8.5), the administrator can regulate the amount of memory NDS will consume for cache by specifying a cache limit. If no limit is explicitly set, NDS will use a default of 8 MB. To find out what the current cache setting are, an administrator can turn on DSTRACE at the console and setting DSTRACE to *P.

For example, to view the cache settings on a NetWare server, you would type following commands at the c.onsole:

SET DSTRACE=ON
SET DSTRACE=*P

Then toggle over the Directory Services screen and you should see something similar to the following:

TUNABLE PARAMETER VALUES

        ServerStateUpThreshold = 30 minutes
        External Reference Life Span = 193 hours
        JanitorInterval = 2 minutes
        FlatCleaningInterval = 720 minutes
        BacklinkInterval = 780 minutes
        Heartbeat Data = 60 minutes
        Heartbeat Schema = 240 minutes
        Schema synchronization enabled = 1
        SMI Max Cache = 60000000 (Allocated = 5283880, Blks In Use = 0)
        SMI Entries Cached Per Thread = 50 (Cached: 3, ~= 6912 bytes)
        SMI Attr Recs Cached Per Thread = 20 (Cached: 0, ~= bytes)
        SMI Partitions Cached Per Thread = 20 (Cached: 5)
        SMI Force Checkpoint Interval = 180 (cannot change)
        SMI Maximum Read Transaction Seconds = 2400 (cannot change)
        SMI Maximum Read Transaction Inactive Seconds = 30 (cannot change)

The "SMI Max Cache" line represents the maximum amount the cache allocated to NDS. In this example, it is currently set to 60000000 bytes or about 60 MB.

If you want to change the NDS 8 cache hard limit (again do not confuse this with eDirectory 8.5), you can type the following syntax to set a hard maximum cache amount on a NetWare server:

SET DSTRACE=!MB<amount_of_RAM_to_use_in_bytes>

For example, to set a hard limit of 8 MB you would type the following command at the NetWare console:

SET DSTRACE=!MB8388608

This command will set a persistent setting in a file called _NDSDB.INI file found in the SYS:\_NETWARE directory on a NetWare server. The following represents the cache parameter that would be set in the _NDSDB.INI file when using the SET command above for 8MB:

cache=8388608


Monitoring and Configuring Cache for NDS eDirectory 8.5

NDS eDirectory 8.5 provides tools for monitoring how well cache is being utilized. NDS iMonitor (release with eDirectory 8.5) allows an administrator to monitor cache hits and cache faults for both the block cache and entry cache. A cache hit occurs whenever eDirectory finds an item in its cache. A cache fault occurs whenever an item is not found in cache and must be fetched from disk. If there are lots of cache faults, increasing the amount of cache memory may help performance. The ideal system would be one in which there were no cache faults. This is only possible if eDirectory is able to cache the entire database.

With the introduction of eDirectory 8.5, two types of mechanisms for controlling cache memory consumption are provided. Each is mutually exclusive of the other. These mechanisms are:

·        Dynamically Adjusting Limit
·        Hard Memory Limit


The administrator can use either one at any time, but the last one used always replaces the prior settings.


Dynamically Adjusting Limit

The first mechanism that is provided for regulating memory consumption is a dynamically adjusting limit. A dynamically adjusting limit causes eDirectory to periodically adjust its memory consumption in response to the needed memory for caching and flow of memory consumption by other processes. The user. specifies the limit as a percentage of available physical memory. Using this percentage, eDirectory recalculates a new memory limit at fixed intervals. The new limit is simply the percentage of physical memory that is available at that point in time.

This limit is used to control cache memory consumption within NDS eDirectory until the next calculation occurs. Thus, eDirectory's responsiveness in adjusting its memory consumption depends on the length of the recalculation interval. The shorter the interval, the more responsive to change. The longer the interval, the less responsive to change. Shorter intervals are not necessarily better, because it will likely cause more memory allocation and freeing to occur.

In connection with the percentage, the user can also set minimum and maximum thresholds. The minimum is the lowest number of bytes that eDirectory will adjust down to. If not set, it defaults to 16 MB. The maximum threshold determines the highest number of bytes that eDirectory will adjust up to. It can be set to either a number of bytes or number of bytes that eDirectory will leave available. If the maximum threshold has not been set using one of these two mechanisms, a default maximum of 4 GB is used.

Note: If a minimum threshold is set and a maximum threshold is set as a number of bytes to leave, it is possible that the calculated maximum will fall below the minimum. In this case, the minimum takes priority over the maximum. For example, suppose that we have following settings:

Minimum threshold: 8 MB
Percentage: 75
Maximum threshold: leave 10 MB

Suppose that there are 16 MB of available physical memory when eDirectory decides to adjust its cache limit. eDirectory would calculate a new limit of 12 MB. It would then check to see if this falls outside the range of the minimum and maximum thresholds.

In this case, the maximum threshold is a calculated value: 16 MB minus 10 MB yields a maximum threshold of 6 MB. The calculated limit of 12 MB is greater that this, so eDirectory would set its limit to 6 MB. eDirectory would then check this limit against the minimum threshold. Since 6 MB is less than 8 MB, eDirectory would set its final limit to 8 MB. Thus, the minimum gets priority over the maximum whenever there is a conflict.
Hard Memory Limit

The second mechanism that is provided for regulating memory consumption is the hard memory limit. This is basically the same mechanism that exists in prior versions of eDirectory. Once a limit like this is set, it is not changed until the user either sets a different hard memory limit or sets a dynamically adjusting limit. Remember that the hard memory limit and dynamically adjusting limits are mutually exclusive. The last setting is what will apply.

An administrator is allowed to specify a hard memory limit in one of three ways:

·        A fixed number of bytes
·        Percentage of total physical memory
·        Percentage of available physical memory

When a hard memory limit is specified by percentages, it is always translated to fixed number bytes..
Default Mechanism for Regulating Memory Consumption

If the server contains a replica, eDirectory will use a dynamically adjusting limit of 51% of available memory, with a minimum of 8 MB and a minimum to leave 24 MB. Otherwise, eDirectory will use a hard memory limit setting of 16 MB.


Settings in _NDSDB.INI File

At startup, eDirectory looks for the database options file, _NDSDB.INI, in the directory where DIB files stored. On Windows NT/2000, this is normally found at \NOVELL\NDS\DIBFILES. On NetWare it is stored in SYS:\_NETWARE. This file is a simple text that can be created or modified with any text editor. If you are using DSTRACE to define your cache settings they will automatically be populated in this file.

NOTE: Although a user may alter the _NDSDB.INI file at any time, the changes will not take effect until the eDirectory module has been restarted. If the change is done via the DSTRACE command no eDirectory module restart is needed.

Below is the syntax for controlling cache memory consumption within NDS eDirectory:


Cache=<cacheBytes>         Set a hard memory limit

Cache=<cache options>         Set a hard memory limit or dynamically adjusting limit. Multiple options may be specified, in any order, separated by commas. All are optional . They are as follows:

DYN or HARD         Dynamically adjusting limit or hard limit
%:<percentage>        Percentage of available or physical memory to use.
AVAIL or TOTAL        Percentage is for available physical memory or total physical memory. Note: These are ignored for a dynamically adjusting limit, because a dynamically adjusting limit is always calculated based on the available physical memory.

MIN: <bytes>        Minimum number of bytes
MAX: <bytes>        Maximum number of bytes
LEAVE: <bytes>        Minimum number of bytes to leave.


Examples:

Set dynamically adjusting limit of 60% of available memory, minimum of 16 MB:

cache=DYN,%:60,MIN:16000000

Set a hard memory limit of 60% of total physical memory, minimum of 16 MB:

cache=HARD,%:75,MIN:16000000

Old style - set a hard memory limit of 8 MB:

cache=8000000

In addition to the cache setting for regulating memory consumption, NDS eDirectory also provides settings for controlling the dynamic adjust interval and the interval for cleaning up older version of entries and blocks. These are as follows:

Cacheadjustinterval=<seconds>         Default is 15 seconds if not set in the _NDSDB.INI file.

Cachecleanupinterval=<seconds> Default is 15 seconds if not set in the _NDSDB.INI file.

Finally, the following setting allows an administrator to control the percentage split between entry and block cache:

Blockcachepercent=<percent>        
<percent> should be between 0 and 100 (inclusive). A value of 70 means that 70 percent of cache memory will used for block cache and 30 percent for entry cache. Default is 50 percent if not set in the _NDSDB.INI file. Note: Setting blockcachepercent to zero is NEVER a good idea.


DSTRACE Options for Regulating Cache

From the console screen in NetWare the following commands can be used to regulate cache memory consumption within NDS eDirectory:

The following is the syntax for setting a simple hard memory limit:

SET DSTRACE=!MB<bytes>         

Here is an example of how to a simple hard memory limit:

SET DSTRACE=!MB8388608

The following is the syntax for setting a calculated hard memory limit:

SET DSTRACE=!MHARD[,AVAIL|TOTAL][,%:<percent>][,MIN:,minimum_bytes]
                [,MAX:<maximum_bytes>][,LEAVE:<min_bytes_to_leave>][,NOSAVE]

Here is an example of how to set a hard memory limit of 75% of total physical memory, with a minimum of 16 MB, and don't save these options to the _NDSDB.INI file.

SET DSTRACE=!MHARD,%75,MIN:16777216,NOSAVE

The following is the syntax for setting a Dynamically Adjusting Limit:

SET DSTRACE=!MDYN,%:<percent>[,MIN:<minimum_bytes>]
[,MAX:<maximum_bytes>][,LEAVE:<min_bytes_to_leave>][,NOSAVE]

Here is an example of how to set a dynamic limit of 75% of available memory, with a minimum of 8 MB.

SET DSTRACE=!MDYN,%:75,MIN:8388608

Note: Configuring cache via DSTRACE does not require a reset of DS or the server. Changes are in effect immediately.

Use iMonitor to Monitor and Configure Cache Settings

The best way to monitor and configure cache settings is use iMonitor, which was released with eDirectory 8.5. Using iMonitor allows you to intuitively view information about the current cache settings, look at the statistics to determine possible adjustments, and make the appropriate settings. Best of all, this is done through an Internet browser for any operating system on the back end (i.e., NetWare, Windows NT/2000, Unix/Linux). Your browser should open up iMonitor if the NDSIMON.NLM is loaded and running on the eDirectory 8.5 server.

The iMonitor section of this information should be referenced in the latest appnote regarding NDS Caching with eDirectory 8.5..

Formerly known as TID# 10060669