Environment
Novell eDirectory 8.8 SP7 for all platforms
Situation
Server has been removed from the tree
Stuck obituaries in DSREPAIR | Check External References
Error: -637 "Previous move in progress"
Servers in obituary list not notified
Obituary flags state=0000
Obituaries won't process
Obituary types, flags, and stages.
Stuck obituaries in DSREPAIR | Check External References
Error: -637 "Previous move in progress"
Servers in obituary list not notified
Obituary flags state=0000
Obituaries won't process
Obituary types, flags, and stages.
Resolution
Obituaries (relevant background information)
- Because servers do not receive NDS updates simultaneously, they may not hold the same information at a given time.
- Each server holds on to the old information until all other servers in a replica ring receive the updates.
- NDS uses obituaries to keep track of this information.
- Move obituaries are placed on both the old object and the new until all replicas acknowledge that the move has completed.
- When obituaries just sit there and do not get removed, it is because one or more servers referencing the object have not yet acknowledged the change.
Obituaries go through 4 flag stages.
- Flags=0000 (Issued)
- Flags=0001 (Notified)
- Flags=0002 (Ok_to_purge)
- Flags=0004 (Purgeable)
Once an obituary is marked Flags=0004 (Purgeable), it is then up to each server's local database to purge the obituary.
Common Obituary Types:
- Type=0003 (Inhibit_Move) - Attached to an object that has been moved from one location to another location (this may or may not be across partition boundaries). This obituary prevents the object from being moved again until the previous move has completed.
- Type=0002 (Moved obituary) - Attached to an object that has been moved.
- Type=0006 (Backlink obituary) · This is a secondary obituary that represents the object on another server·s database which must be notified when the object is modified (IE. deleted, renamed, moved, etc.). Backlink obituaries may represent real copies of the object as found on a READ/WRITE replica or any server with an external reference to the object.
-Type=000C (Used By) - Used By attribute contains a list of partitions that have an interest in this object and need to be notified of changes to the modified object. The creation time of the primary obituary is stored on the Used By obituary value.
Given the following information from DSRepair, it is translated as follows:
Found obituary at VID: 00054980, EID: 11000FE8, DN: CN=User3.OU=Obit_Test.O=Novell.NOVELL_TREE
TV: 1997/08/07 10:28:06 0004, Type = 0001 DEAD, Flags = 0000
- VID = Value ID. This is a record number in the VALUE.NDS file
that has been used as an Obituary Attribute for the Object
identified by the EID.
- EID = Entry ID. This is a record number in the ENTRY.NDS file
that specifies the object that has the Obituary Attribute
assigned by the VID.
- DN = Distinguished Name. This is the fully distinguished name of the object identified by the EID.
- TV = Time Vector. This is the timestamp that denotes when the
Obituary Attribute was created.
- Type = Indicates both a number and a text description. There are 3 categories of types, Primary, Secondary and Tracking. A Primary obituary indicates an action on an object. A Secondary obituary indicates the servers that must be contacted and informed of the Primary obituary action. A Tracking obituary is associated with certain Primary obituaries. The following is a list of the valid obituary types:
Primary obituaries
0000 Restored
0001 Dead
0002 Moved
0005 NEW_RDN - New Relative Distinguished Name
0008 Tree_NEW_RDN - Tree New Relative Distinguished Name. (This does not specify a NDS Tree name but rather a Partition Root name).
0009 Purge All
Secondary Obituaries.
0006 Backlink - Specifies a target server that needs to be contacted regarding an obituary
0010 Move Tree - This obituary is similar to the Backlink obit. There is one Move Tree obit for every server that needs to be contacted regarding a Tree_NEW_RDN operation.
Tracking Obituaries.
0003 Inhibit Move
0004 OLD_RDN - Old Relative Distinguished Name
0007 Tree_OLD_RDN - Tree Old Relative Distinguished Name.
This does not specify a NDS Tree name but rather a Partition Root name.
Flags = Indicates the level or stage that the to which the obituary is processed.
The following is a list of valid Flags:
- 0000 ISSUED - Indicates the obituary has been issued, that is created and ready for processing.
- 0001 NOTIFIED - Indicates that the obituary is at the notify stage which e.ssentially means that the servers identified in the Backlink or Tree Move obituaries have been contacted and notified of the operation or action on an object.
- 0002 OK-TO-PURGE - Indicates that the obituary is being cleaned up on the local database of each server identified in the Backlink or Tree Move obituaries. This clean up includes resolving all objects that reference the object with the obituary and informing them of the change. (Deletion, move, etc).
- 0004 PURGEABLE - Indicates that the obituary is ready to be purged. The purge process essentially recovers the value to the free chain and enables it to be reused.
.
Check for a backlink trying to notify a server that is no longer in the tree. Remove the dead server's objects with NDS Manager. The obituary should then process.
Use the following DSTRACE commands to start the backlink process:
Set DSTRACE=+blink
Set DSTRACE=*b
- Because servers do not receive NDS updates simultaneously, they may not hold the same information at a given time.
- Each server holds on to the old information until all other servers in a replica ring receive the updates.
- NDS uses obituaries to keep track of this information.
- Move obituaries are placed on both the old object and the new until all replicas acknowledge that the move has completed.
- When obituaries just sit there and do not get removed, it is because one or more servers referencing the object have not yet acknowledged the change.
Obituaries go through 4 flag stages.
- Flags=0000 (Issued)
- Flags=0001 (Notified)
- Flags=0002 (Ok_to_purge)
- Flags=0004 (Purgeable)
Once an obituary is marked Flags=0004 (Purgeable), it is then up to each server's local database to purge the obituary.
Common Obituary Types:
- Type=0003 (Inhibit_Move) - Attached to an object that has been moved from one location to another location (this may or may not be across partition boundaries). This obituary prevents the object from being moved again until the previous move has completed.
- Type=0002 (Moved obituary) - Attached to an object that has been moved.
- Type=0006 (Backlink obituary) · This is a secondary obituary that represents the object on another server·s database which must be notified when the object is modified (IE. deleted, renamed, moved, etc.). Backlink obituaries may represent real copies of the object as found on a READ/WRITE replica or any server with an external reference to the object.
Given the following information from DSRepair, it is translated as follows:
Found obituary at VID: 00054980, EID: 11000FE8, DN: CN=User3.OU=Obit_Test.O=Novell.NOVELL_TREE
TV: 1997/08/07 10:28:06 0004, Type = 0001 DEAD, Flags = 0000
- VID = Value ID. This is a record number in the VALUE.NDS file
that has been used as an Obituary Attribute for the Object
identified by the EID.
- EID = Entry ID. This is a record number in the ENTRY.NDS file
that specifies the object that has the Obituary Attribute
assigned by the VID.
- DN = Distinguished Name. This is the fully distinguished name of the object identified by the EID.
- TV = Time Vector. This is the timestamp that denotes when the
Obituary Attribute was created.
- Type = Indicates both a number and a text description. There are 3 categories of types, Primary, Secondary and Tracking. A Primary obituary indicates an action on an object. A Secondary obituary indicates the servers that must be contacted and informed of the Primary obituary action. A Tracking obituary is associated with certain Primary obituaries. The following is a list of the valid obituary types:
Primary obituaries
0000 Restored
0001 Dead
0002 Moved
0005 NEW_RDN - New Relative Distinguished Name
0008 Tree_NEW_RDN - Tree New Relative Distinguished Name. (This does not specify a NDS Tree name but rather a Partition Root name).
0009 Purge All
Secondary Obituaries.
0006 Backlink - Specifies a target server that needs to be contacted regarding an obituary
0010 Move Tree - This obituary is similar to the Backlink obit. There is one Move Tree obit for every server that needs to be contacted regarding a Tree_NEW_RDN operation.
Tracking Obituaries.
0003 Inhibit Move
0004 OLD_RDN - Old Relative Distinguished Name
0007 Tree_OLD_RDN - Tree Old Relative Distinguished Name.
This does not specify a NDS Tree name but rather a Partition Root name.
Flags = Indicates the level or stage that the to which the obituary is processed.
The following is a list of valid Flags:
- 0000 ISSUED - Indicates the obituary has been issued, that is created and ready for processing.
- 0001 NOTIFIED - Indicates that the obituary is at the notify stage which e.ssentially means that the servers identified in the Backlink or Tree Move obituaries have been contacted and notified of the operation or action on an object.
- 0002 OK-TO-PURGE - Indicates that the obituary is being cleaned up on the local database of each server identified in the Backlink or Tree Move obituaries. This clean up includes resolving all objects that reference the object with the obituary and informing them of the change. (Deletion, move, etc).
- 0004 PURGEABLE - Indicates that the obituary is ready to be purged. The purge process essentially recovers the value to the free chain and enables it to be reused.
.
Check for a backlink trying to notify a server that is no longer in the tree. Remove the dead server's objects with NDS Manager. The obituary should then process.
Use the following DSTRACE commands to start the backlink process:
Set DSTRACE=+blink
Set DSTRACE=*b
Additional Information
Communication problems between servers in the tree.
Formerly known as TID# 10013126
Formerly known as TID# 10013126