OES2 Ravsui rebuild including rezid on volumes with hard links causes data corruption.

  • 7007002
  • 05-Oct-2010
  • 27-Apr-2012

Environment

Novell Open Enterprise Server 2 (OES 2) Linux Support Pack 1
Novell Open Enterprise Server 2 (OES 2) Linux Support Pack 2

Situation

On NSS volumes that support hard-links, and where hard-links are in place, it was found that performing a ravsui rebuild in combination with the '-r' (or '--rezid=ZID') action creates data corruption in the following ways :
  • the file (UID) ownership is reset to user 'root'.
  • the hard-link that previously existed is removed
  • file shows as 0 bytes in size
  • file has received a timestamp of Jan 1. 1970

As per section 17.3 of the documentation : "ReZIDing Volumes in an NSS Pool"

<snip>
For NSS on OES Linux, rebuild automatically causes a reZID of a volume if rebuild finds a ZID over 2 billion.

This checks all blocks in the system. Rebuilding can take several minutes to several hours, depending on the number of objects in the pool. For all systems, reZID adds a third pass to the rebuild, which increases the time to rebuild a volume by about 50%.
<snip>

To find out if hard-link support for existing volumes is in place, you can cat the content from
the /_admin/Manage_NSS/Volume/<Volume_name>/VolumeInfo.xml file for the following:

- the mediaLayout tag of this file needs to have at least v38.05
- the <enabledAttributes> section of this file contains the word HARD_LINKS.

Resolution

A solution for this issue has been released with the September 2010 Scheduled Maintenance updates, respectively for:

- OES2 SP1 : oes2sp1-Sept-2010-Scheduled-Maintenance-7143-4
- OES2 SP2 : oes2sp2-Sept-2010-Scheduled-Maintenance-7145-6

Additional Information

[1] created hard-link
OES2:/media/nss/FBACKUP # ln manuals/gw8/gw8_admin/admin_01_system.pdf admin_01_system.pdf
OES2:/media/nss/FBACKUP # ln manuals/gw8/gw8_admin/admin_01_system.pdf file1.test



[2] check for inodes 25870 below
OES2:/media/nss/FBACKUP # ls -il
total 2836
  129 d--x--x--x 1 root  root       0 Aug  5 11:49 ._NETWARE
  395 drwxrwxrwx 1 root  root       0 Aug  5 10:44 4c5a79da.FPD
  397 drwxrwxrwx 1 root  root       0 Aug  5 10:50 4c5a7b4f.FPD
  400 drwxrwxrwx 1 root  root       0 Aug  5 10:59 4c5a7d78.FPD
13325 drwxrwxrwx 1 root  root       0 Aug  5 11:34 4c5a85a4.FPD
13326 drwxrwxrwx 1 root  root       0 Aug  5 11:37 4c5a863b.FPD
  131 drwxrwxrwx 1 root  root       0 Aug  5 09:52 DESKTOP.AFP
25870 -rw-rw-rw- 3 user1 root 1449665 Feb 11  2009 admin_01_system.pdf  <<here
25870 -rw-rw-rw- 3 user1 root 1449665 Feb 11  2009 file1.test  <<here
13332 drwxrwxrwx 1 user1 root       0 Aug  5 13:36 grpwise
13333 drwxrwxrwx 1 user1 root       0 Aug  5 13:54 manuals
  128 -r--r--r-- 1 root  root      24 Aug  5 09:52 ~DFSINFO.8-P


[3] check inode  //inodes 25870 - admin_01_system.pdf//
OES2:/media/nss/FBACKUP # ls -il manuals/gw8/gw8_admin/
total 44384
25870 -rw-rw-rw- 3 user1 root  1449665 Feb 11  2009 admin_01_system.pdf <<here
25871 -rw-rw-rw- 1 user1 root  1999181 Feb 11  2009 admin_02_domains.pdf
25872 -rw-rw-rw- 1 user1 root  1277981 Feb 11  2009 admin_03_post_offices.pdf
25873 -rw-rw-rw- 1 user1 root   975532 Feb 11  2009 admin_04_users.pdf
25874 -rw-rw-rw- 1 user1 root   353363 Feb 11  2009 admin_05_resources.pdf
25875 -rw-rw-rw- 1 user1 root   541528 Feb 11  2009 admin_06_distribution_lists.pdf
25876 -rw-rw-rw- 1 user1 root  2034498 Feb 11  2009 admin_07_libs_and_docs.pdf
25877 -rw-rw-rw- 1 user1 root  1602175 Feb 11  2009 admin_08_databases.pdf
25878 -rw-rw-rw- 1 user1 root  2495873 Feb 11  2009 admin_09_poa.pdf
25879 -rw-rw-rw- 1 user1 root  1706922 Feb 11  2009 admin_10_mta.pdf
25880 -rw-rw-rw- 1 user1 root  2463129 Feb 11  2009 admin_11_internet_agent.pdf
25881 -rw-rw-rw- 1 user1 root  2324921 Feb 11  2009 admin_12_webaccess.pdf
25882 -rw-rw-rw- 1 user1 root   585724 Feb 11  2009 admin_13_calpubhost.pdf
25883 -rw-rw-rw- 1 user1 root  1441669 Feb 11  2009 admin_14_monitor.pdf
25884 -rw-rw-rw- 1 user1 root  1394455 Feb 11  2009 admin_15_client.pdf
25885 -rw-rw-rw- 1 user1 root   767369 Feb 11  2009 admin_16_security_admin.pdf
25886 -rw-rw-rw- 1 user1 root   217553 Feb 11  2009 admin_17_security_policies.pdf
25887 -rw-rw-rw- 1 user1 root 21774655 Nov 21  2008 gw8_admin.pdf

Note: the inode from hard-link is still the same: 25870


[4] New SoftLinks //inodes 13334 & 13335//
OES2:/media/nss/FBACKUP # ln -s manuals/gw8/gw8_admin/admin_01_system.pdf file1.test.soft
OES2:/media/nss/FBACKUP # ln -s manuals/gw8/gw8_admin/admin_01_system.pdf file1.test.soft2


OES2:/media/nss/FBACKUP # ls -il
total 2844
  129 d--x--x--x 1 root  root       0 Aug  5 11:49 ._NETWARE
  395 drwxrwxrwx 1 root  root       0 Aug  5 10:44 4c5a79da.FPD
  397 drwxrwxrwx 1 root  root       0 Aug  5 10:50 4c5a7b4f.FPD
  400 drwxrwxrwx 1 root  root       0 Aug  5 10:59 4c5a7d78.FPD
13325 drwxrwxrwx 1 root  root       0 Aug  5 11:34 4c5a85a4.FPD
13326 drwxrwxrwx 1 root  root       0 Aug  5 11:37 4c5a863b.FPD
  131 drwxrwxrwx 1 root  root       0 Aug  5 09:52 DESKTOP.AFP
25870 -rw-rw-rw- 3 user1 root 1449665 Feb 11  2009 admin_01_system.pdf  <<here
25870 -rw-rw-rw- 3 user1 root 1449665 Feb 11  2009 file1.test  <<here
13334 lrwxrwxrwx 1 root  root      42 Aug  5 14:14 file1.test.soft -> manuals/gw8/gw8_admin/admin_01_system.pdf
13335 lrwxrwxrwx 1 root  root      42 Aug  5 14:14 file1.test.soft2 -> manuals/gw8/gw8_admin/admin_01_system.pdf
13332 drwxrwxrwx 1 user1 root       0 Aug  5 13:36 grpwise
13333 drwxrwxrwx 1 user1 root       0 Aug  5 13:54 manuals
  128 -r--r--r-- 1 root  root      24 Aug  5 09:52 ~DFSINFO.8-

[5] perform the ravsui --rezid rebuild poolname
After ravsui --rezid=0x00000080 has rebuilt FBACKUP:

OES2:/media/nss/FBACKUP # ls -il  //see inodes 13334, 13335 & 13543 //
total 12
  129 d--x--x--x 1 root  root  0 Aug  5 14:28 ._NETWARE
  395 drwxrwxrwx 1 root  root  0 Aug  5 10:44 4c5a79da.FPD
  397 drwxrwxrwx 1 root  root  0 Aug  5 10:50 4c5a7b4f.FPD
  400 drwxrwxrwx 1 root  root  0 Aug  5 10:59 4c5a7d78.FPD
13325 drwxrwxrwx 1 root  root  0 Aug  5 11:34 4c5a85a4.FPD
13326 drwxrwxrwx 1 root  root  0 Aug  5 11:37 4c5a863b.FPD
13546 drwxrwxrwx 1 root  root  0 Aug  5 14:27 4c5aae4c.FPD
  131 drwxrwxrwx 1 root  root  0 Aug  5 09:52 DESKTOP.AFP
13544 -rw-rw-rw- 1 root  root  0 Jan  1  1970 admin_01_system.pdf <<here
13545 -rw-rw-rw- 1 root  root  0 Jan  1  1970 file1.test  <<here
13334 lrwxrwxrwx 1 root  root 42 Aug  5 14:14 file1.test.soft -> manuals/gw8/gw8_admin/admin_01_system.pdf  <<
13335 lrwxrwxrwx 1 root  root 42 Aug  5 14:14 file1.test.soft2 -> manuals/gw8/gw8_admin/admin_01_system.pdf  <<
13332 drwxrwxrwx 1 user1 root  0 Aug  5 13:36 grpwise
13333 drwxrwxrwx 1 user1 root  0 Aug  5 13:54 manuals
  128 -r--r--r-- 1 root  root 24 Aug  5 09:52 ~DFSINFO.8-P

OES2:/media/nss/FBACKUP # ls -il manuals/gw8/gw8_admin/
total 42968
13543 -rw-rw-rw- 1 root  root        0 Jan  1  1970 admin_01_system.pdf  <<here
13526 -rw-rw-rw- 1 user1 root  1999181 Feb 11  2009 admin_02_domains.pdf
13527 -rw-rw-rw- 1 user1 root  1277981 Feb 11  2009 admin_03_post_offices.pdf
13528 -rw-rw-rw- 1 user1 root   975532 Feb 11  2009 admin_04_users.pdf
13529 -rw-rw-rw- 1 user1 root   353363 Feb 11  2009 admin_05_resources.pdf
13530 -rw-rw-rw- 1 user1 root   541528 Feb 11  2009 admin_06_distribution_lists.pdf
13531 -rw-rw-rw- 1 user1 root  2034498 Feb 11  2009 admin_07_libs_and_docs.pdf
13532 -rw-rw-rw- 1 user1 root  1602175 Feb 11  2009 admin_08_databases.pdf
13533 -rw-rw-rw- 1 user1 root  2495873 Feb 11  2009 admin_09_poa.pdf
13534 -rw-rw-rw- 1 user1 root  1706922 Feb 11  2009 admin_10_mta.pdf
13535 -rw-rw-rw- 1 user1 root  2463129 Feb 11  2009 admin_11_internet_agent.pdf
13536 -rw-rw-rw- 1 user1 root  2324921 Feb 11  2009 admin_12_webaccess.pdf
13537 -rw-rw-rw- 1 user1 root   585724 Feb 11  2009 admin_13_calpubhost.pdf
13538 -rw-rw-rw- 1 user1 root  1441669 Feb 11  2009 admin_14_monitor.pdf
13539 -rw-rw-rw- 1 user1 root  1394455 Feb 11  2009 admin_15_client.pdf
13540 -rw-rw-rw- 1 user1 root   767369 Feb 11  2009 admin_16_security_admin.pdf
13541 -rw-rw-rw- 1 user1 root   217553 Feb 11  2009 admin_17_security_policies.pdf
13542 -rw-rw-rw- 1 user1 root 21774655 Nov 21  2008 gw8_admin.pdf