Import of a very large file with ldif2dib leaves a corrupt database

After importing a very large ldif file with ldif2dib (the ldif file should be larger than 8 Gb in size, in this test, of at least 16 million entries) LDAP searches performed on the database report the following errors:

# result: 80 Unknown error

# textNDS error: dib error (-785)

With the introduction of the ldif2dib tool, it is possible to do offline bulkloads of a directory in a very efficient manner. One limitation that was found with this issue was that the tool is not handling correctly very large import files.

The issue has been reported to engineering and is currently under investigation. There is an easy workaround, though, that is simply to split the file into smaller files and import each of the smaller files one at a time. In the case mentioned above, with an import of a file of 8 Gb, it was enough to split the file in two files of 4 Gb each to perform the bulkload correctly.