Resolution
AD Aggregation Problem Description:
If you receive a Null Pointer Exception error or an error like the following when performing an AD Account Aggregation:
Exception during aggregation. Reason: java.lang.RuntimeException: javax.naming.SizeLimitExceededException: [LDAP: error code 4 - Sizelimit Exceeded]; remaining name ''
This may be the result of index problems with AD. Try using a good LDAP tool and querying for all entries, maybe using a search like:
(objectclass=*) # returns all entries
Compare the number of results you get back to the results from the following 2 queries:
(&(sAMAccountName>=m)(sAMAccountName<=0))
(sAMAccountName<m)
One situation in which we saw this issue is when aggregating from a root domain that is not setup to properly enumerate accounts within the child domains. In order to do this, we had to use multiple searchDNs and groupSearchDNs for each of the subdomains along with ensuring we would get no more than one max page size of data from each of those search dns. For some reason the '=' indices were broken and would not return all the results so we had to use '>=' and '<=' in our searchDNs.
Workaround:
There were two ways to work around this issue that produced the same results.
Option 1: Set the following options on your AD Account Aggregation:
<entry key="useHasMoreElements" value="true"/>
<entry key="pageSize" value="100"/>
<entry key="iterateModeOverride" value="DEFAULT"/>
Option 2: Set only the following option on your AD Account Aggregation:
<entry key="pageSize" value="1000"/>
With Option 1, you don't have to use as many search DNs but it runs much more slowly than Option 2.
With Option 2, you have to be sure that none of your searchDNs return more than 1000 results.
Here is a sample set of accountsearchDNs and groupsearchDNs used with Option 2 to ensure that none of the searchDNs returns more than 1000 records (based on the number of accounts and naming conventions of the sample customer). Remember we are searching from the root node of "DC=foo,DC=bar" rather than using searchDNs that reference each of the 6 subtrees that would require far more searchDNs. Also note that certain characters (e.g. '&', '<', '>') should be substituted (e.g., '&', '<', '>') as appropriate for inclusion in an XML document.
accountsearchDNs:
<entry key="accountsearchDNs">
<value>
<List>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName>=x18))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=x18)(sAMAccountName>=x17))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=x17)(sAMAccountName>=x16))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=x16)(sAMAccountName>=x15))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=x15)(sAMAccountName>=x14))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=x14)(sAMAccountName>=x13))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=x13)(sAMAccountName>=x12))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=x12)(sAMAccountName>=x11))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=x11)(sAMAccountName>=m))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=m)(sAMAccountName>=k))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=k)(sAMAccountName>=h))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=h)(sAMAccountName>=c))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=c)(sAMAccountName>=b))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=user)(sAMAccountType=805306368)(sAMAccountName<=b)(sAMAccountName>=a))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
</List>
</value>
</entry>
groupsearchDNs:
<entry key="groupsearchDNs">
<value>
<List>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName>=w))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName<=w)(sAMAccountName>=s))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName<=s)(sAMAccountName>=m))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName<=m)(sAMAccountName>=j))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName<=j)(sAMAccountName>=i))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName<=i)(sAMAccountName>=d))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName<=d)(sAMAccountName>=b))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName<=b)(sAMAccountName>=9))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
<Map>
<entry key="iterateSearchFilter"
value="(&(objectClass=group)(sAMAccountType=268435456)(sAMAccountName<=9)(sAMAccountName>=0))"/>
<entry key="searchDN" value="DC=foo,DC=bar"/>
<entry key="searchScope" value="SUBTREE"/>
</Map>
</List>
</value>
</entry>
This retrieves all accounts and groups.