Red Hat DIRECTORY SERVER 7.1 - ADMINISTRATOR Administrator's Manual page 387

In the redesigned index, the storage manager has visibility into the fine-grain index
structure, which optimizes transaction logging so that only the number of bytes
actually changed need to be logged for any given index modification. The
BerkeleyDB feature provides ID list semantics, which are implemented by the
storage manager. The enhanced Berkeley API supports duplicate index keys,
inserting and deleting individual IDs stored against a common key, and an
optimized mechanism retrieve the complete ID list for a given key.
This means the storage manager has direct knowledge of the application's intent
when changes are made to ID lists. As a result:
• For long ID lists, the number of bytes written to the transaction log for any
update to the list is significantly reduced, from the maximum ID list size
(8Kbytes) to twice the size of one ID (4bytes).
• For short ID lists, storage efficiency, and in most cases performance, is
improved because only the storage manager metadata need to be stored, not
the ID list metadata.
• The average number of database pages marked as dirty per ID insert or delete
operation is very small because a large number of duplicate keys will fit into
each database page.
For each entry ID list, there is a size limit that is globally applied to all index keys
managed by the server. This limit used to be called the All IDs Threshold, which set
a limit on how large a single entry ID list could get because maintaining large ID
lists in memory can affect performance. When a list hit a certain pre-determined
size, the search acted as if the index contained the entire directory.
The difficulty in setting the All IDs Threshold hurt peformance. If the threshold
was too low, too many searches examined every entry in the directory. If it was too
high, too many large ID lists had to be maintained in memory.
The problems addressed by the All IDs Threshold are no longer present because of
the efficiency of entry insertion, modification, and deletion in the BerkeleyDB
design. The All IDs Threshold is removed for database write operations, and every
ID list is now maintained accurately.
Since loading a long ID list from the database can still significantly reduce search
performance, the configuration parameter
on the number of IDs that is read before a key is considered to equal the entire
directory.
nsslapd-idlistscanlimit
only applies to the behavior of the search, not the content of the database. See
"Overview of the Searching Algorithm," on page 392, for more information on
searching and indexes.
nsslapd-idlistscanlimit
is analagous to the All IDs Threshold, but it
Chapter 10
About Indexes
sets a limit
Managing Indexes 387