Indexing and Optimization

The queue parameters in the table below define the following:

• What quantity of data is indexed at one time
• When the queue server triggers optimization

  Queue Parameter	Purpose
  Index Bulk Size	Maximum number of documents to be indexed at one time. If there are more documents awaiting indexing, the queue server distributes them over more than one indexing run. Tip Indexing takes place at hourly intervals. You should only index a maximum of 1000 document at a time. When the start condition is next reached, 3000 documents are ready for indexing. The queue server distributes the documents over three indexing runs. If there are fewer documents ready for indexing, the queue server nevertheless triggers indexing. Tip Indexing takes place at hourly intervals. You should only index a maximum of 1000 document at a time. When the start condition is next reached, only 900 documents are ready for indexing. Although this is less than specified in the Index Bulk Size parameter, the queue server nevertheless triggers indexing.
  Max Size of Index Bulk	Maximum number of bytes to be indexed at one time. The duration of indexing depends on the size of the documents. If documents are several MB in size, indexing takes a corresponding amount of time. This parameter therefore defines an upper limit for the data quantity. If the documents exceed this limit, the queue server distributes them accordingly.
  Optimize Bulk Size	Specifies the number of indexing intervals after which the queue server triggers optimizing. In the optimization phase, the index server rebuilds the index. It inserts new documents in the index, removes deleted objects from the index, and optimizes the index structure so that it can reply to search queries as quickly as possible. While optimization is running, the queue server does not trigger any more indexing. It waits until optimization is completed.
  Initial Indexing Mode	Specifies the situations in which the queue server triggers optimizing. This parameter influences the performances of the initial indexing of large data sets (100,000 documents or more). The following settings are possible: Off The queue server triggers optimization when one of the following situations arises: <Index Bulk Size> * <Optimize Bulk Size> documents have been indexed. The system has indexed all documents that were waiting when the start condition was reached. This setting is suitable if: Large document sets are to be indexed initially The initial indexing of large data sets is complete and you are moving to routine updates of the index On The queue server triggers optimization when it has indexed <Index Bulk Size> * <Optimize Bulk Size> documents. This setting is suitable if large document sets are to be indexed initially. This setting ensures that the index server only optimizes after a larger number of documents have been indexed. The setting Off may also cause small document sets to be optimized by the index server. Note Optimization of a larger quantity of documents generally runs quicker than optimization of numerous small quantities of documents. This is because the index server rewrites the index when optimization takes place. The larger the index, the longer the write process takes. If the index server carries out optimization frequently, it must also rewrite the index frequently. This can hamper performance. Note If you change the parameter from On to Off after the initial indexing run, the Flush function is triggered internally. This causes the queue server to process the remaining content of the queue in its entirety.

When changing the parameters, remember that if the queue has the status Indexing or Optimizing, the changes do not affect actions that are currently taking place. The changes only take effect when the queue server has completed the actions.