Configuration Recommendations
To achieve high-performance preprocessing that does not hamper the other TREX servers, use the following configuration.
Preprocessor hosts
- In accordance with the configuration rules that are specified inNumber of Preprocessors and Preprocessor Threads, start the required number of preprocessors on the preprocessor host.
- Monitor the load on the host during preprocessing. If system resources are still available, you can uses the pool size in the queue server to increase the preprocessor threads up to the maximum number recommended.
Master Host
We recommend that you keep the default configuration for the preprocessor on a master host.
If you give the preprocessor additional system resources, the performance of the queue server and index server suffers. Preprocessing will be faster, but subsequent processing steps will be slower.
Backup host
If the master index server and master queue server are active, there is little load on the backup hosts. If you want to use more load for preprocessing on a backup host, you can start more preprocessors on it, provided the hardware allows this (when doing this, note the configuration rules that are specified inNumber of Preprocessors and Preprocessor Threads.) This allows you to make better use of the system resources on the backup host. However, the performance of the indexing is not so high if either the backup index server or backup queue server is actually active.
Example 1
The following hosts preprocess documents:
- Master host:
2 CPUs - one preprocessor - one queue server
- Preprocessor hosts:
2 CPUs - two preprocessors
Both hosts have two CPUs. Because the preprocessor host only preprocesses documents, it should take more of the load than the master host. The preprocessor host therefore has two preprocessors. The only queue server in the system and a preprocessor are running on the master host. Therefore, a total of three preprocessors are running. Since each preprocessor process should process a maximum of three threads per CPU in parallel, you can calculate the maximum pool size as follows:
pool size = number of all preprocessors * 3 = 3 * 3 = 9
Thus the pool size must be set to the value 9 in the configuration file TREXQueueServer.ini on the master host. As a result, the queue server makes available a total of nine queue server clients for the preprocessors and in turn a total of nine threads are started in the preprocessors.
Example 2
The following hosts preprocess documents:
- Master host 1
2 CPUs - one preprocessor - one queue server
- Master host 2
2 CPUs - one preprocessor - one queue server
- Backup host
2 CPUs - one preprocessor - one queue server
- Preprocessor host
2 CPUs - two preprocessors
The preprocessor host therefore has 2 preprocessors, as in example 1, but no queue server. Since the system consists of two master hosts and one backup host, there are a total of three queue servers. We can assume that two of these three queue servers are always active: Either both master queue servers, or one master queue server and one backup queue server.
The pool size for two active queue servers is determined as follows:
pool size = number of all preprocessors * 3 = 5 * 3 = 15
This pool size divided by the number of active queue servers gives a pool size of 7 or 8 per queue server. This is the pool size that you enter in the configuration file TREXQueueServer.ini of all queue servers.