To achieve high-performance preprocessing that does not hamper the other TREX servers, use the following configuration.
Preprocessor hosts
Master Host
We recommend that you keep the default configuration for the preprocessor on a master host.
If you give the preprocessor additional system resources, the performance of the queue server and index server suffers. Preprocessing will be faster, but subsequent processing steps will be slower.
Backup host
If the master index server and master queue server are active, there is little load on the backup hosts. If you want to use more load for preprocessing on a backup host, you can start more preprocessors on it, provided the hardware allows this (when doing this, note the configuration rules that are specified inNumber of Preprocessors and Preprocessor Threads.) This allows you to make better use of the system resources on the backup host. However, the performance of the indexing is not so high if either the backup index server or backup queue server is actually active.
Example 1
The following hosts preprocess documents:
2 CPUs - one preprocessor - one queue server
2 CPUs - two preprocessors
Both hosts have two CPUs. Because the preprocessor host only preprocesses documents, it should take more of the load than the master host. The preprocessor host therefore has two preprocessors. The only queue server in the system and a preprocessor are running on the master host. Therefore, a total of three preprocessors are running. Since each preprocessor process should process a maximum of three threads per CPU in parallel, you can calculate the maximum pool size as follows:
pool size = number of all preprocessors * 3 = 3 * 3 = 9
Thus the pool size must be set to the value 9 in the configuration file TREXQueueServer.ini on the master host. As a result, the queue server makes available a total of nine queue server clients for the preprocessors and in turn a total of nine threads are started in the preprocessors.
Example 2
The following hosts preprocess documents:
2 CPUs - one preprocessor - one queue server
2 CPUs - one preprocessor - one queue server
2 CPUs - one preprocessor - one queue server
2 CPUs - two preprocessors
The preprocessor host therefore has 2 preprocessors, as in example 1, but no queue server. Since the system consists of two master hosts and one backup host, there are a total of three queue servers. We can assume that two of these three queue servers are always active: Either both master queue servers, or one master queue server and one backup queue server.
The pool size for two active queue servers is determined as follows:
pool size = number of all preprocessors * 3 = 5 * 3 = 15
This pool size divided by the number of active queue servers gives a pool size of 7 or 8 per queue server. This is the pool size that you enter in the configuration file TREXQueueServer.ini of all queue servers.