Tips for improving the performance of the
SAP HANA Output adapter.
Here are tips on scaling the ESP and
SAP HANA system to handle high data rates from
ESP into SAP HANA
(assuming you have machines with a large number of CPUs and NUMA architecture):
- Create multiple ESP Server instances and
have each instance host an SAP HANA adapter
instance. Bind each ESP Server process to a
separate NUMA node on your machines.
- You can use the numactl –hardware command to
figure out how many NUMA nodes are available and what ranges of CPU
cores are available on each NUMA node.
- Then, you can use the taskset –c <range>
command to bind the ESP Server
process to CPU cores that belong to only one NUMA node.
- Determine how to split your incoming data between your multiple
ESP Servers.
- If you use a splitter stream to split your incoming data, then also
use stream binding to send that split data to projects running on
separate ESP nodes.
- Do not host everything on one ESP
node because this will prohibit you from obtaining optimal
performance from all of your NUMA nodes if they all run the same
ESP process. Designate one NUMA
node for each ESP process as
described in step 1 above.
- Keep each ESP Server project as simple as
possible to reduce the number of threads it creates and leave as many CPU
cores available as possible for the SAP HANA
adapter.
- Configure each SAP HANA adapter instance on
the ESP Servers with the number of threads
(threadCount parameter) that gives you the best performance.
- This is likely in the range of 10 to 20.
- Ensure that the bulkInsertArraySize parameter
is set to a value that is big enough to give you good performance
but not so big that latency suffers beyond an acceptable level. A
value in the range of 100,000 to 1,000,000 is usually a good number
for this parameter.
- Increase the value for bufferAgeLimitMSec and
idleBufferWriteDelayMSec timeouts to be at
least 10,000 milliseconds unless you require the latency to be
low.
- Use the most recent support package of SAP Event Stream Processor as
performance has incrementally improved from prior 5.1 SP versions.