Show TOC

Performance and Tuning Tips for the SAP HANA AdapterLocate this document in the navigation structure

Tips for improving the performance of the SAP HANA Output adapter.

  • Install Event Stream Processor on a different machine than your SAP HANA server. SAP does not support installing both servers on the same machine.
  • In HANA, use column-based tables instead of row-based tables.
  • Partitioning a column based table can improve the performance of multi threaded access to the table. Create partitions that your SAP HANA server can handle.
  • Using the threadCount property to adjust the number of threads used to load data in parallel can affect the throughput of the adapter. Start the value at the number of partitions on the table into which you are loading, and keep adjusting it up to approximately 5 times that number, or until performance no longer improves. Keep in mind that there is a limit of 2 billion rows per partition in SAP HANA. See your SAP HANA documentation for more details.
  • If you are running on UNIX, use unixODBC 2.3.O, as SAP HANA supports this version only.
  • Test optimum performance by setting different values for the bulkBatchSize, bulkInsertArraySize, idleBufferWriteDelayMSec, and bufferAgeLimitMSec properties.
  • Increasing bulkBatchSize increases throughput but also increases latency. Increase this number as needed based on what you require for latency.
    Note If you increase bulkBatchSize, also increase bulkInsertArraySize accordingly, as bulkInsertArraySize is a divisor of bulkBatchSize.
  • If a batch contains a bad row, the individual bad row is discarded, the bad rows statistic is incremented, and an error message is logged to the ESP Server log file.
Here are tips on scaling the ESP and SAP HANA system to handle high data rates from ESP into SAP HANA (assuming you have machines with a large number of CPUs and NUMA architecture):
  1. Create multiple ESP Server instances and have each instance host an SAP HANA adapter instance. Bind each ESP Server process to a separate NUMA node on your machines.
    • You can use the numactl –hardware command to figure out how many NUMA nodes are available and what ranges of CPU cores are available on each NUMA node.
    • Then, you can use the taskset –c <range> command to bind the ESP Server process to CPU cores that belong to only one NUMA node.
  2. Determine how to split your incoming data between your multiple ESP Servers.
    • If you use a splitter stream to split your incoming data, then also use stream binding to send that split data to projects running on separate ESP nodes.
    • Do not host everything on one ESP node because this will prohibit you from obtaining optimal performance from all of your NUMA nodes if they all run the same ESP process. Designate one NUMA node for each ESP process as described in step 1 above.
  3. Keep each ESP Server project as simple as possible to reduce the number of threads it creates and leave as many CPU cores available as possible for the SAP HANA adapter.
  4. Configure each SAP HANA adapter instance on the ESP Servers with the number of threads (threadCount parameter) that gives you the best performance.
    • This is likely in the range of 10 to 20.
    • Ensure that the bulkInsertArraySize parameter is set to a value that is big enough to give you good performance but not so big that latency suffers beyond an acceptable level. A value in the range of 100,000 to 1,000,000 is usually a good number for this parameter.
    • Increase the value for bufferAgeLimitMSec and idleBufferWriteDelayMSec timeouts to be at least 10,000 milliseconds unless you require the latency to be low.
  5. Use the most recent support package of SAP Event Stream Processor as performance has incrementally improved from prior 5.1 SP versions.