Simplified Sizing Procedure

A simplified sizing calculation for pipelines may be possible in certain situations.

The simplified sizing could be considered if all of the following conditions are met:

  • The expected number of concurrent pipelines at peak times is relatively low, usually smaller than 20.

  • Each pipeline contains only one group and does not use multiplicity.

  • All operators process data in batches not bigger than 20 MB.

In this case, the resource requirements of pipelines will be smaller than those of the static components (implying that the error of our estimate will be less significant), and typically 1 GB of memory and 1 vCPU will be sufficient for each pipeline.

The simplified formulas to derive the resource requirements are then:
  • RAM 
    RAM_PIP (in MB) = Number of pipelines * 1 GB

  • CPU (in units of vCPUs)

    CPU_PIP = Number of pipelines