Use the Model Compare component to identify the best performer from two algorithms to
solve a complex problem in all scenarios (SAP HANA and other). Add a child component to
perform further analysis.
Prerequisites: It is mandatory to use the
Model Statistics
and
Partition components with the
Model
Compare component to create your model comparison chain.
Take the
following steps to perform a two-component compare:
- In Expert Analytics, connect to a Data Source and navigate to the
Predict room.
- From the Component List choose the Data Preparation section.
- Drag-and-drop a Partition component to the analysis
editor. Alternatively, double-click the Partition
component. Click OK.
- In the Algorithms section, drag-and-drop selected algorithms to the analysis
editor. For example, if solving a classification problem, you might choose three
classification algorithms, Auto Classification,
R-CNR Tree, and Naïve Bayes.
- From the Data Preparation section, add Model Statistics components
for each chosen algorithm. This enables Expert Analytics to calculate
performance statistics on the dataset that the algorithms generate.
- Double-click the Model Statistics components to display
the configuration options. Alternatively, click the context menu icon on the
component and select Configure Settings. The result is a
configured chain that can perform the model comparison.
- Set the Target and Predicted
columns to compute performance statistics in both Model
Statistics components.
- From the Data Preparation section, add the Model Compare
component to the analysis editor.
- Drag-and-drop the Model Compare component that you have added to the
analysis editor over both of the Model Statistics
components that you want to compare. After which, the Model
Compare component is linked to all the components that you wish
to compare.
Note For a two-component compare, the
Model Compare component enables
you to add a child node, which the component indicates by displaying the
following icon:

- To start configuring the comparison, double-click the
Compare component to view its configuration settings.
Alternatively, on the component click the Settings
icon and from the context menu, select Configure
Settings.
- In the Model Compare dialog box, select a
Validate or Test partition to
compare the performance of various components connected to it.
Note The Model Compare component uses the
Validate setting by default to compare
models.
- In the Performance KPI (Key Performance Indicator) section, take any of the following
actions:
- Choose the KPIs for use and sort the order in which they should be
compared. Control over the order is important because if the top KPI
cannot identify a winning algorithm, the component can perform
calculations with the second KPI in the list, and so on.
- Click the arrows to move KPIs up or down in the comparison order. The
input components must be of the same type in the Model
Statistics component. If not, an error message displays.
- Specify the percentage for the Gain comparison. The percentage of the target population
must be between 1% and 100%, to one decimal point (for example, 15.3%).
Note Classification has 7 KPIs = KI, KR, KI + KR, AUC, S(KS), Gain % and Lift %,
whereas Regression has 9 KPIs = KI, KR, KI + KR, R2, L1, L2, LInf, ErrorMean
and ErrorStdDev.
- When you have completed the configuration, click Done.
- The analysis chain is now fully configured and ready to be executed. The summary of the
Model Statistics component shows the KPIs calculated
for all partitions. Titles are in the order set in the Model
Compare component and the Test partitions shows only when the
Model Compare component exists and Test was selected
for comparison.
Note
If all partitions are not available in the algorithm or the Model
Statistics component, the component considers it as
a chain without partition.
- Click the Run Analysis
icon.
Note
The component calculates performance results on every partition, but
only on the partition that you select does it identify a winner. The
result is the best component of those compared only. It is advised
that you ensure that the KI values are greater than 95% before
deploying the component in production.
- The Results tab shows the Summary of the comparison results, and highlights the
best component. The feedback includes the following information:
- A star icon
indicates the best
performing component. This is based on the comparison of performance
statistics for the algorithms, which can be either classification or
regression types. The Summary sorts the model algorithms in order of
performance. It compares the results based on the partition selected,
which can be either Test or Validate.
- Titles are in the order set in the Model Compare
component, with the bolded titles indicating those chosen for
comparison.
- In the case of a classification algorithm, the Profit or the Lift
settings will default to 10% if you have not specified a
percentage.
- Optionally, when you are using two parent components, you can extend the analysis by adding
a child node with a mapping screen to the Model Compare
component. To do so, right-click Model Compare and select
Configure Settings. Alternatively, double-click the
Model Compare component or press F5. After which, a
default mapping occurs that is based on column name and type.
- Optionally, name the columns that result from the mapping for the child component to use.
You can add or remove other columns of the same type. To map all other columns,
manually add the additional rows.
Note
The data in the mapped columns comes from the winning component. None
of the columns in the configuration window can be empty.
- Optionally, you can export the best model as a stored procedure for
consumption. To do so, in the Model Compare component
click the Settings
icon and from the resulting
context menu, select Export as Stored Procedure.
- Optionally, you can save and export the best chain directly from the
Model Compare component. To do so, in
Model Compare click the
Settings
icon and from the resulting context menu, select Save as
Model.
You can now use the Partition, Model Statistics and Model Comparison components in unison
to compare multiple algorithms to find the best one to use in a complex analysis.