AutomaticTimeSeries
- class hana_ml.algorithms.pal.auto_ml.AutomaticTimeSeries(scorings=None, generations=None, population_size=None, offspring_size=None, elite_number=None, min_layer=None, max_layer=None, mutation_rate=None, crossover_rate=None, random_seed=None, config_dict=None, progress_indicator_id=None, fold_num=None, resampling_method=None, max_eval_time_mins=None, early_stop=None, percentage=None, gap_num=None)
AutomaticTimeSeries offers an intelligent search amongst machine learning pipelines for time series tasks. Each machine learning pipeline contains several operators such as preprocessors, time series models and transformer that follows API of hana-ml algorithms.
For AutomaticTimeSeries parameter mappings of hana_ml and HANA PAL, please refer to the doc page: Parameter Mappings
- Parameters
- scoringsdict, optional
AutomaticTimeSeries supports multi-objective optimization with specified weights of each target. The goal is to maximize the target. Therefore, if you want to minimize the target, the weight of target needs to be negative.
The available target options are as follows:
EVAR: Explained Variance. Higher values indicate better performance. It is recommended to assign a positive weight to this metric.
MAE: Mean Absolute Error. Lower values indicate better performance. It is recommended to assign a negative weight to this metric.
MAPE: Mean Absolute Percentage Error. Lower values indicate better performance. It is recommended to assign a negative weight to this metric.
MAX_ERROR: The maximum absolute difference between the observed value and the expected value. Lower values indicate better performance. It is recommended to assign a negative weight to this metric.
MSE: Mean Squared Error. Lower values indicate better performance. It is recommended to assign a negative weight to this metric.
R2: R-squared. Higher values indicate better performance. It is recommended to assign a positive weight to this metric.
RMSE: Root Mean Squared Error. Lower values indicate better performance. It is recommended to assign a negative weight to this metric.
WMAPE: Weighted Mean Absolute Percentage Error. Lower values indicate better performance. It is recommended to assign a negative weight to this metric.
LAYERS: The number of operators. Lower values indicate better performance. It is recommended to assign a negative weight to this metric.
Defaults to {MAE":-1.0, "EVAR":1.0} (minimize MAE and maximize EVAR).
- generationsint, optional
The number of iterations of the pipeline optimization.
Defaults to 5.
- population_sizeint, optional
The number of individuals in each generation in genetic programming algorithm.
Defaults to 20.
- offspring_sizeint, optional
The number of offsprings to produce in each generation.
Defaults to the number of
population_size
.- elite_numberint, optional
The number of elite to output into result table.
Defaults to 1/4 of
population_size
.- min_layerint, optional
The minimum number of operators in a pipeline.
Defaults to 1.
- max_layerint, optional
The maximum number of operators in a pipeline.
Defaults to 5.
- mutation_ratefloat, optional
The mutation rate for the genetic programming algorithm.
Defaults to 0.9.
- crossover_ratefloat, optional
The crossover rate for the genetic programming algorithm.
Defaults to 0.1.
- random_seedint, optional
Specifies the seed for random number generator. Use system time if not provided.
No default value.
- config_dictstr or dict, optional
The customized configuration for the searching space. - {'light', 'default'}: use provided config_dict templates. - JSON format config_dict. It could be JSON string or dict. If it is None, the default config_dict will be used.
Defaults to None.
- progress_indicator_idstr, optional
Set the ID used to output monitoring information of the optimization progress.
No default value.
- fold_numint, optional
The number of fold in the cross validation process.
Defaults to 5.
- resampling_method{'rocv', 'block'}, optional
Specifies the resampling method for pipeline evaluation.
Defaults to 'rocv'.
- max_eval_time_minsfloat, optional
Time limit to evaluate a single pipeline. The unit is minute.
Defaults to 0.0 (there is no time limit).
- early_stopint, optional
Stop optimization progress when best pipeline is not updated for the give consecutive generations. 0 means there is no early stop.
Defaults to 5.
- percentagefloat, optional
Percentage between training data and test data. Only applicable when resampling_method is 'block'.
Defaults to 0.7.
- gap_numint, optional
Number of samples to exclude from the end of each train set before the test set.
Defaults to 0.
References
Under the given
config_dict
andscoring
, AutomaticTimeSeries uses genetic programming to to search for the best valid pipeline. Please see Genetic Optimization in AutoML for more details.Examples
Create an AutomaticTimeSeries instance:
>>> progress_id = "automl_{}".format(uuid.uuid1()) >>> auto_ts = AutomaticTimeSeries(generations=2, population_size=5, offspring_size=5, progress_indicator_id=progress_id) >>> auto_ts.enable_workload_class("MY_WORKLOAD_CLASS")
Invoke a PipelineProgressStatusMonitor instance:
>>> progress_status_monitor = PipelineProgressStatusMonitor(connection_context=dataframe.ConnectionContext(url, port, user, pwd), automatic_obj=auto_ts) >>> progress_status_monitor.start() >>> auto_ts.fit(data=df_ts, key='ID', endog="SERIES")
Output:
Show the best pipeline:
>>> print(auto_ts.best_pipeline_.collect()) ID PIPELINE 0 0 {"SingleExpSm":{"args":{"ALPHA":0.6,"PHI":0.3}...
Plot the best pipeline:
>>> BestPipelineReport(auto_ts).generate_notebook_iframe()
Make prediction:
>>> res = auto_ts.predict(data=df_predict)
If you want to use an existing pipeline to fit and predict:
>>> pipeline = auto_ts.best_pipeline_.collect().iat[0, 1] >>> auto_ts.fit(df_ts, pipeline=pipeline) >>> res = auto_ts.predict(df_predict)
- Attributes
- best_pipeline_: DataFrame
Best pipelines selected, structured as follows:
1st column: ID, type INTEGER, pipeline IDs.
2nd column: PIPELINE, type NVARCHAR, pipeline contents.
3rd column: SCORES, type NVARCHAR, scoring metrics for pipeline.
Available only when the
pipeline
parameter is not specified during the fitting process.- model_DataFrame or list of DataFrames
If pipeline is not None, structured as follows
1st column: ROW_INDEX.
2nd column: MODEL_CONTENT.
If auto-ml is enabled, structured as follows
1st DataFrame:
1st column: ROW_INDEX.
2nd column: MODEL_CONTENT.
2nd DataFrame: best_pipeline_
- info_DataFrame
Related info/statistics for AutomaticTimeSeries pipeline fitting, structured as follows:
1st column: STAT_NAME.
2nd column: STAT_VALUE.
Methods
Generate time series report.
cleanup_progress_log
(connection_context)Cleanup the progress log.
delete_config_dict
([operator_name, ...])Delete the content of the config dict.
Disables the mlflow autologging.
Disables the workload class check.
display_config_dict
([operator_name, category])Display the config dict.
display_progress_table
(connection_context)Return the progress table.
enable_mlflow_autologging
([schema, meta, ...])Enables the mlflow autologging.
evaluate
(data[, pipeline, key, endog, exog, ...])This function is to evaluate the pipeline.
fit
(data[, key, endog, exog, pipeline, ...])The fit function for AutomaticTimeSeries.
generate_html_report
([filename])Display function.
Display function.
Return the best pipeline.
get_workload_classes
(connection_context)Returns the available workload classes information.
make_future_dataframe
([data, key, periods])Create a new dataframe for time series prediction.
Persist the progress log.
pipeline_plot
([name, iframe_height])Pipeline plot.
predict
(data[, key, exog, model, show_explainer])Predict function for AutomaticTimeSeries.
reset_config_dict
([connection_context, ...])Reset config dict.
update_category_map
(connection_context)Update the list of operators.
update_config_dict
(operator_name[, ...])Update the config dict.
- fit(data, key=None, endog=None, exog=None, pipeline=None, categorical_variable=None, background_size=None, background_sampling_seed=None, model_table_name=None)
The fit function for AutomaticTimeSeries.
- Parameters
- dataDataFrame
The input time-series data for training.
- keystr, optional
Specifies the column that represents the ordering of time-series data.
If
data
is indexed by a single column, thenkey
defaults to that index column; otherwisekey
must be specified(i.e. is mandatory).- endogstr, optional
Specifies the endogenous variable for time-series data.
Defaults to the 1st non-key column of
data
- exogstr, optional
Specifies the exogenous variables for time-series data.
Defaults to all non-key, non-endog columns in
data
.- pipelinestr or dict, optional
Directly use the input pipeline to fit.
- categorical_variablestr or list of str, optional
Specify INTEGER column(s) that should be be treated as categorical data.
Other INTEGER columns will be treated as continuous.
- background_sizeint, optional
If set, the reason code procedure will be enabled.
Defaults to None.
- background_sampling_seedint, optional
Specifies the seed for random number generator in the background sampling. - 0: Uses the current time (in second) as seed - Others: Uses the specified value as seed
Defaults to 0.
- model_table_namestr, optional
Specifies the HANA model table name instead of the generated temporary table.
Defaults to None.
- Returns
- A fitted instance of class AutomaticTimeSeries.
- predict(data, key=None, exog=None, model=None, show_explainer=False)
Predict function for AutomaticTimeSeries.
- Parameters
- dataDataFrame
The input time-series data to be predicted.
- keystr, optional
Specifies the column that represents the ordering of the input time-series data.
If
data
is indexed by a single column, thenkey
defaults to that index column; otherwisekey
must be specified(i.e. is mandatory).- exogstr or list of str, optional
Names of the exogenous variables in
data
.Defaults to all non-key columns if not provided.
- modelDataFrame, optional
The model to be used for prediction.
Defaults to the fitted model(i.e. self.model_).
- show_explainerbool, optional
If True, the reason code will be returned. Only valid when background_size is provided during the fit process.
Defaults to False
- Returns
- DataFrame
Predicted result, structured as follows:
1st column: Data type and name same as the 1st column of
data
.2nd column: SCORE, predicted values.
- evaluate(data, pipeline=None, key=None, endog=None, exog=None, categorical_variable=None, resampling_method=None, fold_num=None, random_state=None, percentage=None, gap_num=None)
This function is to evaluate the pipeline.
- Parameters
- dataDataFrame
Data for pipeline evaluation.
- pipelinejson str or dict
Pipeline to be evaluated.
- keystr, optional
Specifies the column that represents the ordering of the input time-series data.
If
data
is indexed by a single column, thenkey
defaults to that index column; otherwisekey
must be specified(i.e. is mandatory).- endogstr, optional
Specifies the endogenous variable for time-series data.
Defaults to the 1st non-key column of
data
.- exogstr, optional
Specifies the exogenous variables for time-series data.
Defaults to all non-key, non-endog columns in
data
.- categorical_variablestr or list of str, optional
Specify INTEGER column(s) that should be be treated as categorical data. Other INTEGER columns will be treated as continuous.
Defaults to None.
- resampling_method{'rocv', 'block'}, optional
The resampling method for pipeline model evaluation.
Defaults to 'rocv'.
- fold_numint, optional
The fold number for cross validation.
Defaults to 5.
- random_stateint, optional
Specifies the seed for random number generator.
0: Uses the current time (in seconds) as the seed.
Others: Uses the specified value as the seed.
- percentagefloat, optional
Percentage between training data and test data. Only applicable when resampling_method is 'block'.
Defaults to 0.7.
- gap_numint, optional
Number of samples to exclude from the end of each train set before the test set.
Defaults to 0.
- Returns
- DataFrame
DataFrame of scores:
Score Name.
Score Value.
- reset_config_dict(connection_context=None, template_type='default')
Reset config dict.
- Parameters
- connection_contextConnectionContext, optional
If it is set, the default config dict will use the one stored in SAP HANA DB.
Defaults to None.
- template_type{'default', 'light'}, optional
HANA config dict type.
Defaults to 'default'.
- build_report()
Generate time series report.
- generate_html_report(filename=None)
Display function.
- generate_notebook_iframe_report()
Display function.
- cleanup_progress_log(connection_context)
Cleanup the progress log.
- Parameters
- connection_contextConnectionContext
The connection object to a SAP HANA database.
- delete_config_dict(operator_name=None, category=None, param_name=None)
Delete the content of the config dict.
- Parameters
- operator_namestr, optional
Deletes the operator based on the given name in the config dict.
Defaults to None.
- categorystr, optional
Deletes the whole given category in the config dict.
Defaults to None.
- param_namestr, optional
Deletes the parameter based on the given name once the operator name is provided.
Defaults to None.
- disable_mlflow_autologging()
Disables the mlflow autologging.
- disable_workload_class_check()
Disables the workload class check. Please note that the AutomaticClassification/AutomaticRegression/AutomaticTimeSeries may cause large resource. Without setting workload class, there's no resource restriction on the training process.
- display_config_dict(operator_name=None, category=None)
Display the config dict.
- Parameters
- operator_namestr, optional
Only displays the information on the given operator name.
Defaults to None.
- categorystr, optional
Only displays the information on the given category.
Defaults to None.
- display_progress_table(connection_context)
Return the progress table.
- Parameters
- connection_contextConnectionContext
The connection object to a SAP HANA database.
- Returns
- DataFrame
Progress table.
- enable_mlflow_autologging(schema=None, meta=None, is_exported=False, registered_model_name=None)
Enables the mlflow autologging.
- Parameters
- schemastr, optional
Defines the model storage schema for mlflow autologging.
Defaults to the current schema.
- metastr, optional
Defines the model storage meta table for mlflow autologging.
Defaults to 'HANAML_MLFLOW_MODEL_STORAGE'.
- is_exportedbool, optional
Determines whether export a HANA model to mlflow.
Defaults to False.
- registered_model_namestr, optional
MLFlow registered_model_name.
Defaults to None.
- property fit_hdbprocedure
Returns the generated hdbprocedure for fit.
- get_best_pipeline()
Return the best pipeline.
- get_workload_classes(connection_context)
Returns the available workload classes information.
- Parameters
- connection_contextstr, optional
The connection to SAP HANA.
- make_future_dataframe(data=None, key=None, periods=1)
Create a new dataframe for time series prediction.
- Parameters
- dataDataFrame, optional
The training data contains the index.
Defaults to the data used in the fit().
- keystr, optional
The index defined in the training data.
Defaults to the data.index or the first column of the data.
- periodsint, optional
The number of rows created in the predict dataframe.
Defaults to 1.
- Returns
- DataFrame
- persist_progress_log()
Persist the progress log.
- pipeline_plot(name='my_pipeline', iframe_height=450)
Pipeline plot.
- Parameters
- namestr, optional
The name of the pipeline plot.
Defaults to 'my_pipeline'.
- iframe_heightint, optional
The display height.
Defaults to 450.
- property predict_hdbprocedure
Returns the generated hdbprocedure for predict.
- update_category_map(connection_context)
Update the list of operators.
- Parameters
- connection_contextstr, optional
The connection to SAP HANA.
- update_config_dict(operator_name, param_name=None, param_config=None)
Update the config dict.
- Parameters
- operator_namestr
The name of operator.
- param_namestr, optional
The parameter name to be updated. If the parameter name doesn't exist in the config dict, it will create a new one.
Defaults to None.
- param_configany, optional
The parameter config value.
Defaults to None.
Inherited Methods from PALBase
Besides those methods mentioned above, the AutomaticTimeSeries class also inherits methods from PALBase class, please refer to PAL Base for more details.