DiffusionProcess¶

class hana_ml.algorithms.pal.diffusion_process.DiffusionProcess(thread_ratio=None, step_num=None, duplicate_num=None, gbt_iter_num=None, gbt_max_depth=None, random_state=None)¶

Diffusion Process is a generative algorithm that learns a reverse process of gradually denoising data and then samples new records from the learned distribution.

Parameters

thread_ratiofloat, optional

Adjusts the percentage of available threads to use, from 0 to 1. A value of 0 indicates single-thread execution.

Defaults to PAL's internal default.

step_numint, optional

Number of diffusion steps used to simulate the input distribution.

Must be positive.

Defaults to 50.

duplicate_numint, optional

Number of duplicated samples used while adding noise during training.

Must be positive.

Defaults to 1.

gbt_iter_numint, optional

Number of iterations used by the internal HGBT model.

Must be positive.

Defaults to 10.

gbt_max_depthint, optional

Maximum depth used by the internal HGBT model.

Must be positive.

Defaults to 6.

random_stateint, optional

Seed for random number generation.

0: uses current time.
Others: uses the specified seed.

Defaults to 0.

Attributes

model_DataFrame: Trained diffusion process model.
stats_DataFrame: Statistics produced during training.
statistics_DataFrame: Alias of stats_.
result_DataFrame: Latest sampled result.
sample_stats_DataFrame: Statistics produced by the latest sample() call.

Methods

`fit`(data[, key, features, categorical_variable])	Train a diffusion process model on the input data.
`predict`([data, key, features, data_num, ...])	Alias of `sample()` for API consistency with other PAL wrappers.
`sample`([data, key, features, data_num, ...])	Sample new records from a trained diffusion process model.

Examples

>>> dp = DiffusionProcess(step_num=20, duplicate_num=3, random_state=1)
>>> dp.fit(data=df, key='ID', categorical_variable=['V3'])
>>> samples, sample_stats = dp.sample(data_num=30, sigma=0.2)
>>> samples.collect()

fit(data, key=None, features=None, categorical_variable=None)¶

Train a diffusion process model on the input data.

Parameters

dataDataFrame

Training data.

keystr, optional

Name of the ID column.

If specified, or if data.index is set to a single column, the first selected column is treated as the ID column and is excluded from training features.

Defaults to data.index when available.

featuresstr or a list of str, optional

Feature columns used for training.

Defaults to all non-key columns.

categorical_variablestr or a list of str, optional

Specifies which INTEGER feature columns should be treated as categorical.

STRING columns are treated as categorical by PAL automatically.

Returns

DiffusionProcess: Fitted instance.

sample(data=None, key=None, features=None, data_num=None, sigma=None, thread_ratio=None, random_state=None, model=None)¶

Sample new records from a trained diffusion process model.

Parameters

dataDataFrame, optional

Prototype table describing the output schema.

Its content can be empty. When not provided, the column layout used in fit() is reused.

keystr, optional

Name of the ID column in data.

Defaults to data.index when available, otherwise falls back to the key used in fit() when possible.

featuresstr or a list of str, optional

Feature columns in data.

Defaults to the columns used in fit() when possible, otherwise all non-key columns.

data_numint, optional

Number of samples to generate.

Defaults to PAL's internal default.

sigmafloat, optional

Noise level added during sampling.

Must be greater than or equal to 0.

Defaults to PAL's internal default.

thread_ratiofloat, optional

Adjusts the percentage of available threads to use, from 0 to 1.

Defaults to the value specified in the constructor.

random_stateint, optional

Seed for sampling.

Defaults to the value specified in the constructor.

modelDataFrame, optional

Diffusion process model.

Defaults to self.model_.

Returns

DataFrame: Sampled result.
DataFrame: Sampling statistics.

predict(data=None, key=None, features=None, data_num=None, sigma=None, thread_ratio=None, random_state=None, model=None)¶: Alias of sample() for API consistency with other PAL wrappers.

Inherited Methods from PALBase¶

Besides those methods mentioned above, the DiffusionProcess class also inherits methods from PALBase class, please refer to PAL Base for more details.