UMAP¶

class hana_ml.algorithms.pal.decomposition.UMAP(n_neighbors=None, min_dist=None, spread=None, n_components=None, distance_level=None, minkowski_power=None, knn_method=None, low_memory=None, n_epochs=None, init=None, eigen_tol=None, seed=None, learning_rate=None, optimization_parallel=None, calc_trustworthiness=None, distance_method=None, embedded_knn_method=None, max_neighbors_trustworthiness=None, thread_ratio=None)¶

Python wrapper for PAL UMAP

Parameters

n_neighborsint, optional

The size of local neighborhood (in terms of number of neighboring sample points) used for manifold approximation. Larger values result in more global views of the manifold, while smaller values result in more local data. In general values should be in the range 2 to 200.

Defaults to min(15,N-1), N is the number of data points.

min_distfloat, optional

The effective minimum distance between embedded points. Smaller values will result in a more clustered/clumped embedding where nearby points on the manifold are drawn closer together, while larger values will result in a more even dispersal of points. The value should be set relative to the spread value, which determines the scale at which embedded points will be spread out.

Defaults to 0.1.

spreadfloat, optional

The effective scale of embedded points. In combination with min_dist this determines how clustered/clumped the embedded points are.

Defaults to 1.0.

n_componentsint, optional

The dimension of the space to embed into. This defaults to 2, for visualization.

Defaults to 2.

distance_level{'manhattan', 'euclidean', 'minkowski', 'chebyshev', 'standardized_euclidean', 'cosine'}, optional

The distance level determines the distance metric used in the embedding space. The following distance levels are available:

'manhattan' : Manhattan distance
'euclidean' : Euclidean distance
'minkowski' : Minkowski distance
'chebyshev' : Chebyshev distance
'standardized_euclidean' : Standardized Euclidean distance
'cosine' : Cosine distance

Defaults to 'euclidean'.

minkowski_powerfloat, optional

The power parameter for the Minkowski distance metric. This is only used if distance_level is set to 'minkowski'.

Defaults to 3.0.

knn_method{'brute_force', 'matrix_enabled'}, optional

The method used to compute the k-nearest neighbors of the input data. The following methods are available:

'brute_force' : Brute Force searching
'matrix_enabled' : Matrix-enabled searching

Defaults to 'brute_force'.

low_memorybool, optional

In KNN searching, whether to keep pairwise distances. Keeping pairwise distances will consume a lot of memory, especially for the large data set, but will reduce the calculation of trustworthiness. - False : Keeps pairwise distances - True : Does not keep pairwise distances.

Defaults to False.

n_epochsint, optional

The number of training epochs to be used in optimizing the low-dimensional embedding. Larger values result in more accurate embeddings, but will take longer to compute.

Defaults to 200 is data size is larger than 10000. Otherwise, defaults to 500.

init{'random', 'spectral'}, optional

The initialisation method to use for the low-dimensional embedding. The following methods are available:

'random' : Random initialization
'spectral' : Spectral embedding initialization

Defaults to 'spectral'.

eigen_tolfloat, optional

Stopping criterion for eigendecomposition of the Laplacian matrix.

Defaults to 1e-10.

seedint, optional

Random seed.

0 : current time
other values : specified seed

Defaults to 0.

learning_ratefloat, optional

The initial learning rate for stochastic gradient descent. After the second iteration, the learning rate will decrease by LEARNING_RATE/N_EPOCHS after each iteration.

Defaults to 1.0.

optimization_parallelbool, optional

Whether to enable parallel optimization.

False : Disable parallel optimization.
True : Enable parallel optimization.

Defaults to False.

calc_trustworthinessbool, optional

Whether to calculate the trustworthiness of the embedding.

False : Do not calculate trustworthiness.
True : Calculate trustworthiness.

Defaults to True.

distance_method{'brute_force', 'matrix_enabled'}, optional

The method for calculating the distances in original high dimensional space when calculating trustworthness. The following methods are available:

'brute_force' : Use formula to calculate distances
'matrix_enabled' : Matrix-enabled calculation

Defaults to knn_method.

embedded_knn_method{'brute_force', 'matrix_enabled', 'kd_tree'}, optional

The method used to compute the k-nearest neighbors of the embedded data when calculating trustworthiness. The following methods are available:

'brute_force' : Brute Force searching
'matrix_enabled' : Matrix-enabled searching
'kd_tree' : KD-Tree searching

Defaults to 'brute_force'.

max_neighbors_trustworthinessint, optional

The maximum number of neighbors to consider when calculating trustworthiness.

Defaults to min(15, int(2(N+1)/3-1e-8)), N is the number of data points.

thread_ratiofloat, optional

Adjusts the percentage of available threads to use, from 0 to 1. A value of 0 indicates the use of a single thread, while 1 implies the use of all possible current threads. Values outside the range will be ignored and this function heuristically determines the number of threads to use.

Default to 1.0.

Attributes

result_DataFrame: Data with reduced dimensions.
statistics_DataFrame: Statistics.

Methods

`fit`(data[, key, features])	Fit UMAP model and reduce the dimension of data.
`fit_transform`(data[, key, features])	Fit UMAP model and reduce the dimension of data.
`transform`(data)	Reduce the dimension of data using the fitted UMAP model.

Examples

>>> umap = UMAP(n_neighbors=5, n_components=2,
                knn_method='brute_force', init='random', min_dist=0.1,
                distance_method='brute_force', embedded_knn_method='brute_force', seed=12345)
>>> res = umap.fit_transform(data=df, key='ID', features=['X1', 'X2', 'X3', 'X4', 'X5'])
>>> res.collect()
    ID  COMPONENT_1  COMPONENT_2 COMPONENT_3 COMPONENT_4 COMPONENT_5
0   1    -0.254690    14.510877        None        None        None
1   2    -1.088082    14.509798        None        None        None
2   3    -0.830867    15.124327        None        None        None
3   4    -1.210757    16.280894        None        None        None
4   5    -0.521130    16.062825        None        None        None
5   6    -0.363815    16.667533        None        None        None
6   7     2.553547    14.905408        None        None        None
7   8     1.955557    15.191549        None        None        None
8   9     1.953629    14.607795        None        None        None

fit(data, key=None, features=None)¶

Fit UMAP model and reduce the dimension of data.

Parameters

dataDataFrame

Input data.

keystr, optional

Name of the ID column in data.

If key is not provided, then:

if data is indexed by a single column, then key defaults to that index column;
otherwise, key` defaults to the first column;

featuresa list of str, optional

Names of the feature columns. If features is not provided, it defaults to all non-ID columns.

transform(data)¶

Reduce the dimension of data using the fitted UMAP model.

Parameters

dataDataFrame: Input data.

fit_transform(data, key=None, features=None)¶

Fit UMAP model and reduce the dimension of data.

Parameters

dataDataFrame

Input data.

keystr, optional

Name of the ID column in data.

If key is not provided, then:

if data is indexed by a single column, then key defaults to that index column;
otherwise, key` defaults to the first column;

featuresa list of str, optional

Names of the feature columns. If features is not provided, it defaults to all non-ID columns.

Returns

-------

DataFrame

Data with reduced dimensions.

Inherited Methods from PALBase¶

Besides those methods mentioned above, the UMAP class also inherits methods from PALBase class, please refer to PAL Base for more details.