dtw

hana_ml.algorithms.pal.tsa.dtw.dtw(query_data, ref_data, radius=None, thread_ratio=None, distance_method=None, minkowski_power=None, alignment_method=None, step_pattern=None, save_alignment=None)

DTW is an abbreviation for Dynamic Time Warping. It is a method for calculating distance or similarity between two time series. It makes one series match the other one as much as possible by stretching or compressing one or both two.

Note that this function is a new function in SAP HANA Cloud QRC01/2021.

Parameters

query_dataDataFrame

Query data for DTW, expected to be structured as follows:

• 1st column : ID of query time-series, type INTEGER, VARCHAR or NVARCHAR.

• 2nd column : Order(timestamps) of query time-series, type INTEGER, VARCHAR or NVARCHAR.

• Other columns : Series data, type INTEGER, DOUBLE or DECIMAL.

ref_dataDataFrame

Reference data for DTW, expected to be structured as follows:

• 1st column : ID of reference time-series, type INTEGER, VARCHAR or NVARCHAR

• 2nd column : Order(timestamps) of reference time-series, type INTEGER, VARCHAR or NVARCHAR

• Other columns : Series data, type INTEGER, DOUBLE or DECIMAL, must have the same cardinality(i.e. number of columns) as that of data.

radiusint, optional

Specifies a constraint to restrict match curve in an area near diagonal.

To be specific, it makes sure that the absolute difference for each pair of subscripts in the match curve is no greater than radius.

-1 means no such constraint, otherwise radius must be nonnegative.

Defaults to -1.

thread_ratiofloat, optional

Controls the proportion of available threads to use. The ratio of available threads.

• 0 : single thread.

• 0~1 : percentage.

• Others : heuristically determined.

Defaults to -1.

distance_method{'manhattan', 'euclidean', 'minkowski', 'chebyshev', 'cosine'}, optional

Specifies the method to compute the distance between two points.

• 'manhattan' : Manhattan distance

• 'euclidean' : Euclidean distance

• 'minkowski' : Minkowski distance

• 'chebyshev' : Chebyshev distance

• 'cosine' : Cosine distance

Defaults to 'euclidean'.

minkowski_powerdouble, optional

Specifies the power of the Minkowski distance method.

Only valid when distance_method is 'minkowski'.

Defaults to 3.

alignment_method{'closed', 'open_begin', 'open_end', 'open'}

Specifies the alignment constraint w.r.t. beginning and end points in reference time-series.

• 'closed' : Both beginning and end points must be aligned.

• 'open_end' : Only beginning point needs to be aligned.

• 'open_begin': Only end point needs to be aligned.

• 'open': Neither beginning nor end point need to be aligned.

Defaults to 'closed'.

step_patternint or ListOfTuples

Specifies the type of step patterns for DTW algorithm.

There are five predefined types of step patterns, ranging from 1 to 5.

Users can also specify custom defined step patterns by providing a list tuples.

Defaults to 3.

Note

A custom defined step pattern is represented either by a single triad or a tuple of consecutive triads, where each triad is in the form of \((\Delta x, \Delta y, \omega)\) with \(\Delta x\) being the increment in query data index, \(\Delta y\) being the increment in reference data index, and \(\omega\) being the weight.

A custom defined step pattern type is simply a list of steps patterns.

For example, the predefined step patterns of type 5 can also be specified via custom defined step pattern type as follows:

[((1,1,1), (1,0,1)), (1,1,1), ((1,1,0.5), (0,1,0.5))].

For more details on step patterns, one may go to PAL DTW for reference.

save_alignmentbool, optional

Specifies whether to output alignment information or not.

• True : Output the alignment information.

• False : Do not output the alignment information.

Defaults to False.

Returns

DataFrames

• Result for DTW, structured as follows:

  • QUERY_<ID column name of query data table> : ID of the query time-series.

  • REF_<ID column name of reference data table> : ID of the reference time-series.

  • DISTANCE : DTW distance of the two series. NULL if there is no valid result.

  • WEIGHT : Total weight of match.

  • AVG_DISTANCE : Normalized distance of two time-series. NULL if WEIGHT is near 0.

• Alignment information table, structured as follows:

  • QUERY_<ID column name of query data table> : ID of query time-series.

  • REF_<ID column name of input table> : ID of reference time-series.

  • QUERY_INDEX : Corresponding to index of query time-series.

  • REF_INDEX : Corresponding to index of reference time-series.

• Statistics for DTW, structured as follows:

  • STAT_NAME: Statistics name.

  • STAT_VALUE: Statistics value.

Examples

Query data:

>>> data1.collect()
   ID  TIMESTAMP  ATTR1  ATTR2
0   1          1      1    5.2
1   1          2      2    5.1
2   1          3      3    2.0
3   1          4      4    0.3
4   1          5      5    1.2
5   1          6      6    7.7
6   1          7      7    0.0
7   1          8      8    1.1
8   1          9      9    3.2
9   1         10     10    2.3
10  2          1      7    2.0
11  2          2      6    1.4
12  2          3      1    0.9
13  2          4      3    1.2
14  2          5      2   10.2
15  2          6      5    2.3
16  2          7      4    4.5
17  2          8      3    4.6
18  2          9      3    3.5

Reference data:

>>> data2.collect()
   ID  TIMESTAMP  ATTR1  ATTR2
0   3          1     10    1.0
1   3          2      5    2.0
2   3          3      2    3.0
3   3          4      8    1.4
4   3          5      1   10.8
5   3          6      5    7.7
6   3          7      5    6.3
7   3          8     12    2.4
8   3          9     20    9.4
9   3         10      4    0.5
10  3         11      6    2.2

Call dtw():

>>> res, align, stats = dtw(data1, data2,
...                         step_pattern=[((1,1,1),(1,0,1)), (1,1,1), ((1,1,0.5),(0,1,0.5))],
...                         save_alignment=True)
>>> res.collect()
  QUERY_ID REF_ID   DISTANCE  WEIGHT  AVG_DISTANCE
0        1      3  48.027427    10.0      4.802743
1        2      3  36.933929     9.0      4.103770