hanaml.GRUAttion is an R wrapper for PAL Gated Recurrent Units(GRU) based Encoder-Decoder Model with Attention Mechanism for time-series prediction.

hanaml.GRUAttention(
  data = NULL,
  key = NULL,
  endog = NULL,
  exog = NULL,
  learning.rate = NULL,
  batch.size = NULL,
  time.dim = NULL,
  hidden.dim = NULL,
  num.layers = NULL,
  max.iter = NULL,
  interval = NULL
)

Arguments

data

DataFrame
DataFrame containting the data.

key

character, optional
Name of the time stamp column that represents the order of values in the time-series.
The type of this column should be INTEGER.

endog

character, optional
The endogenous variable, i.e. time series.
Defaults to the first non-ID column.

exog

character or list of characters, optional
An optional array of exogenous variables.
Defaults all other columns in data exclusive of key and endog.

learning.rate

double, optional
Specifies the learning rate for gradient descent.
Defaults to 0.005.

batch.size

integer, optional
Number of pieces of data for training in one iteration.
Defaults to 32.

time.dim

integer, optional
Specifying how many time steps in a sequence that will be trained by attention and then for time series prediction.
This value must be smaller than the length of input time series minus 1.
Defaults to 16.

hidden.dim

integer, optional
Number of hidden neurons in every GRU layer.
Defaults to 64.

num.layers

integer, optional
Number of layers in GRU unit at encoder part and decoder part.
Defaults to 1.

max.iter

integer, optional
Maximum number of iterations, equivalent to maximum number of batches of data by which attention model is trained.
Defaults to 1000.

interval

integer, optional
Output the average loss within every interval iterations.
Defaults to 100.

Value

Returns a "GRUAttention" instance with the following attributes:

  • loss: DataFrame
    For storing the the average loss in every interval iterations.

  • model: DataFrame
    Fitted GRU+Attention model.

Details

Attention allows the recurrent network to focus on the relevant parts of the input sequence as needed, accessing all the past hidden states of the encoder, instead of just the last one. At each decoding step, the decoder gets to look at any particular state of the encoder and can selectively pick out specific elements from that sequence to produce the output.

Examples

Input DataFrame df:


> df$Head(3)$Collect()
   TIME_STAMP   TARGET
1           0    129.0
2           1    148.0
3           2    159.0

Create an LSTM instance:


> att <- GRUAttention(data = df,
                      key = "TIMESTAMP",
                      time.dim=6,
                      max.iter=1000,
                      learning.rate=0.005,
                      batch.size=8,
                      hidden.dim=64,
                      num.layers=2,
                      interval=50)

Peform predict on the fittd model:


> res <- predict(att, df.predict)

Expected output:


> res$Select(c("ID", "VALUE"))$Head(3)$Collect()
   ID      VALUE
1   0  11.673560
2   1  14.057195
3   2  15.119411