Modeling Guide for SAP Data Hub

Copy File

The Copy File operator is used to copy files in a storage service. The expected input message is specified in the Input section in this document.

This operation is recursive, meaning it will copy any file/dir under the given source path.

If destination is a file, it will be overwritten; if it is a non-empty directory, it will fail.

  • Copying source: a/file1.txt to destination: newfile.txt, would succeed, since the destination does not exist.

  • Copying source: a/file1.txt to destination: b/f1.txt, would succeed and overwrite b/f1.txt, since the destination is an existing file.

  • Copying source: a/file1.txt to destination: b/, would fail, since b/ already exists and is not empty.

  • Copying source: a/ to destination :b/ would fail, since b/ already exists and is not empty.

  • Copying source: a/ to destination: b/dir/ would succeed, since b/dir/ does not exist.

Configuration Parameters

Parameter

Type

Description

service

string

The file service to operate. Additional parameters may depend on the selected service.

Default: "GCS"

timeoutInMs

int

Sets the time limit to execute the operation. If `0`, no timeout is used.

Default: 0

retryPeriodInMs

int

The time interval in milliseconds between connection trials.

Default: 0

numRetryAttempts

int

The number of times to retry a connection.

Default: 0

simultaneousRequests

int

The number of simultaneous requests generated on recursive calls (only available for GCS, S3 and WASB).

Default: 1

stopRequestOnError

boolean

Sets if simultaneous requests from recursive calls should stop at first error (only available for GCS, S3 and WASB).

Default: false

terminateOnError boolean A flag that indicates whether or not the graph should stop if this operator encounters an error. If set to false, an output may represent an error, in which case its attribute message.error will be true and its body will contain a string describing the error.

Default: true

connection object Holds information about connection information for the services. Each service connection parameters is documented separately:
configurationType string
connection parameter
Which type of connection information will be used: Manual (user input), or retrieved by the Connection Management Service.

Default: ""

connectionID string
connection parameter
The ID of the connection information to retrieve from the Connection Management Service.

Default: ""

connectionProperties object
connection parameter
All the connection properties for manual input.

Input

Input

Type

Description

in

message

A message whose body has the source file path and destination. Expected message attributes:
storage.src
Mandatory. Path to source file or directory.
storage.dst
Mandatory. Path to destination file or directory (should match types with source). The destination path may contain <dirname> and <basename>, referring to the given source path's dirname and basename. E.g: move src:data/A/user.csv dst:<dirname>/processedFiles/<basename> would map to the destination data/A/processedFiles/user.csv.
storage.srcBucket
Overrides bucket from configuration on service GCS and S3 for the source file or directory.
storage.dstBucket
Overrides bucket from configuration on service GCS and S3 for the destination file or directory.

Output

Output

Type

Description

out

message

A message whose header message.error (bool) reports whether the operation was successful. The body and any other header from the input message are copied to this one.