Move File

This operation is recursive, meaning it will move any files under the given source path. Interpretation of a path may differ on service as a directory (ADLS, file, HDFS, WebHDFS) or as a prefix (GCS, S3, WASB).

If destination is a file, it will be overwritten; if it is a non-empty directory, it will fail.

Supported services are:

In the given file structure:

.
|
+-- a
|   +-- file1.txt
|   +-- file2.txt
+-- b
    +-- f1.txt
    +-- f2.txt

Moving source: a/file1.txt to destination: newfile.txt, would succeed, since the destination does not exist.
Moving source: a/file1.txt to destination: b/f1.txt, would succeed and overwrite b/f1.txt, since the destination is an existing file.
Moving source: a/file1.txt to destination: b/, would fail, since b/ already exists and is not empty.
Moving source: a/ to destination:b/ would fail, since b/ already exists and is not empty.
Moving source: a/ to destination: b/dir/ would succeed, since b/dir/ does not exist.

Configuration Parameters

Parameter	Type	Description
service	string	The file service to operate. Additional parameters may depend on the selected service. Default: "file"
timeoutInMs	int	Sets the time limit to execute the operation. If `0`, no timeout is used. Default: 0
retryPeriodInMs	int	The time interval in milliseconds between connection trials. Default: 0
numRetryAttempts	int	The number of times to retry a connection. Default: 0
simultaneousRequests	int	The number of simultaneous requests generated on recursive calls (only available for GCS, S3 and WASB). Default: 1
stopRequestOnError	boolean	Sets if simultaneous requests from recursive calls should stop at first error (only available for GCS, S3 and WASB). Default: false
terminateOnError	boolean	A flag that indicates whether or not the graph should stop if this operator encounters an error. If set to `false`, an output may represent an error, in which case its attribute `message.error` will be `true` and its body will contain a string describing the error. Default: true
connection	object	Holds information about connection information for the services. Each service connection parameters is documented separately: Azure Data Lake (ADL) Google Cloud Storage (GCS) Hadoop File System (HDFS) Amazon S3 Microsoft Azure Blob Storage (WASB) WebHDFS
configurationType	string	connection parameter Which type of connection information will be used: Manual (user input), or retrieved by the Connection Management Service. Default: ""
connectionID	string	connection parameter The ID of the connection information to retrieve from the Connection Management Service. Default: ""
connectionProperties	object	connection parameter All the connection properties for manual input.

Input

Input	Type	Description
`in`	message	A message whose body has the source file path and destination. Expected message attributes: storage.src Mandatory. Path to source file or directory. storage.dst Mandatory. Path to destination file or directory (should match types with source). The destination path may contain `<dirname>` and `<basename>`, referring to the given source path's dirname and basename. E.g: `move src:data/A/user.csv dst:<dirname>/processedFiles/<basename>` would map to the destination `data/A/processedFiles/user.csv`. storage.srcBucket Overrides `bucket` from configuration on service GCS and S3 for the source file or directory. storage.dstBucket Overrides `bucket` from configuration on service GCS and S3 for the destination file or directory.

Output

Output	Type	Description
`out`	message	A message whose header `message.error` (bool) reports whether the operation was successful. The body and any other header from the input message are copied to this one.