File/Hadoop JSON Output Adapter Studio Properties

Property ID: jsonColsMappingList

Type: string

(Required) Specify JSONPath expressions which represent JSON data to map to ESP streams. Separate these values using commas. There are two types of JSON data: array or object.

{
"firstName": "John",       
"lastName": "Smith", 
"phoneNumbers": [
        {
            "type": "home",
            "number": "212 555-1234"
        },
        {
            "type": "fax",
            "number": "646 555-4567"
        }
],
"friends": [
["female1","female2","female3"],
["male1","male2","male3"]
]
}

The first JSONPath expression is mapped to the first column of s stream, the second expression is mapped to the second column of a stream, and so on.

Property ID: jsonSecondDateFormat

Type: string

(Advanced) Specify the format for parsing seconddate values.

For example, yyyy-MM-dd'T'HH:mm:ss is the default value.

Property ID: jsonMsDateFormat

Type: string

(Advanced) Specify the format for parsing msdate values.

For example, yyyy-MM-dd'T'HH:mm:ss.SSS is the default value.

Property ID: jsonTimeFormat

Type: string

(Advanced) Specify the format for parsing time values.

For example, HH:mm:ss is the default value.

Property ID: jsonBigDatetimeFormat

Type: string

(Advanced) Specify the format for parsing bigdatetime values.

For example, yyyy-MM-dd'T'HH:mm:ss.SSSSSS is the default value. Using less than six Ss gives precision to that exact number of Ss and ignores values past that specification. Using more than six Ss truncates any values beyond the sixth, and replaces them with zero. This may result in slower behavior.

Property ID: charsetName

Type: string

(Advanced) Specify the name of a supported charset.

Type: string

(Required) Specify the absolute path to the data files that you want the adapter to write to. For example, <username>/<foldername>.

The default value is ".", meaning the current directory in which the adapter is running.

Alternatively, you can leave this value empty and specify the absolute path in the File property.

To use Hadoop system files, use an HDFS folder instead of a local file system folder. For example, hdfs://<hdfsserver>:9000/<foldername>/<subfoldername>/<leaffoldername>.

To use Hadoop, download the binaries for Hadoop version 1.2.1 or 2.2.0 from http://hadoop.apache.org. For version 1.2.1, copy the hadoop-core-1.2.1.jar file to %STREAMING_HOME%\adapters\framework\libj.

Use a forward slash for both UNIX and Windows paths.

Property ID: file

Type: string

(Required) Specify the file to write to.

Property ID: appendMode

Type: boolean

(Advanced) Specify whether data should be appended to an existing file.

Property ID: fileSizeLimit

Type: integer

(Advanced) Specify the maximum size, in bytes, of the output file. If this property is set, the adapter starts writing a new file every time the size of the current output file becomes greater than this value. The files are named <filename>, <filename>.001, <filename>.002, and so on where <filename> is the value of the File element.

If you do not specify a value or you specify a value of 0 for this property, the output file does not have a size limit.

No default value.

Property ID: timeBasedRotate

Type: boolean

(Advancedl) Specify whether to rotate files at predefined intervals.

The default value is false.

Property ID: timeBasedRotateInterval

Type: interval

(Advanced) Specify the amount of time, in seconds, to wait between file rotations.

The default value is 24 hours.

Property ID: timeBasedRotateStartAt

Type: time

If you do not specify a time zone, the adapter defaults to UTC.

Property ID: timestampInFilenames

Type: boolean

(Advanced) Specify whether to append the system time, in UTC, to the output file name when a new output file is created.

The default value is false.

Property ID: timestampInFilenamesFormat

Type: string

(Advanced) Specify the timestamp format that gets appended to the output file name. Valid formats include any formats accepted by the Java SimpleDateFormat class. For example, yyyy-MM-dd_HH-mm-ss.

On Windows, the following symbols are not permitted in the file name: \ / : * ? " < > |.

Property ID: BatchSize

Type: int

(Advanced) If the adapter is running in GD (guaranteed delivery) mode, specify the number of message rows after which the adapter issues a commit command to the external data source and a GD commit to the stream to which the adapter is attached.

If the adapter is runing without GD mode, specify the number of message rows after which the adapter issues a commit command to the external data source to which it is attached. The default value is 1.

Increasing this value improves performance at the expense of latency. It also increases memory consumption in the ESP Server because the uncommitted rows need to be preserved for redelivery in case of failure.

If running this adapter in GD mode, set either this or the BatchPeriod property to greater than 0. If neither property is set to greater than 0, a warning is sent and this property is set to 1.

Property ID: BatchPeriod

Type: int

(Advanced) Specify the number of seconds after which the adapter issues a commit command to the external data source and a GD commit to the stream to which the adapter is attached.

If the adapter is running without GD mode, specify the number of seconds after which the adapter issues a commit command to the external data source. The default value is 0.

PropertySet

Property ID: propertyset

Type: string

(Advanced) Specifies the name of the property set. Property sets are reusable sets of properties that are stored in the project configuration file. Using these sets allows you to move adapter configuration properties out of the CCL file and into the CCR file. If you specify the same properties in the project configuration file and the ATTACH ADAPTER statement, the values in the property set override the values defined in the ATTACH ADAPTER statement. No default value.

Property ID:outputBase

Type: boolean

(Optional) If set to true, the adapter outputs the initial stream contents in addition to stream updates.

If this option is enabled and the adapter is running in GD mode, once the adapter has done a GD commit on the entire base data, the ESP Server does not redeliver the base data on adapter restart and only sends deltas that are saved for delivery. The default value is false.

Property ID: onlyBase

Type: boolean

(Advanced) Sends a one-time snapshot of initial contents in a stream. Default value is false. If set to true, outputBase automatically becomes true.

Property ID: EnableGdMode

Type: boolean

(Advanced) Specifies whether the adapter runs in guaranteed delivery (GD) mode. GD ensures that data continues to be processed in the case that the ESP Server fails, or the destination (third-party server) fails or does not respond for a long time. See Guaranteed Delivery in the SAP Event Stream Processor: Developer Guidefor details on enabling GD for your project.

The default value is false.

Property ID: EnableGdCache

Type: boolean

(Advanced)If set to true, only rows that can be recovered (that is, checkpointed) by the ESP Server on restart are sent to the end source. Other rows are cached internally by the adapter.

The default value is false.

Property ID: GdSubscriberName

Type: string

(Advanced) If the adapter is running in GD mode (the EnableGDMode property is set to true), specify a unique name to identify the GD subscription client. If this value is empty when running in GD mode, the adapter logs an error and shuts down.