Validation Rule

Create rules and route records that pass through the 'pass' output port. Route failed records through the 'fail' output port.

You may want to create a validation rule to output a list of users who are the age of 18 or older, for example. You can create a rule that specifies the AGE column is >= (greater than or equal to) 18. Any values that are 18 or more are output through the 'pass' port, while those records that are less than 18 or have a null or invalid value are output through the 'fail' port.

You can view defined examples of this operator by clicking the Graphs tab and searching for "Basic Validation" and "IoT Validation".

Configuration Parameters

Parameter	Type	Description
Label	string
Rules	array	Required. Enter one or more rules. Click the Open EditorRequired. Enter the name of the validation rule operator that describes what the defined rule does. icon, and then click +Add item and complete the following options: Enter the column name. For example, 'AGE'. Enter the condition. For example, '>='. The following conditions are available: IS NULL IS NOT NULL < > Required. Enter the name of the validation rule operator that describes what the<= >= <> = LIKE MATCH PATTERN: Enter a pattern based on the match_pattern function. BETWEEN: Use to specify a range of values. IN SET CUSTOM: Select to call a more complex function that is defined in the Validation Functions option. Depending on the condition you selected, you may need to enter a value or a values list. For example, enter a value of 18. Enter a value list when you select BETWEEN or IN SET. Click +Add item to set the minimum value, for example, 9. Click +Add item again to set the maximum value, for example, 18. The options IS NULL and IS NOT NULL do not require a value. Enter the Fail Action for those records that do not pass the rule. The following options are available: Fail: Sends the record to the Fail port. Pass: Sends the record to the Pass port. Both: Sends the record to the Pass and Fail ports. Click Save.
Substitutions	array	If you choose Pass or Both for a rule's Fail Action, you can enter a substitution value for the failed values that go to the Pass port. For example, if you have a rule that the column AGE Is Not Null, you can create a substitution variable that assigns the AGE column with a value of 999. Therefore, the failed records are output to both the failed and passed ports. The failed records included in the Pass port have an age of 999, whereas the same records in the failed port continue to have a null value.
Validation Functions	array	If you want to create a more complex function, click the Open Editor icon, and then click +Add item and complete the following options: Enter the name of the function. For example, EighteenAndOlder. Enter the function in the Body option. For example, `if($arg >= 18) return 1; else return 0;.` This function reads: if the argument passed is greater than or equal to 18, return a 1 (true). If the value is less than 18, then return 0 (false). Click +Add item to create the arguments. Enter the name of the argument. For example, $arg. Define the data type of the argument: String, Number, or Integer. If you chose a String data type, then enter the Length. For example, 256. Click Save. In the Rules option, set the Condition to CUSTOM. In the Value option, call the name of the validation function. For example, EighteenAndOlder(AGE). Note that the 'AGE' in parentheses is the name of the input column name from your defined schema. Make sure that the input column's data type in the schema matches the data type defined in this argument. Click Save.
Input Schema	array	Required. Define the column names and data types for the input source used in the Validation Rule operator. Click the Open Editor icon, and then click +Add item for each input column. If you have FIRST_NAME, LAST_NAME, and AGE, you would create three values. Enter the name of the first column. Select the data type. If you chose a String data type, then enter the Length. For example, 256. Repeat for the additional columns, and then click Save.

Input

Input	Type	Description
`in`	string	The input port to the Validation Rule operator expects the data to be a string type.

Output

Output	Type	Description
`pass`	string	Contains the list of passed records. It shows the contents of each column. If the Failed Action indicator was sent to P or B (P=Pass, B=Both), and a substitution value was defined, the affected column will have the value updated with the defined substitution value.
`fail`	string	Contains the list of records that failed the rule or rules. Each record has the columns, the Failed Action indicator (B=Both, F=Fail), and which rules the record failed.
`failInformation`	string	Contains information about why the record failed. It contains a unique ID for each record, the name of the rule that failed, and the column name.
`outError`	string	When mapped, the outError port receives error messages for issues detected in the input stream. If the outError port is not mapped and errors are detected in the input stream, the graph processing stops.