Parquet

Connection Settings

File Directory

Specifies the directory where files should be sourced for processing.

Processed File Directory

Specifies the directory where files should be moved to after processing successfully. This is only used for inputs that enable “Process File”.

Error Directory

Specifies the directory where files should be moved to after processing unsuccessfully. This is only used for inputs that enable “Process File”.

Input Settings

File Name

The file name to read. This can be the full name of a single file or a regular expression. As an example, (.*).parquet would match the first .parquet file in the directory based on alphabetical order.

Relative Path

The relative path of the starting directory in regard to the connection level File Directory setting.

Max Rows Per Read

The max number of rows in the file. Leave this blank if you just want to read to the end of the file. Set this if you only want to limit the rows to read.

Process File

When enabled, the file is moved to the connection’s processed directory after being read.

Output Settings

Base File Name

The base filename to write out. This field supports dynamic outputs to control the file name based on the output data. A timestamp and guid is also included with the file name to ensure uniqueness. The format of this string is yyyy-MM-dd-HH-mm-ss-guid.

Relative Path

The relative path of the starting directory with respect to the connection level File Directory setting.

File Duration (minutes)

Specifies how much data is in each file. The default duration is 1 minute. If set to 1 minute, each file contains one minute of data.