Zip

The Zip stage creates a ZIP archive containing a single file with the compressed event value. This is useful for sending file data to cloud storage and can significantly reduce the size and time of transmission.

For example, common Pipeline patterns include buffering, formatting as CSV, and then GZip before sending to AWS S3 or Azure Blob storage.

The name of the file included in the archive, archive comment, and compression level can be configured.

File name

Specifies the name of the file in the Zip archive. This field supports in-line references to allow for the file name to be based on the information from the input event. For example, assuming the input event has a metadata key called filename with value process_data.csv, setting the File name to {{event.metadata.filename}} would name the file in the archive process_data.csv. The default value is file.

Archive comment

Specifies the archive comment added to the Zip archive that is produced. This optional field supports in-line references.

Compression Level

Specifies the compression level to use. Higher compression may result in smaller archive sizes but take more time to generate. When Compression Level is set to Default a balance between size and speed is used.