Pipeline
Pipelines move data between sources and targets while applying logic and transformations. For example, a pipeline can monitor an OPC UA tag for data changes. When a change occurs, it can read an Instance composed of data from the OPC UA server and publish it to MQTT on a unique topic.
Pipelines consist of Triggers and Stages, enabling them to process and manipulate data effectively.
Callable
When Callable is enabled, Pipelines can be called by Namespace Nodes, other Pipelines, and the REST Data Server. When called, a pipeline processes the incoming data and can optionally return a value using the Return Stage. Users can disable the callable setting to prevent the pipeline from being called.
Triggers
Triggers generate events that start the pipeline. Triggers can be event or time based. When data enters a pipeline, the data is called an event. All events enter the pipeline at the pipeline start stage.
Stages
Stages perform transformation and I/O using the events. Stages have an input and one or more outputs depending on the stage type. Example stages include writes, reads, modeling, buffering, etc.
Creating a pipeline
- Click Pipeline in the configuration’s Main Menu and then click New Pipeline to get started.
- Enter a Name to represent the pipeline. Optionally enter a Description, Tags, and Grouping.
- Choose to create an Empty Pipeline or choose Build Flow.
- Empty Pipeline creates a pipeline with no triggers or stages that must be manually configured.
- Build Flow continues the wizard and displays flow triggers settings to create a simple pipeline that reads and writes.
- Once the Pipeline is created, navigate to Stages in the top left of the Pipeline to view and drag and drop Trigger or other Stages onto the Pipeline.
Pipeline Status
The list view shows the name, enabled status, processing status, active features, and tags for each Pipeline.
The Name, Enabled, and Tags columns can be sorted by clicking on the column header.
All triggers and the Callable flag can be enabled/disabled on multiple pipelines at the same time by selecting them (using the checkboxes on the left) and then using an action from the Actions dropdown
| Column | Description |
|---|---|
| Name | The name of the pipeline or group |
| Enabled | This will show a play button if the pipeline has any enabled triggers, or if it is Callable. |
| Status | Status is based off of the last execution of the pipeline. Not Started - The pipeline has never run Good - The pipeline is executing successfully Error - An error occurred while processing Warning - The pipeline queue is growing faster than it is being processed. |
| Features | A blue downward pointing array will be shown if this pipeline is Callable. A warning icon will be shown if Track Activity is enabled. |
Pipeline Auto Layout
For large pipelines use the auto layout feature in the lower left hand corner of the pipeline canvas to automatically layout the stages.
Pipeline Stage Cloning
To clone a stage, click on the desired stage and press the clone button.
Pipelines are a powerful tool to move and transform data. See the links below to learn more about pipeline components, debugging, monitoring, and more.