File
The File Connector supports reading and writing files locally on the file system, via SMB (Server Message Block), FTP, and SFTP.
Connection Settings
Communication Protocol
Specifies the communication protocol used to access the files. The following options are supported.
| Option | Description |
|---|---|
| File Protocol | Used for local directory access. Local in this context means on the same machine the Intelligence Hub is running on. |
| SMB | Used to access shared network directories. |
| FTP | Access files on an FTP server. |
| SFTP | Access files over SFTP. |
Communication Protocol: SMB
The following connection settings are available for the SMB protocol.
| Setting | Description |
|---|---|
| Share | Specifies the server and share name of the access shared network directory.Ex. \ServerName\ShareName |
| Domain | Specifies the domain of the server used to access the shared network directory. |
| Username | Username required to authenticate with the share. Leave this blank for anonymous access. |
| Password | Password required to authenticate with the share. Leave this blank for anonymous access. |
Communication Protocol: FTP
The following connection settings are available for the FTP protocol.
| Setting | Description |
|---|---|
| Server Type | In most cases this should be set to Unix. In some cases, Windows specific FTP servers may need the Windows option selected. |
| Use Passive Mode | In most cases passive mode should be used. Active mode requires the FTP server to connect back to the FTP Client, and is more difficult to configure securely. |
| Host | Specifies the hostname of the FTP server. |
| Port | Specifies the port of the FTP server. |
| Username | Username required to authenticate with the share. Leave this blank for anonymous access. |
| Password | Password required to authenticate with the share. Leave this blank for anonymous access. |
Communication Protocol: SFTP
The following connection settings are available for the SFTP protocol.
| Setting | Description |
|---|---|
| Host | Specifies the hostname of the SFTP server. |
| Port | Specifies the port of the SFTP server. |
| Connect Timeout | The duration, in seconds, that the file connection will allow the SFTP host to respond during a connection attempt before aborting the attempt. |
| Session Timeout | The duration, in seconds, that the file connection will allow the SFTP host to respond before aborting the request. |
| Username | Username required to authenticate with the server. |
| Password | Password required to authenticate with the server. |
Common Connection Settings
The following settings are applicable to all protocols.
| Setting | Description |
|---|---|
| File Directory | Specifies the root directory for file processing. |
| Processed File Directory | Specifies the directory where files are moved to after processing successfully. This is only used by inputs that enable “Process File”. |
| Error Directory | Specifies the directory where files are moved to after processing unsuccessfully. This is only used by inputs that enable “Process File”. |
Browse
Input and Output Browsing
After the connection has been configured, the browse button can be used to navigate through the file system and create an Input or Output for any file or directory.
Input Settings
File Input Type
The File input type returns the content of a single file and has the following settings
File Search Type
How the file should be found. This can either be a direct search based on the full path of the file, or a regex search for a file name with an incomplete path.
Full Path Search Type
The full path search type allows a user to read a file when the full path to the file is already known. Due to the reduced searching involved, the full path search type will generally be more performant especially in large file systems. When a file is not found with this search type, an error is thrown.
File Path
The full path the a file in the file system, including its name.
Regex Search Type
The regex search type allows a user to find a file which matches a regex expression. It primary use is for reading a file or set of files when names, locations, or existence is unknown. When no file is found, nothing is returned and no error is thrown.
File Regex
The file name to read. This can be the full name of a single file or a regular expression. As an example, (.+) matches any file in the directory.
Relative Path
The relative path of the starting directory in regards to the connection level File Directory setting.
File Encoding
The character set used to decode the contents of the file.
Process File
When enabled, the file is moved to the connection’s processed directory after being read.
Include Metadata
When enabled, the contents of the file are returned alongside metadata describing the file. The read result contains the following attributes.
| Attribute | Description |
|---|---|
_filename | The name of the file, relative to the connection level File Directory setting. |
_fullPath | The full path to the file or directory located absolutely from the connection’s root directory. |
_creationTime | The time the file was created. Only included when the protocol and server report a creation time. |
_updateTime | The time the file was last updated. Only included when the protocol and server report an update time. |
_fileSize | The size of the file in bytes. |
value | The contents of the file, decoded using the configured File Encoding. |
When this setting is disabled, the read returns only the contents of the file.
Directory Input Type
The Directory input returns a list of the contents of a directory and has the following settings
Relative Path
The relative path of the starting directory in regards to the connection level File Directory setting.
Recursive
Designates whether the directory input should recursively explore sub directories.
Directory Contents
Each entry returned by the Directory input contains the following attributes.
| Attribute | Description |
|---|---|
name | The name of the file or directory. |
relativePath | The path of the entry relative to the connection level File Directory setting. For a file, this is the path of the directory that contains it. For a directory, this includes the directory’s own name. |
fullPath | The full path to the file or directory located absolutely from the connection’s root directory. |
creationTime | The time the entry was created. Only included when the protocol and server report a creation time. |
updateTime | The time the entry was last updated. Only included when the protocol and server report an update time. |
fileSize | The size of the file in bytes. Only included for files. |
type | Either File or Directory. |
Event Input Type
The Event input monitors a directory on the file system and produces an event each time an object within it is created, modified, or deleted. It is also a subscribable source for Event Triggers .
Note: The Event input type is only available on connections using the File Protocol. The SMB, FTP, and SFTP protocols do not support file system events.
Relative Path
The relative path of the directory to monitor in regards to the connection level File Directory setting. Leave this blank to monitor the File Directory itself.
Only the immediate contents of the monitored directory produce events. Changes made within a sub directory are not reported, though creating, modifying, or deleting the sub directory itself is.
When the path points to a file rather than a directory, the containing directory is monitored and only events matching that path are reported.
Event Type
The types of event which produce a result. At least one type must be selected. By default, all three are selected.
| Type | Description |
|---|---|
| Create | An object was added to the monitored directory. |
| Modify | An object in the monitored directory was changed. |
| Delete | An object was removed from the monitored directory. |
Object Search Type
Designates whether events are produced for files, directories, or both. At least one type must be selected. By default, only Files is selected.
Event Contents
Each event returned by the Event input contains the following attributes.
| Attribute | Description |
|---|---|
file | The name of the file or directory which triggered the event. |
eventType | The event which occurred. Either create, modify, or delete. |
path | The monitored path located absolutely from the connection’s root directory. |
Common settings
Time Filter
The time filter can be set to filter on either creation or update time, and will cause the input to return only files or directories which have been either created or updated after a provided ISO 8601 time.
Note: Time filter is dependent on both the protocol and server operating system. Protocols such as FTP and SFTP which don’t support creation time will not be able to be filtered on creation.
Output Settings
Output Type
Specifies the type of file operation to perform when the output is triggered.
| Type | Description |
|---|---|
| Write | Writes the payload to a file at the specified location. |
| Move | Moves a file from one location to another. |
| Delete | Deletes a file at the specified path. |
Write Output Type
File Name
Specifies the name of the file to write to. This includes the file
extension and directories. The file path can include dynamic outputs (e.g., /{{this.site}}/file.bin)
Relative Path
The relative path of the starting directory in regard to the connection level File Directory setting.
Payload Reference
When working with complex payloads, this setting uses dynamic outputs to specify the attribute that contains the file payload (e.g. {{this.filePayload}}).
Note: If left blank, the file connector will write the entire payload out to a file.
Base64 Decoding
When enabled, payloads are base64 decoded before being output as a file. This is used in when receiving Base64 encoded file payloads over other connections such as REST or MQTT.
Move Output Type
File Search Type
How the source file should be found. This can either be a direct search based on the full path of the file, or a regex search for a file name with an incomplete path.
Destination
The destination path the file is moved to. Supports dynamic outputs (e.g., /archive/{{this.site}}/).
Delete Output Type
File Path
The full path to the file to delete, relative to the connection’s File Directory setting.
Note: If the specified file does not exist, the delete operation completes successfully without error.