AWS IoT SiteWise

The AWS IoT SiteWise connector supports writing data to SiteWise. Models and Instances in the Intelligence Hub are converted to Models and Assets in SiteWise. The Connector supports browsing the SiteWise hierarchy and writing data in realtime using the SiteWise API as well as SiteWise Buffered Ingest using S3.

Connection Settings

Authentication Type

The type of authentication to use.

Region

Region of the AWS IoT SiteWise instance (e.g., us-east-1)

Buffered Ingest Settings

Buffered Ingest supports writing data to SiteWise through S3, which can be more cost-effective for large volumes of data that are not needed in real-time. The Buffered Ingest settings are used by Outputs that are set to Buffered mode. If Buffered Ingest is not being used, these settings can be ignored.

Input Settings

Inputs are not currently supported

Output Settings

Parent Asset

The parent asset to write to. This must be the full path to the parent (ex. boston/seaport/building1). Use Find to browse for the asset. If left blank the write occurs at the root of the SiteWise hierarchy. If the parent is not found the write fails.

Hierarchy Name

The name of the parent Asset’s hierarchy to write to. Use Find to browse for available hierarchies. As an example a “Region” model may have a hierarchy named “sites”. When writing a new site the parent asset would be “North America” and hierarchy would be “sites”.

Leave this blank when writing to the root. If the hierarchy name is not found and Create is enabled, the hierarchy is created on the parent Model. This field is only visible if the Parent Asset setting is not blank.

Create

When enabled Assets, Models, and Model hierarchy are created if they don’t already exist. Create behaves as follows.

• Look up the Model by name. If it doesn’t exist create a new Model
• If the Model is found, check to see if model definition has changed. If it has, update the model with the new attributes or hierarchy
• Look up the Asset using the full path (including parent hierarchy). If it doesn’t exist create a new Asset

Create is additive only. Renaming, moving, or deleting of Assets, Models, or Attributes is not supported.

Asset Name

The name of the Asset to write. If left blank the Asset name is taken from the output data (ex. instance name, or “_name” attribute in the payload).

Model Name

The name of the Model to write. If left blank the Model name is taken from the output data (ex. model name or “_model” attribute in the payload).

Timestamp

The timestamp used for VQTs in the write. This can be taken from the payload using a dynamic reference. By default the timestamp comes from the source data. Timestamps can be in epoch millisecond or ISO 8601 format

Write Path

The write path controls how data is written to SiteWise. It does not impact Model and Asset creation.

When set to Realtime, the BatchPutAssetPropertyValue API is used. The Output attempts to batch writes into 10x10 chunks, where each write contains 10 measurements (i.e. VQTs), each with 10 values. This results in the fastest possible ingest time for realtime data, but can result in higher ingest cost. Buffering in a pipeline may be required to take advantage of the chunking feature.

When set to Buffered, writes are sent to S3 in CSV format. Each write is a unique CSV file. The output then calls the SiteWise CreateBulkImportJob API to load the file from S3. This can be more efficient for large writes, and can result in sub-minute load times. In general this ingest method is more efficient for large amounts of data that’s not needed in realtime. SiteWise recommends 12,000 VQTs (i.e. data changes or rows in the CSV file) in a single file. This results in a file size of around 10MB.

Note careful consideration should be given to each write path type given data requirements. Using Buffered mode for small amounts of realtime data can be just as inefficient and costly as using Realtime mode for large amounts of non-realtime data.

A single Connection can support Outputs configured in both Realtime and Buffered mode. |

How Writes Work

The SiteWise Model and Asset information is taken from the payload. Below is an example write. This write could come from a raw input (ex. MQTT message), be generated manually, or be generated by the Models and Instances configured in the product. Regardless of the source, the SiteWise output translates the JSON below into SiteWise Assets and Models.

{
  "_name": "North_America",
  "_model": "Region",
  "regionCode": 123,
  "sites": [
    {
      "_name": "Portland",
      "_model": "Site",
      "address": "100 Congress Street"
    },
    {
      "_name": "Portsmouth",
      "_model": "Site",
      "address": "100 Main Street"
    }
  ]
}

There are two models in the above payload named Region and Site. Region has two Site Models linked by a “sites” label, and an attribute named “regionCode”. Site has a single attribute “address”. When this payload is written to an output the following happens.

1. The models Site and Region are looked up by name. If either doesn’t exist and create is off, the write fails. If they both exist and create is off, proceed to the next step. If create is on, the output creates the models if they don’t exist. If they do exist the output checks to see if the JSON schema is the same as the SiteWise model schema. If it’s different the output updates the models with new attributes or relationships. All attributes of the JSON that are not relationships (ex. sites is a relationship) are converted to Measurements in SiteWise. The output produces an error message “Building Models” and “Building Assets” until all Models and Assets in the JSON structure are built out.
2. The assets Portsmouth, Portland, and North_America are looked up by name. If create is off, and they don’t exist the write fails. If they do exist proceed to the next step. If create is on and they don’t exist they are created.
3. Once the Models and Assets exist in SiteWise, writes are issued for each attribute/measurement. Attributes are matched by name. For example, if the attribute “address” doesn’t exist on the Site model in SiteWise, it is skipped in the write. Writes for the entire JSON object are issue together.

Note that when create is enabled, all changes to Models and Assets are additive. The output will not delete or rename existing SiteWise Measurements or Hierarchy.

Asset and Model Cache

On the initial connection the SiteWise connector caches all Models and Assets in SiteWise. If Assets or Models are added by other systems after the connection, the user must re-save the connection reconnect and cache the state again.

AWS IAM Best Practices

Please see AWS documentation on IAM best practices. HighByte strongly recommends following the policy of least privilege when granting the IAM role for the connection.

https://docs.aws.amazon.com/iot-sitewise/latest/userguide/security-iam.html https://docs.aws.amazon.com/iot/latest/developerguide/security-best-practices.html

It is also recommended that users occasionally rotate new IAM credentials and manually update the Intelligence Hub configuration with the new credentials.

The following IAM permissions are used by the SiteWise Connection.

{
    "Version": "2012-10-17",
    "Statement": [
    {
          "Sid": "VisualEditor0",
          "Effect": "Allow",
          "Action": [
               "iotsitewise:CreateAsset",
               "iotsitewise:CreateAssetModel",
               "iotsitewise:ListAssets",
               "iotsitewise:ListAssetModels",
               "iotsitewise:ListAssociatedAssets",
               "iotsitewise:ListAssetRelationships",
               "iotsitewise:DescribeAssetModel",
               "iotsitewise:DescribeAsset",
               "iotsitewise:AssociateAssets",
               "iotsitewise:UpdateAssetModel",
               "iotsitewise:BatchPutAssetPropertyValue",
               "iotsitewise:CreateBulkImportJob",
               "iotsitewise:DescribeStorageConfiguration"
          ],
"Resource": "*"
    }
  ]
}

Buffered Ingest Setup

Buffered Ingest is not enabled by default in SiteWise. To enable Buffered Ingest you must do the following. See the SiteWise documentation for more details.

1. Login to the SiteWise console and go to Settings | Storage and enable “Activate Cold and Warm tier storage”
2. Setup an IAM role that has permission to write to SiteWise, the S3 Bucket, and the IAM:passRole permission
   • Below is an example of the passRole permission. This is required by the CreateBulkImportJob SiteWise API call

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "VisualEditor0",
         "Effect": "Allow",
         "Action": "iam:PassRole",
         "Resource": "*"
      }
   ]
}