How-To Optimize a Solution to Send Real-Time Data to AWS IoT SiteWise
What Does This Article Cover?
Intelligence Hub includes a configurable Connection that allows Intelligence Hub solutions to write real-time data to AWS IoT SiteWise. AWS IoT SiteWise is a high-speed time-series data store and solutions can be created to send data from Intelligence Hub to AWS IoT SiteWise with low latency. This article provides recommendations for creating solutions that are performant. This article focuses on sending data to AWS IoT SiteWise and it assumes that an asset model has been created in SiteWise. This article covers the following topics.
- What is AWS IoT SiteWise
- AWS Service Quota
- Design Considerations
- Design Recommendations
- Other Related Material
What is AWS IoT SiteWise?
AWS IoT SiteWise is an Amazon Web Services managed service that enables manufacturers to collect, model, analyze, and visualize data from industrial equipment at scale. AWS IoT SiteWise provides an asset modeling framework that may be used to build representations of industrial devices, processes, and facilities. Intelligence Hub is used to create solutions that send modeled data payloads to AWS IoT SiteWise for storage in the time-series datastore.
AWS Service Quota:
AWS services have default quota limits. The AWS service quota that pertains to solutions that send real-time data from Intelligence Hub to AWS IoT SiteWise is named “Rate of data points ingested”. It is prudent to design solutions that limit data flowing from Intelligence Hub to AWS IoT SiteWise to a rate that is less than the service quote per Intelligence Hub AWS SiteWise Connection. However, there may be situations where this is not possible. In this case, a request can be made to change the service quote limit value. These requests are made per account and AWS region and they are reviewed by the IoT AWS SiteWise service team. To view the current value for the service quota, go to the AWS console, and open the profile drop-down in the upper right corner (make sure you are in the region you are using for SiteWise). Select “Service Quotas”. In the Manage Quotas box, select “AWS IoT SiteWise”. Select “View Quotas”. Scroll down to the bottom of the window and view the value for “Rate of data points ingested”. A change request can be made here if needed.
Design Considerations:
Most design considerations are use case specific however these general design considerations may be helpful.
What is the slowest data resolution that is acceptable? When OT systems monitor a process they do so at high speed. OT systems are also capability of exposing data at high frequency. However, an analytics or visualization solution may not needed to obtain data at the frequency that the OT system is capable of providing it.
What is the slowest level of data latency that is acceptable? It may be advantageous for performance reasons to buffer data in Intelligence Hub before sending an aggregated payload of data to AWS IoT SiteWise. For a given solution what is long acceptable period of time between each refresh of new data.
Must all values be sent to AWS IoT SiteWise each time that a payload is sent or only changed values? In a given modeled Intelligence Hub Instance some Attribute values may change more frequently than others. It might beneficial to send only the values that have changed to SiteWise.
Design Recommendations:
Consider the following recommendations when configuring an Intelligence Hub solution to send data to AWS IoT SiteWise. Generally, the best design approach is to use Intelligence Hub Pipelines, a Pipeline Size Buffer Stage to buffer data, optimize the buffer size based on the use case, scale using multiple Connections, and increase the AWS Service Quota if necessary.
Use Intelligence Hub Pipelines - By using a Pipeline, you can inspect the Pipeline queue from the Pipeline Statistics and learn if the Pipeline can keep up with the incoming data. Pipelines have a bounded queue size and a warning is created in the Log when the queue is full.
Use a Size Buffer Pipeline Stage - To achieve efficient data uploads to SiteWise utilize a Size Buffer Stage. The size of the buffer may be determined based on use case.
Consider multiple AWS IoT SiteWise Connections - Multiple AWS IoT SiteWise Connections may be configured in Intelligence Hub if the AWS Service Quote becomes an issue. In this scenario it is practical to designate a Connection per use case or Asset type.