Skip to main content
dmahoney1-Visitor
1-Visitor
November 15, 2017
Question

Stream confusion

  • November 15, 2017
  • 1 reply
  • 2220 views

I'm running Thingworx 7.2.

Situation: I've got a greenhouse with 5 sensors recording temperature, humidity and light. I'm getting data and life is good.

Request: The greenhouse owner has asked forDaily High and Lows for each sensor over time

Maintaining the current Hi/Low is easy. I get and am testing it now.

To get the Daily Hi/Low, I assume that I need to do the following:

  • create a Data Shape with the 6 Hi/Low values
  • create a Stream to store that data
  • create a Scheduler that goes off at midnight
  • create a Thing to subscribe to that event and write a data shape to the stream
  • create a mashup that displays the stream

This may sound like I know what I'm doing, but..

1) Do I need a separate stream for each sensor?

2) Should my Sensor ThingTemplate subscribe to the event?

Any help is appreciated,

Dan

1 reply

ttielebein5-Regular Member
5-Regular Member
December 11, 2017

Hey Dan, the steps you outlined are exactly right. You have good intuition to set these values up in a stream instead of calculating them every time you need them (as some unfortunately attempt to do)! The stream should not be written to on the ThingTemplate level, which could in fact result in deadlocks and performance issues. Instead, it is better to loop through all the things once a day, outside of business hours, calculate the high and low, and then store this in the stream. All of the entries can go in the same stream, and you can retrieve them by device ID or name, however you would like. The question of whether or not you need a different stream per sensor will depend on how many data entries you think this stream will have.

If you are just storing the high/low values of each sensor per day for 1 year, that would be something like 365*5 = 1825 stream entries. This is very small, and any queries on this would be relatively quick. If the number of sensors goes up to say 20, though, and you choose to store this data for 3 years, then now it is something like 365*3*20 = 21,900. This is still a small enough number to not cause system performance issues, but queries on the stream for both the device ID and the sensor name will be significantly slower than you would like. This is because only the datetime fields of a stream are indexed in the database, and there is no option to add an additional index (though there is this option in a data table). Sending in start and end dates into queries for the data will help to mitigate this issue, as will making the "source" of the stream entry the name of the thing (as then this can be passed into the service outside of the query field as well). As a general rule, anything passed into a query or find or get service as a parameter is applied before the data is returned, and any queries passed in are applied after.

For your use case, I would say a stream is sufficient, but it is important to consider scaling when designing the application. Check out this KCS Article for more information about streams, and our full ThingWorx Best Practices Hub for even more related articles. Hope this helps!

Terms & ConditionsAccessibility statement