ThingWorx Agent - frequent property update with storage in Stream
Hello,
I am wondering what is the best approach to take when I have an TWX Edge which is pushing properties update (~60-70 properties) every 25ms and I want to store every single property data set (i.e. in Stream). I have an remote event which is triggered by TWX Edge when all properties are updated and I am subscribing to it, so that whenever all properties are updated I am creating JSON with all properties and persist it in a stream:
var properties = me.GetPropertyValues();
var params = {
table: properties /* INFOTABLE */
};
// result: JSON
var json = Resources["InfoTableFunctions"].ToJSON(params);
var values = Things["RawStream"].CreateValues();
values.thing= me.name; //THINGNAME
values.properties = json; //JSON
var params = {
sourceType: undefined /* STRING */,
values: values /* INFOTABLE*/,
location: undefined /* LOCATION */,
source: me.name /* STRING */,
timestamp: undefined /* DATETIME */,
tags: undefined /* TAGS */
};
// no return
Things["RawStream"].AddStreamEntry(params);
However, I am wondering what if TWX will not yet manage to store Stream entry (execute full subscription, especially execute me.GetPropertyValues()) and in the meantime Edge will update properties with new values (which will trigger yet another subscription thread) - so there is a risk of data loss (as 'first' subscription will retrieve updated values of properties)? I want to be 100% that I do not lose any data at all.
Do you have any other ideas (not necessarily using Streams) which I can use to persist high volume of data coming from Edge with high frequency (to multiple (>1000) devices simultaneously)?
Thanks in advance for any ideas.
Regards,
Jacek

