Triggering a Property 'DataChange' subscription when using AddNumberValueStreamEntry

[pschmaltz]

In order to load some historic data into a logged property I've written a data loader service that uses Things[].AddNumberValueStreamEntry() and its variants (depending on the data type of the property). This lets me set the timestamp to a value in the past, and so far it seems to work well.

However I find that my subscriptions to 'DataChange' for these properties are not fired when I do this. Is that expected behavior when using entity services like AddNumberValueStreamEntry?

The same 'DataChange' subscription works just fine if I merely 'set' the property value of course, but that can only be used to set the property value entry for the current time.

I'd like to be able to load some property value stream data and also generate other updates triggered off of data changes to those properties. Is that possible?

[CarlesColl]

You have to trigger by yourself, instead of using AddNumberValueStreamEntry wrap it inside a custom service ( for instance AddNumberValueStreamEntryCustom ) and once you added the value trigger the DataChangeEvent:

  me.DataChange({

                    newValue: "infotable-with-new-value-inserted-on-the-stream' ,

                    oldValue: undefined

                });

[pschmaltz]

Thanks for the pointer Carles - it hadn't occurred to me that I might trigger my own datachange event. Follow up question though - I must not quite be using this correctly as it is throwing a warning in the application log. Specifically the log says: Unable to publish event in MyThingName : Event DataChange not found

Any explanation for why my attempt to call a DataChange is rebuffed as not found?

Perhaps the way I call DataChange is at fault. The newValue infotable parameter I'm using employs the namedVTQ datashape (seemed reasonable choice and I don't see any errors arising from that, but it wasn't clear if a particular datashape is needed). A snipped of my test service for this is below (I pass in a 'thing' as an input parameter, and then create a value stream entry not shown below for brevity):

Things[thing].AddNumberValueStreamEntry(aparams); // This works fine. I get my value stream entry for the date I want etc.

var params = {

  infoTableName : "InfoTable",

  dataShapeName : "NamedVTQ"

};

var namedVTQ = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params);

// NamedVTQ entry object - used this since I wasn't entirely sure what shape the DataChange expects.

var newEntry = new Object();

newEntry.name = "TEMP"; // STRING - isPrimaryKey = true

newEntry.time = now; // DATETIME

newEntry.value = "98.6"; // VARIANT

newEntry.quality = "GOOD"; // STRING

namedVTQ.AddRow(newEntry); // Logging this out shows a valid data row in the infotable

var params = {

  newValue: namedVTQ /* INFOTABLE */,

  oldValue: undefined /* INFOTABLE */

};

Things[thing].DataChange(params); // This would appear to be throwing that Warning in the application log.

TIA for any other suggestions you may have. As a workaround, I've augmented my service that creates the historical entry to the value stream to also write the additional values I was otherwise going to derive by my subscription to the datachange event. But it ends up being duplicate code so I'd rather not do that. And mostly I just want to get a handle on correctly using the DataChange event in a service for my own future reference!

[CarlesColl]

I see,

Actually I'm using AnyDataChange, not DataChange, I've done some tests and yes it throws the warning you mentioned. If you can change it to AnyDataChange will work.

By the way, it will be good to know how to throw DataChange event from code, any TW guy can answer to it?

Best Regards,

Carles.

[paic]

DataChange is based on the actual property value changing/updating.

It's a bit against best practice to trigger this manually, without changing the actual value.

If you already have a Subscription to datachange, move the code into a service, trigger that both from this addvaluestream code and from the subscription itself.

[CarlesColl]

Hi Pai,

This a limited view of it, really there's plenty of situations where you want to simulate a DataChange Event, when you deal with historical data that's so common. Actually TW it's so limited to deal with the combination of Real Time and Historical data at the same time, that's one of the reasons we need to simulate DataChange events.

Best Regards,

Carles.

[paic]

The Edge MicroServer does come with a method to submit a set of data with original timestamps intact and each can be processed as a datachange as they are written to the Property.

So that might be a service to consider, there probably is an equivalent of some sort within Thingworx is what I'm thinking.

[CarlesColl]

Well it's more complex than that, maybe you already have live data on the property, and you want to load historical data, and still you want that all the events are triggered... And data doesn't comes from Edge, for instance it comes from csv files that other systems generate.

Carles.

[paic]

I certainly am not saying it is trivial, but I think that if you take all the different available options together, you can probably come up with a satisfactory solution.

Without knowing the exact use cases, it is difficult to give an exact solution.

[CarlesColl]

Yes I have the solution, but the solution comes with a lot of pain and custom code...

[pschmaltz]

In lieu of using Things[thingName].AddNumberValueStreamEntry(params) to enter values with my specific time stamp, I linvestigated using  Things[thingName].UpdatePropertyValues(params). Some other discussions on this community were helpful in wrestling with how to dynamically create the infotable that params for this service requires.

Along the way, I was pleasantly surprised to find that if I included a 'time' value in as one of the params infotable fields, not only could I specify a value stream entry timestamp in the past, it also triggers the DataChange for the property (which I wasn't getting when I used AddNumberValueStreamEntry).

I'm not entirely sure what the pros and cons are of using UpdatePropertyValues vs. Add(type)ValueStreamEntry, but perhaps one distinction would be whether you want to trigger a DataChange event for what might be a historical value.

A snipped from my testing of this is below, for others reference:

var myUpdate = Resources["InfoTableFunctions"].CreateInfoTable({infoTableName: "MyUpdateTable"});

// For some reason, creating the dataShape in advance doesn't work for me (as others also noted on a separate discussion)

// Creating the fields on the fly

myUpdate.AddField({'name': 'name', 'baseType': 'STRING'}); 

myUpdate.AddField({'name': 'value', 'baseType': 'NUMBER'});

myUpdate.AddField({'name': 'time', 'baseType': 'DATETIME'});

myUpdate.AddRow({'name': attrname, 'value': attrvalue, 'time': inputTime});  // attrname, attrvalue, and inputTime are inputs to the svc.

Things[thingName].UpdatePropertyValues({values: myUpdate});