How to insert past-data to a Thing?

[mbereket]

Dear TW community members!

So, this time we are working on "Local Data Storage" feature for our devices when the internet connection is not available.

Here is a quick back-ground info:

- We have smart sensors which updates thingworx every minute to post temperature,pressure, humidity etc.

- We store all these information using "Value Streams" for each device.

- and then we use "QueryNumberPropertyHistory" service to retrieve history of data to display meaningful graphs and other info

When there is no internet connection, our devices skip sending information and whenever the connection is back, devices will start re-updating ThingWorx again.

Now, we have managed to store information locally within devices even when there is no internet.

However, I am curious, what would be the best way to update thingworx and insert all of those locally stored information in one single update? and I still want to be able to see meaningful (not collapsed) [temperature - timestamp] data pairs when I use "QueryNumberPropertyHistory" service on Things. (Generic Services)

Thanks

[CarlesColl]

If you use standard TW subscriptions it will preserve timestamps if you send it on property updates.

If you use you own services to send data back to the platform, you can leverage "Add"+baseType+"ValueStreamEntry" service in order to add historical data ( but if you have subscriptions that depend on this past time DataChange event it won't be thrown ).

Just my two cents, I don't know if it helps.

[ankigupta]

Hi mustafa bereket​,

As per my understanding you can create your own custom service which takes infotable as input parameter. Then iterate through all the rows and use UpdatePropertyValues service to update values in system. UpdatePropertyValues service takes timestamp as one of the input so we can use the same timestamp when the property was generated and make it look like it was inserted without any delay.

I hope it helps.

[ankigupta]

For more information on how to use UpdatePropertyValues service; please check following blog: How to use UpdatePropertyValues Service in ThingWorx

[mbereket]

Hi Carles Coll and Ankit Gupta,

Thanks for your help, I was curious to see if there is anyway of updating a property with a specific timestamp data.

Like, I have a "Thing", that thing has "temp" attribute,

When I only update [MyThing][temp] in real time, everything is perfect like following;

operation 1: [MyThing][temp] = 25;    

operation 2: [MyThing][temp] = 26;

operation 3: [MyThing][temp] = 27;

operation 4: [MyThing][temp] = 28;

But then, I receive another update package stating it is for the past (offline- data collected during internet outage) , then I will need to stick following data, before operation 1

operation -3: [MyThing][temp] = 19;

operation -2: [MyThing][temp] = 11;

operation -1: [MyThing][temp] = 20;   

operation  0: [MyThing][temp] = 16;

I am not sure if it is possible but if I somehow can manage to do that just by using ThingWorx generic services, that would be great.

Because this way, I can just use "QueryNumberPropertyHistory" service, and keep generating my reports.

PS: I can definitely provide timestamps for offline data stream.

[ankigupta · Accepted solution]

mustafa bereket​,

UpdatePropertyValues is the generic service which also takes timestamp as input.

[mbereket]

Hi Ankit,

I did not notice that, I am trying it out now.

Thanks,

[mbereket]

Thank you  Ankit Gupta​

This service worked perfectly and solved my problem.

[ankigupta]

mustafa bereket​, I am Glad it helped!

[nagrawal-2]

Hello Ankit,

what format should be the timestamp to update the values in ValueStream ? I am reading data from CSV file and storing it in Valuestrem using custom timestamp but seems like my data got uploaded with wrong dates. Please see my code below :

var params = {
path: filename /* STRING */,
columnMappings: undefined /* STRING */,
hasHeader: undefined /* BOOLEAN */,
longitudeField: undefined /* NUMBER */,
dateFormat: "YYYY-MM-DD" /* STRING */,
fileRepository: repository /* THINGNAME */,
latitudeField: undefined /* NUMBER */,
fieldDelimiter: undefined /* STRING */,
stringDelimiter: undefined /* STRING */,
dataShape: "Building_DataShape" /* DATASHAPENAME */
};
// result: INFOTABLE
var csvData = Resources["CSVParserFunctions"].ReadCSVFile(params);
var length = csvData.rows.length;
var params2 = {
infoTableName : "InfoTable",
dataShapeName : "Infotable_DataShape"
};
// CreateInfoTableFromDataShape(infoTableName:STRING("InfoTable"), dataShapeName:STRING):INFOTABLE(TestCSV_DataShape)
var properties = Resources["InfoTableFunctions"].CreateInfoTableFromDataShape(params2);
for each (var row in csvData.rows) {
    // Thing1 entry object  
var newEntry = new Object();  
newEntry.time = row.date; // DATETIME - isPrimaryKey = true  
newEntry.quality = undefined; // STRING 
newEntry.name = "Consumption"; // STRING  
newEntry.value = row.consumption; // STRING  
properties.AddRow(newEntry);
newEntry.name = "Demand"; // STRING  
newEntry.value = row.demand; // STRING  
properties.AddRow(newEntry);
// no return
Things["Building"].UpdatePropertyValues({ values: properties });
}
var result = properties;

The format of data in my CSV file is listed below:

2016-01-01,1227.75,123.58

2016-01-02,153,34.82

2016-01-03,167.21,28.05

2016-01-04,181.05,24.55