Modifying Entries in InfoTable / Behaviour changed with newer Thingworx

[atondorf]

Hi all,

for some aggregation in memory I used an Infotable.

So my Thing has an property of type INFOTABLE and I can access the rows to read/write single entries. e.g:

var now = new Date();
me.tab.rows[0].ts_set = now;
var result = me.tab;

Thingworx 8.4 & 8.5 give me a result with the correct now() timestamp in the first row.

I now tested this code in Thingworx 9.1 and it does not give me the correct now() timestamp in the first row!

So is it not possible to modify single entries in INFOTABLES in place any more?
Do I really have to copy the INFOTABLE every time I access it and change the code to:

var now = new Date();
var tab = me.tab;
tab.rows[0].ts_set = now;
me.tab = tab;
var result = me.tab;

Any suggestion, please?

Greetings

Andreas

[atondorf]

I attached a small datashape and a thing to test this.

TestInfotableDS only contains two values: id (long), ts (Date).

The thing TestInfoTable has an infotable of this shape and one service addRow().

var tab = me.tab;
var count = tab.getRowCount(),
var entry = {
    id: count,
    ts: new Date()
};
tab.AddRow(entry);
if( count > 0 )
    tab.rows[count-1].id -= 1;  // does not work in TWX 9.1
/* 
 * uncomment to make it work in TWX 9.1! 
 * But why does AddRow reflect back to property and in place mod not?
 */
// me.tab = tab;  
var result = me.tab;

As described in the Thingworx Help - Infotables the local variable tab reflects back to the property me.tab.

But each time a row is added to the infotable the id of the previous row should be decremented by one.

This happens in Thingworx 8.4 & 8.5 but not in THingworx 9.1.

Is this intended behaviour? How can I modify values in an infotable in place?

[DmitryTsarev]

@atondorf What do you mean as "does not give me the correct now() timestamp"?

I don't have 9.1 at hand atm, but I did a quick test on 9.0 and timestamps look good.

Is the behaviour / output in 9.0 correct / expected and the same as you observe on 8.4 / 8.5?

 

[atondorf]

Hello Dmitry,

no that is exactly the not planned behaviour ...
To make it more clear I modified my code and extended the DataShape by a string value:

var tab = me.tab;
var count = tab.getRowCount();
var entry = {
	id: count,
    ts: new Date(),
    value: "Original"    
};
tab.AddRow(entry);
if( count > 0 ) {                             // if there is a prev row, modify it!
    tab.rows[count - 1].id -= 1;              // does not work in TWX Version > 9.0
    tab.rows[count - 1].value = "Modified";   // does not work in TWX Version > 9.0
}
// me.tab = tab;    
var result = me.tab;

So when I execute this Service 3 times I get different results for TWX 8 & 9.

In Thingworx 8.x it's:

 When I do the same in Thingworx 9.x:

So it seems that in place modification of Infotables is not possible any more strating with THingworx 9.0!

That's horrible for many of our internal logics ...

If this is intended behaviour I would expect a huge exclamation mark in the Documentation. 

ATTENTION!! Behaviour of in place operations changed!

[c_lowy]

Hello @atondorf,

Starting ThingWorx 9.0 individual infotable cell values cannot be modified using a syntax like me.InfoTableName.rows[0].FieldName = 999; as it used to in previous versions. This was implemented voluntarily, I believe to avoid issues in HA environments. 

The recommended syntax is the following :

• Instead of writing :

me.InfoTableName.rows[0].FieldName = 999;

• You would write :

var tempInfoTable = me.InfoTableName;

tempInfoTable.rows[0].FieldName = 999;

me.InfoTableName = tempInfoTable;

So going back to your example, your service is already partly designed as recommended. I believe you are simply missing the part where the value of the tab variable you defined at line 1 is assigned back to me.tab :

var tab = me.tab;
var count = tab.getRowCount();
var entry = {
    id: count,
    ts: new Date()
};
tab.AddRow(entry);
if( count > 0 )
    tab.rows[count-1].id -= 1;
me.tab = tab;
var result = me.tab;

I tested this successfully in 9.1.0 and attached a screenshot for reference.

[atondorf]

Hi @c_lowy 

yes i also discovered this, thats why i had the line in the code already.

So it's intended behaviour.

But how much overhead does it produce to always copy the whole table?

Or is it not copied, as documentation is talking of proxy (PTC Help Center)?
So if it's for protection in HA environments I am missing a secured service to update a single row.

There is a service (at Java side) setRow(idx, valuecollection), but it is not available at Javascript.

Greetings

Andreas

[atondorf]

Hi @c_lowy ,

ok I did some more tests and things are getting even worse!!!

As the copy (or proxy) does not reflect back to the property directly I get into race conditions, when some services share the infotable. 
So I have one service creating new rows and one event-subscription, that updates these rows.

Before it was no issue at all, as every service just manipulated one column of the infotable.

Now with every service call I update the complete infotable and as Thingworx is lacking synchronisation I am loosing data!

So there are two possibilities:

- I need a service to manipulate an infotable cell in place directly.
  (That was the approach working with Thingworx 8.x)

or

- I need to be able to use synchronisation primitives like mutex/event/atomic to synchronize the access.

  (There are several requests for this in the community already).

Greetings

Andreas

[c_lowy]

Hi @atondorf,

Thanks for sharing the details of your use cases.

We do recommend to clone infotables, update them and set their value back but it is true that you can still get unpredictable results when it is assigned back. The use of try catch statements can help identify these errors.

What happens is that the infotable properties are not thread safe, i.e. if an infotable property is read and updated in different threads there is a chance for some ConcurrentModificationException. As per the "Concurrency patterns" section under 

Creating, Implementing, and Testing Services "If two services are trying to write the same property at the same time, both writes will be successful. The property value will end up in whatever state was executed last, which may be different than the order in which the writes were submitted to the server."

And as you pointed out synchronization primitives are not supported in ThingWorx scripting as of today so I don't believe there is a solution to avoid 100% of the concurrency issues if you want to stick with infotables. I have mentioned your use case / request in ongoing internal discussions around this topic.

Could you please file a Product Idea regarding the missing support of synchronization primitives as another way of emphasizing this request ? feel free to link it in this Community thread so that others users can vote an comment on it.

Thanks !

[atondorf]

Hi @c_lowy,

as I said, sync has been mentioned many times before and also in the Product Ideas. 

But what I really need is a storage where I can quickly aggregate some values over certain time periods.

Our goal was to do some short term KPI calculations in memory.

So the infotable as an property in memory with possibility to write in place was the best solution for this,
but now it's just gone by decision ... and I am not sure if synchronistaion is the best solution here.

Suppose there are syn primites in Thingworx (e.g.: GitHub - ThingWorxConcurrencyExtension - Yes i am not completely uninvolved), how much overhead does the code produce, when the table has ~500 rows and 20 Columns with simple types?

var tab_l = me.tab_p;
// modify tab_l cells 
me.tab_p = tab_l;

So does the last assignment really copies the whole infotable or is tab_l tab still a proxy that just applies the changes to tab_p on the assignment operator. So how long would the function lock the table?

A function setRow() maybe could massively decrease the locking time, then.

[c_lowy]

Hi @atondorf,

I do understand your point but unfortunately do not have all the answers on this topic. What I can offer is the following :

• I noticed that there is no existing Product Idea about having the setRow() function available in ThingWorx, would you mind creating one and sharing the link here ?
• I will then contact PTC Product Management to bring this topic to their attention

Thanks !

[atondorf]

Dear c_lowy,

in fact as I thought about it even the setRow() does not help in our condition without concurrency synchronisation.

The situation is:

- We have "active" things that monitor the machine condition by a main service that is called once a second by a timer.

- Additionally there can be some outer events triggered by rest-calls from our MES.

- To calculate some quick KPIs we used an InfoTable to aggregate some main performance properties like current speed, status etc...

- There was no synchronisation issue, as each thread was limmited to one column of the infotable and wrote it' s information there.

So this is not working any more ... i need the access to the columns of the infotable.

Greetigngs 

Andreas

[bot1q]

Sometimes I think the usecase for Thingworx is connect to RasPI and make some fast POC to receive a contract