Persistent Infotable of type DataTable gets deleted upon server restart

[vmihai]

Hello,

I have an infotable that I set to be of type DataTable and I made it persistent. Despite this it gets emptied when the server is restarted.

It doesn't have a default value assigned to it.

I was expecting the data from it to be stored across server restarts.  Is this behavior normal or is some sort of bug ?

Thank you,

Veronica

[jpinto]

I also have this situation with persistent properties, even for simple type properties such as Numbers and Strings, at each tomcat restart their values do not persist.. I am wondering, if I am missing something or like you said, it is just a bug.

[Aanjan]

Veronica, is this a neo install?

[vmihai]

Hi Aanjan,

This happens on a ThingWorx 6.6 with H2 and also  a ThingWorx 7.0 with Postgress.

It doesn't happen on a ThingWorx 5.4 with Neo... with the same entities. I have imported them in the exact form.

Thank you,

Veronica

[Aanjan]

I've seen this happen with neo; especially when data locks or thread locks happen. The change would commit to memory, but won't commit to disk. With a Tomcat restart, they essentially get wiped out as they don't get written to disk. I'll test this out on 7.0 postgres.

[rwiseman]

I have something that sounds exactly like this issue in ThingWorx 7.3.0 using PostgreSQL.

I have an Infotable that's populated via a mashup.  It had one row already and the mashup adds a new row.  If I click the "Set" button on the property, the both rows are there, but if I look in the database (in property_vtq) only the original row is there.  (I have to get the hex string and decode it to see what text there is, but I can clearly see that only the original row is there.)

Then what happens depends upon whether I click the "Set" button on the "Set value of property" dialogue:

1. If I DO click it and then re-check the database, the new row is now there (in addition to the original row).  Restarting tomcat causes no problems now, because the row is in the database.
2. If I DO NOT click it, but instead restart tomcat, the new row never gets written to the database.  The original row is still in the database after the restart but the new row is not.

[Aanjan]

Richard, just to make sure - if you add a row via the Mashup, and don't hit Set, it doesn't set, correct? That would be the behavior as it still is in memory and won't commit to the database unless you 'set' the values/ row.

[rwiseman]

No, the row has already been added via the mashup using the AddRow service of Infotable.

I add the row via the mashup, then I go to Composer and go to the Thing where the Infotable is a property.  I click "Set" for the Infotable to check its current value and can see that the row I just added via the mashup is there.

The problem is that although the value is set, it's not in the database: unless I click the "Set" button in the "Set value of property" dialogue (NB: this is without making any changes to the displayed value) the value does not get committed to the database.

[CarlesColl]

Hi Richard / Aanjan,

Make sure to set it explicitely when modifying an Infotable Property. What does it means?

When modifying a Infotable Property never ever do:

me.myInfotableProperty.AddRow(whatever);

me.myInfotableProperty.Delete(whatever);

me.myInfotableProperty.rows.value = whatever;

me.myInfotableProperty."WhateverModifierAction"

Always ( repeat with me this mantra and burn in fire on your mind ):

1. Clone Infotable Property
2. Modify the Cloned Infotable Property Variable
3. Set the Infotable Property with the Cloned Instance

Sample:

var tempInfotable = Resources["InfoTableFunctions"].Clone({ t1: me.myInfotableProperty }); // -- 1

tempInfotable.AddRow(whatever); // -- 2 ( or any other kind of modification on the Infotable

me.myInfotableProperty = tempInfotable; // -- 3

[rwiseman]

Thanks Carles.

Just to make sure I understand, you're saying that it's not safe (or at least not reliable) to modify an Infotable that's a property?

Is this the official ThingWorx line, or just something you've found to be true?

Either way, if I need to add a row to a large Infotable, it seems very inefficient way of having to do it!!

It sounds like a useful workaround, though, so I'm grateful for that!

Thanks again,

Richard

[CarlesColl]

Hi Richard,

Yes it's the official way, at least for Persistent Infotable Properties, for non persistent you can go and modify directly the property.

Yes it's inefficient, and yes you can have concurrency problems..., but anyway you are not dealing with a Database rows here it's just a property of type infotable which it's persisted ( I think ) as a JSON object no real tables are here.

Carles.

[jkaczynski]

Hello Carles,

Yes, I'd like just to confirm, that InfoTable from the back-end perspective is stored and treated as a JSON string (because it's a JSON string in fact).

Regards,

J.

[rwiseman]

A variant that is (in a way) more efficient and is closer to the code that people are likely to write before they know about having to clone, is:

1. Modify the Infotable property variable.
2. Set the Infotable property variable to a clone of itself.

So instead of:

var tempInfotable = Resources["InfoTableFunctions"].Clone({ t1: me.myInfotableProperty });

tempInfotable.AddRow(whatever);

me.myInfotableProperty = tempInfotable;

it would be:

me.myInfotableProperty.AddRow(whatever);

me.myInfotableProperty = Resources["InfoTableFunctions"].Clone({ t1: me.myInfotableProperty });

This should work, right...?

[CarlesColl]

Hi Richard,

Why it's more efficient? Did you do performance tests?

And by the way, if it's a non-persistent property two threads can be modifying the same property at the same time causing bigger problems.

Carles.

[rwiseman]

That's why I said "in a way" -- I didn't mean more efficient from an execution point of view, I just meant fewer lines of code to do the same thing.  But I think the more relevant argument for doing cloning last is that the code is then closer to what's natural when you don't know about the cloning requirement.

Do we need to worry about the non-persistent property case?  Isn't cloning only required for persistent properties -- to ensure that they get written to the database?  As I understand it, there's no need to clone an Infotable property if it's not persistent.  In my experience, the problem of data disappearing only occurs when the server is restarted, and non-persistent properties' values are lost on server restart anyway aren't they?

[CarlesColl]

Yes non-persistent properties are lost when a server restart, but on those you have concurrent updates happening at the same time, then modifying the same Infotable by two threads can cause bigger problems ( code crash, infinite loops,... ) then better to clone also, and it's better to just have a way to code that way you don't have to think if it's persistent or not, and sometimes properties nature change over time.

About Clone last --> my first paragraph says why not, neither I see it clearer than clone first ( it's confusing and not natural, clone first can be understood at first sight clone last not ).

[rwiseman]

Fair enough -- I find clone-last clearer though!

[jkaczynski]

Hello,

That's true, answer of Carles Coll​ should be marked as a correct answer. Other ways don't work and just causes troubles.

So, againg with the Carles mantra:

1. Clone Infotable Property
2. Modify the Cloned Infotable Property Variable
3. Set the Infotable Property with the Cloned Instance

;-)

Regards,

J.