Modifying the data shape in Production

[rkandasamy-2]

HI Team,

we have a scenario where we need to modify the fields of data shape which is used in data table again which holds like 10000 records. we have done a quick test and found out the following behaviors.

I have added around 100 entries to a data table.

Case 1       : Existing Field was Deleted from data shape

Behavior    : Then also Remaining field values were available.

Case 2       : Deleted field was added again to data shape

Behavior    : Respective field values were showing up again

Case 3       : Added a new field to data shape

Behavior    : it was empty and the field does not have any default values. It was undefined.

Note: It is not possible to rename the existing field of a data shape. Need to be delete it and create with new name.

Details:

Thingworx : 7.1.0 Neo4j version

Tomcat      : 8.0.38

Just wanted to confirm, If it is safe to modify the fields of a data shape where its data table holds thousands of records.

Please let me know if there are any additional things that needs to be take care in this case.

[posipova]

Making major modifications in production directly is never a recommended approach. While it might sound and seem like a safe thing to do, it's always recommended to apply the change in dev/testing environment first.

[rkandasamy-2]

Hi Polina,

Thanks . We used to do deployment as you said.

Just wanted to understand, is there any other extra steps that we need to be aware of doing that datashape modifications in the dev system.

[CarlesColl]

We do modify DataShapes on production, also we allow users to Add/Remove columns, all associated to Streams ( not DataTables but they are almost the same kind of storage).

All the DataTables/Streams data on rows are stored as a JSON per row ( not as columns as a traditional relational datatable on a DataBase ), knowing this you can know what will happen when you do modification in it.

Few things:

• If you remove a column from the datashape the data on the row won't be removed
• You can't rename a DataShape field name
• It's safe to modify DataShapes for Streams and DataTables, it won't make any difference on the stored data itself.

On our case, where we have totally dynamic Streams user manager, we did use "code" field names ( column1, column2, column3,... ) and used DataShape Field Description as Column name ( field description can be modified ) then built few services to use Field Descriptions as Field Names