ImportSourceControlledEntities

[jensc]

Hello,

I've been building on a project to help with deployments of projects to QA/BV/Pilot/Prod instances after development has been completed.

The use case (which I haven't been able to find an alternative to, so if you have one please let me know) is that when you develop a larger project, you often end up with entities that gets deprecated.

If you have an especially large project, it can be annoying to have to manually clean up all other environments of these no longer used entities as you deploy your application.

The thought I had was that after using the ImportSourceControlledEntities function, I would delete any entities that were not imported from source control.

But when doing some tests I have run into issues with this error:

JavaException: java.lang.Exception: Import Failed: Unable to create group: [1,002] Data store integrity violation error: [Error occurred while accessing the model provider.]

It seems to me like it happens if the ImportSourceControlledEntities fails for whatever reason (mine has failed because of missing ThingShapes for example).

But when I read on the PTC help center, it says that any failure will make it so no entities are imported.

My understanding of the error message reading this support article is that this is due to entity id's have been saved in the database.

Either my understanding is incorrect, or there seems to be some bug here.

I am asking here before reaching out to PTC support, has anyone else had similar issues?

Thanks,

Jens

[PaiChung]

Please reach out to PTC support.

Failed imports should do a full roll back and not leave 'artifacts' or 'ghost' items

[jensc]

Hello @PaiChung,

I agree with you and I have opened a case. Let's see what we can find.

Thanks,

Jens

[nmutter · Accepted solution]

Hey,

1. for your project you should create an extension project. This results in a .zip file in the end which you can install first in QA and after validation in the next environments. The extension project can also only include .xml files (the ones you exported for Source Control). This is how we manage to keep the environments on a defined state. (It will also remove entities previously existing in your extension)

2. For you issue at hand you need to have a look into the persistence provider logs (e.g. PostgreSql) - if twx logs don't tell you. It may give you more information regarding what caused the error (like which insert into which table caused an issue).

[jensc]

Hello @nmutter,

#1: I have seen this used by PTC as well before, but I have not been able to find any documentation or recommendations on how to use it.

Would you happen to have any?

It seemed simple enough to add a metadata.xml file together with an Entities folder like so:

But if this is not "supported" by PTC, I'm hesitant to use it.

#2: We checked into the DB, although not the logs and found the entities there. But will do more investigation with PTC.

Thanks,

Jens

[nmutter]

The general documenation for extension is in the help center Best Practices for Packaging and Deploying ThingWorx Solutions . One point of it is 

Importing Composer-Created Entities 

It does create the structure via the Eclipse plugin. But in the end it is in the structure you showed (Entities folder and metadata.xml).

[jensc]

Hello,

Ah okay, yes I have seen this before as well and thought as you said that it was meant to be used if you created the project in Eclipse.

Especially as they do not give an example (from what I can tell) on what the metadata.xml file should include.

But it would be interesting to do it this way instead of trying to build something on our own.

However, I would need to check with my colleagues what they think and also perhaps check with PTC to see what they recommend.

Thanks,

Jens

[nmutter]

We also used the Eclipse plugin for the initial structure. It also creates a gradle build script to package the contents into the .zip file (can be used e.g. in CI).

But eclipse is not needed for us as we do not have Java Code. So in the end we just used it initially for the structure and build scripts.

[jensc]

Hello,

How do you deal with property values?

Because I assume they get overwritten using this method as well? 

Some values might not be so important if they get overwritten as we could have some release thing that sets important property values.

But some values might be things that users change, so it shouldn't be overwritten.

Thanks,

Jens

[nmutter]

Actually, property values are not overwritten. There was some scenario were they were overwritten sometimes (not sure if resolved by PTC). We have installation instructions to validate property values have the correct value after an update.

But for settings we mostly use the "Configuration tables". They are not overwritten. With them the only issue is that they are "merged"-> if the extension has e.g. 2 rows and in the instance there is only 1 row - it will leave row 1 as is but add the 2nd row. Needs to be known and handled accordingly. Good thing is that "row 1" will not be touched during import (so the config from instance will be kept)

[jensc]

Hello,

Thank you for the information. It definitely sound like something we want to use in our group, at least until PTC comes up with a better way of deploying applications.

Regards,

Jens

[nmutter]

You need to make sure to put your entities as "not editable" in the extension. Otherwise, they are not updated in the next version you install. That's something really important. Either you don't define the attribute at all or if so set it to aspect.isEditableExtensionObject="false"

I think SolutionCentral may be also one possibility. It somehow packages all your files related to a "project" and offers to deploy it to different instances. But I did not use it yet as it is some cloud service hosted by PTC. 

[jensc]

Hello,

Some more information:

After looking into the model_index table in the platform database we noticed that the imported entities existed in the table.

The imported entities at this point did not show up in the composer.

We were thinking that perhaps they would appear as ghost entities, but running the ghost entities service nothing showed.

Then we restarted the platform and this caused the entities to show up in the composer.

I have started a case for this as it seems to me like perhaps there could be some issue with the import service (or I am just using it wongly).

Will update this post once we figure it out.

Thanks,

Jens