Import data from a datatable programmatically

[JayT9000]

Hi everyone,

we built in our environment a CICD pipeline to deploy our Thingworx applications automatically.
After importing all entities and extensions to a new thingworx deployment we would like to import the data of datatables too.
We tried to import the data via http request like the Composer do.

var ImportTask = Resources["ContentLoaderFunctions"].PostText({
headers: {
"appKey" : ApplicationKeys["MYKEY"].GetKeyID(), 
'X-XSRF-TOKEN': 'TWX-XSRF-TOKEN-VALUE',
'Accept':'application/json',
'Content-Type':'application/json'
},
url: Hostname + "/Thingworx/DataImporter?repositoryName=" + me.Repository + "&path=/MyRepositoryPath/" + exportDataDirName +"&IgnoreBadValueStreamData=false&WithSubsystems=false&purpose=import&usedefaultdataprovider=false&_="+Math.random(),
ignoreSSLErrors: true
}); 
logger.debug(me.name + ".ImportData: Import data triggered " + ImportTask);

After execution the info log message (Task-1576673643334 Data Import Task Started) appears in the application log but the asynchronous import process never ends.
We can't see any log messages depending to the import process on the server and Thingworx hangs with 30% CPU usage. We have to kill the thingworx process to stop the service.
The behavior is the same on different machines.

Has someone any ideas what might be wrong or how we can trace or debug in more detail?

Best regards
Jan

[slangley]

Hi @JayT9000.

Have you tested doing a manual import?  If so, does it succeed?  What version of ThingWorx are you running?  When ThingWorx hangs, does anything show up in the Tomcat logs?

You could turn up the logging level to possibly capture additional information during the import process.  To do so, go to Monitor.  Under the logs section, select the desired log and select Configure to change the logging level, then re-test.

Regards.

--Sharon

[JayT9000]

Hi Sharon,

Yes, the import of data tables works fine manually and sometimes the programmatic way too (one datatable with a few rows). The logfiles only contains the info messages regrading to the import process. Should there some more messages while the asyncronous import process is running?
Also in the tomcat logfiles hasn't any entries which points to an specific issue.

We're using Thingworx 8.4.2

Regards
Jan

[slangley]

Hi @JayT9000.

How many records are contained in the data tables and how many data tables are you attempting to update in a single import?  You may need to import tables individually.

Regards.

--Sharon