We are facing unexpected behavior of Async service when called from sync service.
While calling Async thingworx service, database related operations were not functioning well when called from sync service after logging entry in Data table. When async service tries to fetch the entry from data table using primary ID, no result is getting returned.
Sync service A adds entry in data table and calls Async service B (pass primary ID of same data table entry) for further processing of request.
But this seems to be working windows platform and issue encountered on linux ONLY.
As per the understanding, this could be due to the way platform handles multi-threading as async execute on separate thread or may be accessing data table before commit operation pf previous update.
Please let us know in case anybody faced this issue and what all options resolutions are available.
Just to update that we have analyzed further on this issue and tried to reproduce the same issue on local Linux VM using same client VM configuration and things are working fine.
I am not sure what exactly the issue is and there might be some difference in the installed packages. We are still working on this and update further on this thread if I get some more details.
You are facing what you are saying here:
"As per the understanding, this could be due to the way platform handles multi-threading as async execute on separate thread or may be accessing data table before commit operation pf previous update."
As async operation it's executed before you sync process ends ( the transactions start when your services starts and ends when your service ends ).
Hello Carles Coll,
On client VM (RHEL 7.2 version, 64 bit arch), using one service I am inserting a data in data table and using async to retrieve the already inserted record using the same ID, I am not able to perform or fetch the data entry made sync service and getting message that record doesn't exist.
This works well when tested locally on windows platform but facing issue when the same is deployed on Linux VM. My understanding was that this could be due to different platforms multi-threading capability.
But another update provided above as per further analysis, this problem is not due to different platforms and everything is working fine even with linux VM when tested locally. There might be some difference in installed packages on both the Linux VMs.
I am still analyzing this and I am not sure what exactly is the problem. It could be configuration differences.
You never know when a async process will be called ( at the same time, after, sometimes at the same time some times after,..), then it may act differently depending on the platform, but you should avoid this kind of situation, it's a shame but you will need to execute it synchronously or put on a queue ( there's no queues system on TW you will have to build your own or import a queue system as an extension ).
Synchronous execution is not the way because we expect large processing and to avoid long wait time for client.
Also, we are using database (Data table) queue for scheduler to handle failed requests.
Put the async operation on your own queue where you control the execution ( for instance delay the execution half a second or alike ), on this queue you can check if data it's persisted and if not, you can requeue the operation for instance if info it's not ready, I don't know your logic...
Thanks Carles Coll,
As per our logic, we want to log requests in data table queue so that failed requests (due to internal error or server failure) can be re-tried at some later point of time and we don't lose any business data.
I agree, queue system can resolve access problem but queued requests will be stored in-memory only and can be lost in case of server failure.
What "queues" do you have in mind? I am wondering whether it is possible to queue DataTable services execution (which are coming from multiple Things from multiple threads)?
I am facing an issue: I have multithreaded SDK Agent which push data to ThingWorx for ~15 devices every 0.5sec (a lot of remote properties) and very, very complex logic for subscription which is also triggered by the Agent whenever there is an update of the properties - in this logic I am interacting with the datatables (especially one) multipletimes (Queries, AddOrUpdate, etc) which causes data inconsistency (as multiple threads are accessing data which may be updated in the meantime by other thread and moreover - sometimes PersistenceProvider is not able to handle the data and I have an error from it).
I know that I should go most likely for external DB, but I would like to consider possibilities in TWX (DataTables OOTB Queue mechanism for DataTables seems very interesting).
Twx 7.3.5 (PostgreSQL).
Thanks in advance for any hint.
When I mean DataTable/Queue, I mean a custom code which uses DataTables + Timers to build a queue:
Then you control concurrency on subscription calls, the subscription to data change event just adds a Pending Service Call tot he DataTable, and then the consumer ( Timer ) consumes the queue.
TW should have Queues out of the box...
We are using it effectively on our production server, for instance we have a Delayed Data Change Event, which automatically adds the Data Change Event to the queue for execution later on. We also use it for instance to Iterate over a large amount of long running process in order to cut transactions in small pieces, as the worst thing that can happen to TW it's a long running service ( aka > 10minutes )
Just a workaround,
We have added pause function (seconds of delay) on async thread to avoid race condition for time being. Ideally, there should be only one triggering point for async call but we designed it as per our use case requirement. We are still looking for better implementation and might re-design this solution accordingly.
We have dona a good amount of analysis for this and anyone can contact if required.
Not, with pause you don't solve anything, you will face exactly the same problem, as your async process starts at the same time as the other transaction it's running, then on the starting point of the Async process the Database status it's without data.
You don't control the transaction, the transaction it's controlled by TW and it starts at Service Call.