Best way to implement 5000 api calls in a single service execution

[SP_10199785]

I am calling an API 5000 times in a for loop with separate key id's , its taking sometime to fetch the data and update in the datatable and is getting script timeout after 10 minute(600 seconds). 

Can anyone suggest an efficient way to handle this scenario and for an optimized solution.

[nmilleson]

@SP_10199785 ,

How often do you need to run this service?  Once a day? Once a minute/hour? If you don't have to run it very often, you can just set up a scheduler and increase your script timeout in the ThingWorx configuration.

-Nick

[SP_10199785]

Hi,

I need to run this service every 2hrs and sometimes manually triggered by user on updations. 

Increasing the timeout to 30min would cause an impact on the thingworx performance and others services too.

Thank You

[danmorin · Accepted solution]

Hi @SP_10199785 

I have 2 ideas to optimize your case

1. You can try to make the service an async service by ticking Async on the left-hand side script editor. This way ThingWorx would create another thread 
https://www.ptc.com/en/support/article/CS261852
https://www.ptc.com/en/support/article/CS222280

2. Optimize your script by following JavaScript best practices
https://support.ptc.com/help/thingworx/platform/r9/en/index.html#page/ThingWorx/Help/Best_Practices_for_Developing_Applications/JavaScriptCodingBestPractices.html

Hope this answer helps you

Thanks

[SP_10199785]

Hi 

The Async service is causing the server to be down as async service is fetching the data from the API and updating the datatable in a synchronous manner which is creating a deadlock.

Can you please suggest 

Thanks

Sonali

[danmorin]

Hi @SP_10199785 

In that case, you need to look down at this table, 5000 API Calls mean 5000 HTTP requests per second. You need to queue it manually to match this RPS base on your ThingWorx Server Size.

https://support.ptc.com/help/thingworx/platform/r9/en/index.html#page/ThingWorx/Help/Installation/Sizing/compare_sizing_criteria_to_guidelines.html#

Thanks