Windchill 13.0.2.0: Intermittent Timeouts Experienced During Batch Requests to WRS Endpoints

Version: Windchill 13.0

Use Case:

We are currently hosting Windchill on a Kubernetes cluster deployed in Azure AKS. The setup consists of 4 Method Server pods and 15 Background Method Server pods, with the following resource configurations:

Method Server Pods:

Resource Limits: 
CPU: 500m 
Memory: 512Mi 

Resource Requests: 
CPU: 250m 
Memory: 256Mi

Background Method Server Pods: 

Resource Limits: 
CPU: 7 cores 
Memory: 30Gi 

Resource Requests: 
CPU: 500m 
Memory: 10Gi

We are running batch requests following the guidelines outlined in the Windchill REST Services documentation (https://support.ptc.com/help/windchill_rest_services/r1.6/en/index.html#page/windchill_rest_services/WCCG_RESTAPIsBatchSupport.html). Our workload involves performing approximately 25 batch requests across 10 threads simultaneously.

Sample Requests:

Below are examples of the batch requests being executed: 

--batch_request
Content-Type: application/http
Content-Transfer-Encoding:binary

GET /Windchill/servlet/odata/v7/ChangeMgmt/Changeables('OR:wt.part.WTPart:123*******')?$select=Number,Version&$expand=ResultingByLinks($expand=ResultedByObjects($expand=AffectedObjects($select=Name);$filter=isof(PTC.ChangeMgmt.TypeUsageChangeTask);$select=Name,State);$select=TARGETUSAGEMATURITY,CreatedOn;$orderby=CreatedOn%20desc) HTTP/1.1


--batch_request
Content-Type: application/http
Content-Transfer-Encoding:binary

GET /Windchill/servlet/odata/v7/ChangeMgmt/Changeables('OR:wt.part.WTPart:123*******')?$select=Number,Version&$expand=ResultingByLinks($expand=ResultedByObjects($expand=AffectedObjects($select=Name);$filter=isof(PTC.ChangeMgmt.TypeUsageChangeTask);$select=Name,State);$select=TARGETUSAGEMATURITY,CreatedOn;$orderby=CreatedOn%20desc) HTTP/1.1


--batch_request
Content-Type: application/http
Content-Transfer-Encoding:binary

GET /Windchill/servlet/odata/v7/ChangeMgmt/Changeables('OR:wt.part.WTPart:123*******')?$select=Number,Version&$expand=ResultingByLinks($expand=ResultedByObjects($expand=AffectedObjects($select=Name);$filter=isof(PTC.ChangeMgmt.TypeUsageChangeTask);$select=Name,State);$select=TARGETUSAGEMATURITY,CreatedOn;$orderby=CreatedOn%20desc) HTTP/1.1


--batch_request
Content-Type: application/http
Content-Transfer-Encoding:binary

GET /Windchill/servlet/odata/v7/ChangeMgmt/Changeables('OR:wt.part.WTPart:123*******')?$select=Number,Version&$expand=ResultingByLinks($expand=ResultedByObjects($expand=AffectedObjects($select=Name);$filter=isof(PTC.ChangeMgmt.TypeUsageChangeTask);$select=Name,State);$select=TARGETUSAGEMATURITY,CreatedOn;$orderby=CreatedOn%20desc) HTTP/1.1

Description:

We are encountering intermittent timeouts with our batch requests. When a timeout occurs, we retry the request up to 3 times, with a 10-minute wait between retries. Despite being generous with the timeout duration, the issue persists intermittently.
From our analysis of pod performance metrics, it appears that the pods may be overloaded or busy at the time the requests are executed. However, we want to ensure there are no other underlying causes for these timeouts.
Key Concerns:

1. Could there be other factors beyond pod load causing these timeouts?
2. Is there anything in our request structure or execution pattern that could be contributing to the issue?
3. What additional steps can we take to mitigate these timeouts on our end?

Additional Context:
Given that we already allow a 10-minute timeout, we are questioning if the requests we make (e.g., expanding multiple nested entities or filters) are particularly resource-intensive or unreasonable. We aim to improve system reliability and reduce the occurrence of timeouts while ensuring efficiency in our operations.
We would appreciate guidance on:

• Identifying potential root causes for the timeouts.
• Optimizing our request patterns or system configurations.
• Any best practices to manage system load or improve Windchill's performance.