Skip to main content
1-Visitor
January 9, 2017
Question

Why TWX Platform could not receive/queue all value streams updated by edges?

  • January 9, 2017
  • 8 replies
  • 5051 views

We made one edge update a temperature property increased by 1 every 5 secs. If 300k ~ 400k edges update logged temperature properties(value streams) every 5 secs concurrently, the ThingWorx Platform(TWX) could only receive/queue 85% ~ 90% of properties. When the edges' connections are established stably, We observed the 'totalWritesQueued' value of Value Stream Substem every 5 secs and computed its minus, then divided this minus by the amount of edges.

My question is, why the TWX couldn't handle 100% of properties when  more than 300k edges update properties concurrently?


+ TWX System Resource:

  - CPU E5-2650 v3 @ 2.30GHz with 10 cores * HT * 2-way

  - CPU usage: less than 50%

  - Total Memory: 128GB

  - JVM Memory: 96GB


+ TWX Subsystem Configuration:

  - Default except:

    > Value Stream Subsystem:

      ' maximumBlockSize: 50,000

      ' maximumQueueSize: 10, 000, 000

      ' maximumWaitTime: 1000

      ' numberOfProcessingThreads: 80

+ Logs:

  - Tomcat:

    com.thingworx.exceptions.CouldNotConnectException: 172.21.128.30-temp13270 is not connected

    java.util.concurrent.TimeoutException: Timed out APIRequestMessage [requestId: 1398634, endpointId: -1, sessionId: -1, method: POST, entityName:

    172.21.128.30-temp11300, characteristic: Services, target: NotifyPropertyUpdate]

  - TWX:

    [L: ERROR] [O: c.t.p.p.BatchStreamEntryProcessor] [I: ] [U: SuperUser] [S: ] [T: pool-8-thread-10] Error executing batch

    [L: WARN] [O: c.t.s.s.w.WSCommunicationsSubsystem] [I: ] [U: ] [S: ] [T: WsCommunication-151] Could not dispatch async service request [NotifyPropertyUpdate] :

      Timed out APIRequestMessage [requestId: 1405707, endpointId: -1, sessionId: -1, method: POST, entityName: 172.18.142.102-temp540, characteristic: Services,

      target:NotifyPropertyUpdate]

    [L: WARN] [O: c.t.c.s.e.DispatchingServerEndpoint] [I: ] [U: ] [S: ] [T: http-nio-8080-exec-68] NULL MessageSynchronizationContext! Request either timed-out waiting

      for this response, or it was received by mistake: ResponseMessage [requestId: 1401202, endpointId: 175200, sessionId: 1406274, code: STATUS_SUCCESS,

      multipart: false, packet #: 0, total packets: 0]

+ QueryPropertyHistory for One Thing: NOT CONTINUOUS

temperature

timestamp

98

2017-01-04 03:49:27.962

93

2017-01-04 03:49:01.006

91

2017-01-04 03:48:49.861

87

2017-01-04 03:48:26.911

86

2017-01-04 03:48:21.909

85

2017-01-04 03:48:16.907

79

2017-01-04 03:47:28.103

77

2017-01-04 03:47:17.009

74

2017-01-04 03:47:00.646

72

2017-01-04 03:46:46.981

70

2017-01-04 03:46:34.941

69

2017-01-04 03:46:29.372

66

2017-01-04 03:46:11.139

65

2017-01-04 03:46:05.810

63

2017-01-04 03:45:54.443

62

2017-01-04 03:45:48.696

61

2017-01-04 03:45:42.753

60

2017-01-04 03:45:37.495

58

2017-01-04 03:45:26.171

53

2017-01-04 03:44:57.037

50

2017-01-04 03:44:37.082

48

2017-01-04 03:44:26.089

46

2017-01-04 03:44:13.205

44

2017-01-04 03:44:03.188

42

2017-01-04 03:43:52.557

38

2017-01-04 03:43:29.961

33

2017-01-04 03:42:55.120

31

2017-01-04 03:42:44.548

27

2017-01-04 03:42:21.796

26

2017-01-04 03:42:12.657

25

2017-01-04 03:42:07.615

24

2017-01-04 03:42:02.288

21

2017-01-04 03:41:42.045

19

2017-01-04 03:41:30.359

18

2017-01-04 03:41:24.820

16

2017-01-04 03:41:13.063

15

2017-01-04 03:41:04.834

14

2017-01-04 03:40:59.453

13

2017-01-04 03:40:54.440

12

2017-01-04 03:40:49.361

11

2017-01-04 03:40:40.500

10

2017-01-04 03:40:28.502

9

2017-01-04 03:40:23.246

8

2017-01-04 03:40:15.742

7

2017-01-04 03:40:10.398

6

2017-01-04 03:40:04.640

5

2017-01-04 03:39:49.397

4

2017-01-04 03:39:38.949

3

2017-01-04 03:39:33.756

2

2017-01-04 03:39:28.749

1

2017-01-04 03:39:17.345

8 replies

5-Regular Member
January 11, 2017

Are you saying that you have 300,000 - 400,000 edge devices updating the Platform, with no Connection Server, and you are wondering why the Platform cannot handle this? The Platform is not designed to handle this many edge devices on its own. You need the Connection Server for this. Here is a KCS Article on the subject. Hope this helps!

mren1-VisitorAuthor
1-Visitor
January 12, 2017

Thanks so much for your reply!

To simplify the problem, I didn't mention the Connection Server. In fact, I made 300k ~ 400k edge devices update properties via 2 ~ 3 optimized high-performance Connection Servers. I've checked that the TWX platform can handle 200k value streams sent by remote devices every 5 secs via only one Connection Server.

I really appreciate your help:-)

5-Regular Member
January 12, 2017

Ah, so the issue can't be the fault of too many connections. You're actually not the first person to post about this, and I am pretty sure there was no resolution on the other thread. You may want to consider opening a support case for this issue. How are your properties updated? Are you sure the updates are propagating to the Platform? Are these remote properties? What are their remote bind settings?