Can Thingworx handle 10000 properties which values change in 100ms?

[hliu-21]

The platform is Thingworx + PostgreSQL + ConnectionServer.

I have 10,000 devices which update values every 100ms. Assuming one property per device, a total of 10,000 properties need to be submitted to Thingworx. Submitting data at this speed should be able to query 600 data per minute per property. However, at present I can only find about 300 data.

Don't know if submitting data to Thingworx at this speed is beyond its capabilities?

What parameters should I adjust to enable Thingworx to process high-frequency data uploads?

What I am currently doing is adjusting the following parameters in platform-settings.json.

But I don't understand the meaning of many parameters. For example, most of the parameters in ValueStreamProcessorSettings. These parameter's explanation in the Installing_ThingWorx_8_2_1_0_174097.pdf file is too brief and completely incomprehensible. Can someone help me to explain it?

{
"PlatformSettingsConfig": {
"BasicSettings": {
"BackupStorage": "C:/ThingworxBackupStorage2",
"Storage": "C:/ThingworxStorage2"
}
},
"PersistenceProviderPackageConfigs": {
"PostgresPersistenceProviderPackage": {
"ConnectionInformation": {
"acquireIncrement": 5,
"acquireRetryAttempts": 3,
"acquireRetryDelay": 10000,
"checkoutTimeout": 1000000,
"driverClass": "org.postgresql.Driver",
"fetchSize": 5000,
"idleConnectionTestPeriod": 60,
"initialPoolSize": 5,
"jdbcUrl": "jdbc:postgresql://localhost:5432/twdb8_2_5",
"maxConnectionAge": 0,
"maxIdleTime": 0,
"maxIdleTimeExcessConnections": 300,
"minPoolSize": 5,
"maxPoolSize": 500,
"maxStatements": 10000,
"numHelperThreads": 8,
"password": "newstar123",
"testConnectionOnCheckout": false,
"unreturnedConnectionTimeout": 60,
"username": "twadmin"
},
"ValueStreamProcessorSettings": {
"maximumBlockSize": 4000000,
"maximumQueueSize": 4000000,
"maximumWaitTime": 100,
"numberOfProcessingThreads": 20,
"scanRate": 1000,
"sizeThreshold": 4000000
}
}
}
}

[Rick-Stanley · Accepted solution]

100,000 property writes per second is beyond what the setup you described is certified to handle. Here is some info about other databases that can be used with ThingWorx that can handle that number of property writes:

https://developer.thingworx.com/resources/guides/persistence-provider-comparison/persistence-provider-options

You should talk to a sales engineer about options to handle your use case. I would guess they will suggest federating multiple ThingWorx instances behind a load balancer. 

[hliu-21]

I noticed that the database is really the bottleneck. Thank you!