Application Design for more than 100K devices

[TanmeyTWX]

Hi Team,

Kindly suggest a design to handle more than 100K devices and keep the Application performance in-tune at the same time.

I have got ,say 150K devices to be monitored and tracked over a geographical area. So what should be the approach to have minimal response time of loading data onto Mashups/Grids.

Note: Each device has 10 parameters.(common ThingShape)

1. One Thing per device?

2. One Thing for a cluster of devices?

3. One Thing itself has multiple Things in it (not sure if possible).

Please revert your comments.

[PaiChung]

How are the devices connected back to Thingworx?

How much data are you transmitting per second? minute? hour?

Are you adding any logic as data comes in, alerts?

These questions will help you to determine if you need connection servers, multiple servers, hardware sizing.

I would say that trying to load 150K assets into a mashup will likely crash or be incredibly slow.

I recommend you do some aggregated type status screens with specific drill downs.

PTC offers architecture/design Rapid Outcome workshops to help with just these type of challenging situations.

https://www.ptc.com/en/Success-Services/thingworx-design-workshop

[TanmeyTWX]

Hi Pai,

Thanks for replying to my query.

Please find my inline reply below.

How are the devices connected back to Thingworx? -- MQTT

How much data are you transmitting per second? minute? hour? --  per device * 10 properties per Hour

Are you adding any logic as data comes in, alerts? -- not as of now but surely in next phase of application.

I recommend you do some aggregated type status screens with specific drill downs -- Already added filters to list down small list but still 5000 devices to display after drill down.

Thanks!

[PaiChung]

I don't have a lot of experience with MQTT, but you will need to make sure that the listeners on the Thingworx side do not get 'overloaded' so try an approach with multiple MQTT listener Things and distribute the topics.

It'll probably be a trade off between queuing and using up additional threads to process the incoming messages.

5000 devices after drill down sounds manageable although that is still quite a lot.

Not sure if you have the ability to set up some frame work to run some load testing by firing a lot of MQTT messages to begin with. But it will help to have a tool like that to at least cover the ingest scale.