Skip to main content
T
TanmeyTWX17-Peridot
17-Peridot
October 7, 2020
Solved

Individual streams per asset

  • October 7, 2020
  • 2 replies
  • 2082 views

Hi,

Is it recommended to create stream per device? any major impact or performance issues if create a independent stream for each device Thing.

example: for 10k devices there will be 10K streams for data storage of a device.

 

Please revert your suggestions.

TIA.

Best answer by nmilleson

@TanmeyTWX ,

 

I don't think the number of Things (in this case Streams) will impact the performance, as even the Small ThingWorx sizing shows a limit of about 30,000 Things (see https://www.ptc.com/support/-/media/BC2F28281E4142AFA6FC79EC49D9DBD3.pdf?sc_lang=en page 12)

 

However, when your streams have over 100,000 data points in them, it's recommended that you use an external data store (see https://support.ptc.com/help/thingworx_hc/thingworx_8_hc/en/index.html#page/ThingWorx/Help/Best_Practices_for_Developing_Applications/DataStore.html under "Best Practices for Choosing Your Data Store Option")

 

Hope that helps!

2 replies

C
Ciprian-Traian5-Regular Member
5-Regular Member
October 7, 2020

Hello @ @TanmeyTWX,

 

The amount of historical data to be stored on the platform impacts both database and platform system sizing.  
Larger data sources (streams, data tables, and value streams)will require longer transactionsto query from the database.
These long transactions can also cause the stream and value stream queues to back up and utilize additional memory.

The ThingWorx Platform Sizing Guide it is a good reference to check in this situation.

It may be of help to check the Application Development Guide for ThingWorx 8 because it gives more details on

how ThingWorx handles data storage.
Choosing the correct data storage is very important since it affects the outcome of the project,its scalability and reliability, and the user experience.

 

 

T
TanmeyTWX17-PeridotAuthor
17-Peridot
October 7, 2020

Hi,

I guess you didn't get my query. let me rephrase it.

 

--> I have a Stream X for all the assets data and each Asset will generate lot of data that will sumup the records around 10 million

--> Now I want to store each Asset's data to its corresponding Stream A for Asset A, Stream B for Asset B,etc rather than in a single Stream.

My concern is whether its a good approach to create separate streams for Assets for storing data OR single Stream for all assets data, when it is going to be huge data.

N
nmilleson17-Peridot
17-Peridot
October 7, 2020

@TanmeyTWX ,

 

All of the stream data goes into one table on the ThingWorx database (tables are not created per Stream entity).  Using one stream per Thing should in theory make your queries faster.  There are a few metadata columns which include things like timestamp, source, and tags. The source will be your Entity name.  Hope that helps.

S
slangleyCommunity Manager
Community Manager
October 16, 2020

Hi @TanmeyTWX.

 

If you feel your question has been answered, please mark the appropriate post as the Accepted Solution for the benefit of others with the same question.

 

Regards.

 

--Sharon

Terms & ConditionsAccessibility statement