cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Individual streams per asset

SOLVED
Highlighted
Moonstone

Individual streams per asset

Hi,

Is it recommended to create stream per device? any major impact or performance issues if create a independent stream for each device Thing.

example: for 10k devices there will be 10K streams for data storage of a device.

 

Please revert your suggestions.

TIA.

1 ACCEPTED SOLUTION

Accepted Solutions
Highlighted

Re: Individual streams per asset

@TanmeyTWX ,

 

I don't think the number of Things (in this case Streams) will impact the performance, as even the Small ThingWorx sizing shows a limit of about 30,000 Things (see https://www.ptc.com/support/-/media/BC2F28281E4142AFA6FC79EC49D9DBD3.pdf?sc_lang=en page 12)

 

However, when your streams have over 100,000 data points in them, it's recommended that you use an external data store (see https://support.ptc.com/help/thingworx_hc/thingworx_8_hc/en/index.html#page/ThingWorx/Help/Best_Prac... under "Best Practices for Choosing Your Data Store Option")

 

Hope that helps!

View solution in original post

6 REPLIES 6
Highlighted

Re: Individual streams per asset

Hello @ TanmeyTWX,

 

The amount of historical data to be stored on the platform impacts both database and platform system sizing.  
Larger data sources (streams, data tables, and value streams)will require longer transactionsto query from the database.
These long transactions can also cause the stream and value stream queues to back up and utilize additional memory.

The ThingWorx Platform Sizing Guide it is a good reference to check in this situation.

It may be of help to check the Application Development Guide for ThingWorx 8 because it gives more details on

how ThingWorx handles data storage.
Choosing the correct data storage is very important since it affects the outcome of the project,its scalability and reliability, and the user experience.

 

 

Highlighted

Re: Individual streams per asset

Hi,

I guess you didn't get my query. let me rephrase it.

 

--> I have a Stream X for all the assets data and each Asset will generate lot of data that will sumup the records around 10 million

--> Now I want to store each Asset's data to its corresponding Stream A for Asset A, Stream B for Asset B,etc rather than in a single Stream.

My concern is whether its a good approach to create separate streams for Assets for storing data OR single Stream for all assets data, when it is going to be huge data.

Highlighted

Re: Individual streams per asset

@TanmeyTWX ,

 

All of the stream data goes into one table on the ThingWorx database (tables are not created per Stream entity).  Using one stream per Thing should in theory make your queries faster.  There are a few metadata columns which include things like timestamp, source, and tags. The source will be your Entity name.  Hope that helps.

Highlighted

Re: Individual streams per asset

nmilleson,

Thanks for writing back.

 

Can you tell me creating 20K STREAMS in composer will impact the application performance?

 

Highlighted

Re: Individual streams per asset

@TanmeyTWX ,

 

I don't think the number of Things (in this case Streams) will impact the performance, as even the Small ThingWorx sizing shows a limit of about 30,000 Things (see https://www.ptc.com/support/-/media/BC2F28281E4142AFA6FC79EC49D9DBD3.pdf?sc_lang=en page 12)

 

However, when your streams have over 100,000 data points in them, it's recommended that you use an external data store (see https://support.ptc.com/help/thingworx_hc/thingworx_8_hc/en/index.html#page/ThingWorx/Help/Best_Prac... under "Best Practices for Choosing Your Data Store Option")

 

Hope that helps!

View solution in original post

Highlighted

Re: Individual streams per asset

Hi @TanmeyTWX.

 

If you feel your question has been answered, please mark the appropriate post as the Accepted Solution for the benefit of others with the same question.

 

Regards.

 

--Sharon

Announcements

Thingworx Navigate content has a new home! Click here to access the new Thingworx Navigate forum!