Question about the Article - CS204091 Difference between Streams,Value Streams and Datatables in ...

[seanccc]

Hi, 

I'm reading the article https://www.ptc.com/en/support/article/CS204091?&posno=1&q=204091&ProductFamily=ThingWorx&source=search  , but feel a little bit confused about the phrase "Why is there a performance hit at 100,000 records?" . 

The phrase has two statements: 

1. If using a Stream to store more than 100,000 records and the query or get that is being exercised on that Stream to return data always has the timestamp fields filled out there should not be a huge performance hit because, only a specific number of records will be returned for that subset of time that is specified in the initial query based on the timestamp
2. The key here is to utilize the default timestamp field when adding records to the Stream and not attempting to query on a custom timestamp field within the query parameter. This is because the initial pull of data from the Stream will return the entire Stream instead of grabbing a subset of records based on the default timestamp field and that is where the performance hit will be seen

#1 said it's NOT a huge performance hit as the initial query based on the timestamp returns the subset of time, 

but #2 said it's a performance hit because initial pull returns entire stream instead of subset even if the query based on default timestamp.

So,  is it a performance hit or not for the query on stream based on default timestamp ? 

Regards,

Sean 

[CarlesColl · Accepted solution]

Hi Sean,

If you query by the default timestamp field there isn't "any" performance hit, at least with PostgreSQL, as it's indexed by this field and only returns the desired data. 

Regards,

Carles.

[seanccc]

@CarlesColl ,

Yes, I have the same understanding with you .   that's why I feel the article is a little bit contradictory between #1 and #2.   If thingworx does use the indexed default timestamp field  instead of loading the whole stream  entries, then it should not be limited by 100,000 records.    Thank you . 

Regards,

Sean

[CarlesColl]

Yup, I have millions rows on Streams and ValuStreams, no issues if you use timestamp query.

[slangley]

Hi @seanccc.

If you feel an article is misleading or incomplete, please feel free to submit your feedback via the button as seen in the screenshot below.  Please provide specifics on the information in question so our engineers can re-evaluate the content.

Regards.

--Sharon

[seanccc]

@slangley ,

yes, I just sent my feedback in the article page. 

Regards,

Sean

[slangley]

Hi @seanccc.

Aside from any article updates, if you feel your question has been answered, please mark the appropriate reply as the Accepted Solution for the benefit of others with the same question.

Thank you for your participation in our community!

Regards.

--Sharon