How get Stream Entries by Timestamp

[rnfonseca]

Hello everyone,

I've been trying to invoke a service from a Stream, in order to retrieve data based on a DATETIME.

However, the only functions I see available retrieve the full Stream data. This is something I'd like to prevent since we are aiming to store millions of records on this stream.

Is there any way to query this data from the Stream with the out of the box solutions ThingWorx provides?

I'm using ThingWorx v9.0

[rnfonseca · Accepted solution]

Manage to find the 'QueryStreamEntries' services by default on the Stream

[VladimirRosu]

I saw there the statement "we aim to store millions of rows in this Stream".

Is that a Stream or Value Stream?

They behave quite differently on disk, even if they share a relatively similar ingestion mechanism.

Before you go ahead and store this amount of data on disk, I strongly suggest making a dry run test, that would also include a number of your projected users and their queries on that stream.

Have you done a calculation estimation to see how many rows you will store, and for how long ?

Hint: there is an article published here that touches the testing subject, https://community.ptc.com/t5/IoT-Tech-Tips/Announcing-the-Final-Installment-JMeter-for-ThingWorx/m-p/694477#M1711

[rnfonseca]

Hello Vladimir,

We aim to use a Stream for this matter. We expect to store roughly around 5 million records for over one year period. We did some rough testing to try to understand the performance when querying this data. Namely understanding how the QueryStreamEntries services behave when executing queries with a timestamp filter.

Will JMeter allow me to stress test my solution (simulate data, and make concurrent calls to my services) in an easy manner? We have been looking at how we could achieve this.

Best regards,

[VladimirRosu]

Hi, yes, absolutely. In that article there's collection of other articles that would allow you understanding how to setup jmeter for this task.

The end-purpose of such a Jmeter test would be to simulate a number of users and see how the system behaves on that load.