Value Stream Query not fetching the records Always

[TanmeyTWX]

Hi,

I'm using QueryNamedPropertyHistory to fetch data from Value Stream and bind it to Mashup. But using same timestamp(start and end date) differently,causing partial data fetch.

Here is the scenario.

I have a data record at 2020-10-20 09:00:00.000

1.  Execute service with start date (2020-10-20 08:00:00.000) and end date (2020-10-20 10:00:00.000), timerange is 2 hours.

result = empty record for that Property

2. Execute service with start date (2020-10-20 12:00:00.000) and end date (2020-10-20 10:00:00.000),timerange is 10 hours.

result = expected record

so same record is being fetched when time range is more but no records found when time range is closer to the actual record timestamp.

Additional Info:

when using QueryNumberPropertyHistory with same date 24 hours timerange for that Property, able to see only 8 records in actual.

when using QueryPropertyHistory with same date 24 hours timerange , can see multiple records with repeated values and some blank values for that property

Is it a kind of expected behavior of Thingworx value stream query? Not able to understand. Please suggest.

[Ciprian-Traian]

Hello @TanmeyTWX ,

Have you tried to use the logged option for properties whose values update continuously ?

What database are you using ?

[TanmeyTWX]

Hi  Ciprian-Traian,

Properties are already logged only. some properties on Thing are sending continuous( every second) and some are like(every hour).

I'm using Postgres DB Thingworx

[Ciprian-Traian]

Hello @TanmeyTWX,

After using the QueryPropertyHistory service,it checks the data flow for each property in the Thing and collects all latest updates (each has a different updatetime) into one infotable result.

Split Things across several Value Streams to improve index performance.

You can also check ThingWorx subsystems in Composer > Monitoring menu > Value Stream Processing Subsystem > this subsystem is being used for ingesting data from your remote devices or business logic.

If the queueSize in the monitoring section is increasing constantly, it indicates that the system is not able to ingest the data.

[TanmeyTWX · Accepted solution]

Hi,

I have found the root cause.

Its the behavior of Value Stream Query. 

QueryPropertyHistory returns a data set created by querying each logged property on a Thing and then combining the individual datasets into one big data set. A "row" is created for each time stamp in the individual data sets. If there is a logged value with that time stamp, it is entered in the row of the data set. If there is no logged value, the last logged value for that property is used. If there is no logged property; the entry is left blank. This combiner process removes duplicates for a given property based on timestamp.