I'm trying to load historical data into Thingworx and tried several approaches but still can't get the desired result and/or performance
So, I have a bunch of data (~30 devices, ~40 sensors per device, ~120 000 data points per sensor, with timestamps).
I'm using ValueStreams to store data.
The first thing I tried is just to use PUT with "https://<thingworx>/Thingworx/Things/<thingname>/Properties/*". This way I can load all 40 sensors woth a single REST request, and Thingworx consumes something like 150 PUT requests / sec. Works like a charm and quite fast for a laptop VM, but I couldn't find a way to pass my datetime values into timestamp and timestamps I'm getting are as of when data was loaded. I can create a separate parameter, like 'event_time', but I'd have to write wrapper services for timestamp-based Thignworx OOTB services, like QueryPropertyHistory, It looks like a kludge / workaround / hack to me, rathert than a feasible solution, and I'd prefer to avoid doing this.
I could probably update timestamp after loading the data, but, again, couldn't find a way to do this.
So I tried using AddNumberValueStreamEntry. It has the 'timestamp' as one of the input parameters so I can put my datetime there. But I couldn't find a way to load data with this service other than by calling it for every single sensor datapoint, so to load entire dataset I'd have to make 120 000 * 40 REST requests and it's really slow.
Well, after all, it works, and I can probably live with this, but there must be a better way for sure?
There is AddStreamEntries service, but it's not available for ValueStreams, only for Streams.
I could load data directly to the DBMS of choice (I'm using Influx), but it's usually a frowned upon way and I guess not supported by PTC.
What could be done about this? Loading historical data looks like quite a common task as I see it. Are there "best practices for this? Do I overlook some approach or a service?
A bonus question - I'm not really a seasoned Thingworx guy so I wasn't sure when choosing ValueStream vs Stream for storing data. Is there a simple Yes/No answer to whether choosing ValueStream was a good decision or not, or that's a mattert for a separate post / topic?