cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Asyncronous Streams and Delayed Calculations Execution?

Peridot

Asyncronous Streams and Delayed Calculations Execution?

Hi All,

I'm importing on batch old data ( in blocks ), this data it's automatically stored on a Stream ( either Value Stream or Stream ), and I need to do a calculation based on current value and previous value (accumulated consumption counter). I don't have previous value in memory, hence I have to go to the Stream and get previous value, and here the async stream problem arises: if I do too fast the previous value ( which I'm importing in batch "super fast" ) won't be stored on the Stream it's on the write queue.

As I control Data Change event on the property ( accumulated consumption counter ), I would like to execute a delayed Data Change event, there's an easy way to do it? All the ways I think of are too much or I don't like it:

  • Build a Delayed Events queue, which will execute events after a while. I think this is the best way, but It's a lot of work to build it correctly, also I will need a timer to trigger events once in a while and this will add load to the server,...
  • Do the calculations once a day . I don't like this solution, as not always I will load data on batch, for the same property on another Thing may I have "real time" data and I want on that case that the calculations are also in "real time"
  • Delay a service call ( something like setTimeout() ) --> I think this is not possible on TW? maybe a combination of Async Service call plus a pause(), but I don't know the performance impact of such a solution on the TW Threading model, something like this
    • Async Service:  void SetTimeout(thing,service,params,milliseconds) {

         pause(milliseconds);

         Things[thing][service](params);

      }

Best Regards,

Carles.

5 REPLIES 5

Re: Asyncronous Streams and Delayed Calculations Execution?

Thingworx supports the pause service

basically

pause(1000) would be pause for 1000 miliseconds.

Re: Asyncronous Streams and Delayed Calculations Execution?

Hi Pai,

Yes, already it's on my proposals, but the question about using pause(n) it's how will this affect on the threading model... I mean, let's say I import 100k registers in batch, which in turn does 100k SetTimout ( function as shown above ) what will happen to TW?

Best Regards,

Carles.

Re: Asyncronous Streams and Delayed Calculations Execution?

I'm doing test with the fake SetTimeout function ( which works ) , first limitation: pause(n) it's limited to 60000 milliseconds . Yes I know I can iterate over pause(n) calls, but it's awful.

Re: Asyncronous Streams and Delayed Calculations Execution?

Hi Carles,

Is there a reason that you are not keeping the previous value you need in memory?  Since I do not know the full scope of the transactions I was not sure if the data value was large or some further reason.  If you can store the previous value it should eliminate the need to pause?

Thanks,

Adam

Re: Asyncronous Streams and Delayed Calculations Execution?

Hi All,

This post got lost on the community ( it went to another place instead of Developer Community ).

Just for the record, and if someone has the same needs, or maybe anyone can think on other options...

  • Build a Delayed Events queue, which will execute events after a while. I think this is the best way, but It's a lot of work to build it correctly, also I will need a timer to trigger events once in a while and this will add load to the server,...
    • I followed this approach:
      • Created a DataTable where Events are pushed with adding a Timeout, and with a timer I trigger delayed events when Timeout it's reached.
  • Do the calculations once a day . I don't like this solution, as not always I will load data on batch, for the same property on another Thing may I have "real time" data and I want on that case that the calculations are also in "real time"
  • Delay a service call ( something like setTimeout() ) --> I think this is not possible on TW? maybe a combination of Async Service call plus a pause(), but I don't know the performance impact of such a solution on the TW Threading model, something like this
    • I Tested this approach, but it ends ups consuming lots of RAM, as it creates a new Thread for each async SetTimeout call. If someone want's it, here it's, it can be useful but not for lot's of calls ( 10k calls = 1Gb RAM ),

async function SetTimeout(thing,service,params,milliseconds) {

if (milliseconds>=60000) {

    var nTimes = parseInt(milliseconds/60000,10);

    for(var i=0;i<nTimes;i++) pause(59999);

    // -- As the following instruction doesn't works and it should: case: 12808137

    // -- we will just trigger half a minute more

    //pause(parseInt(milliseconds%60000,10));

    pause(30000);

} else {

    pause(milliseconds);

}

if (thing) {

    Things[thing][service]((params?JSON.parse(params):null));

} else {

    me[service]((params?JSON.parse(params):null));

}

}

Best Regards,

Carles.