delayed workflow with timer

[tcoufal]

Hi guys,

I am facing following scenario.

I need to create sort of workflow, which starts upon customer action (aka button) it will start certain process/remote test (started with REST Call - 3rd party system, no callbacks, no promises etc.) and after a while I need to start another service which than parses the result.

I wanted to do it with timer (set for 1minute, but that is not the important part).

The timer is set to:

Automatically Enable Timer on Startup - False

Thing is active - False

Enabled - Flase

User action will start the service, which makes 3rd party RPC (remote procedure call), then activates the Timer Thing, restarts the Thing (otherwise I have some errors....) enable Timer thing. In my another Thing I have a subscription to that Timer, (timer event) which makes the parsing and than disable Timer and inactivate it.

Problem is that sometime Event is fired at the same time as the TimerThing is Active - true (since I have my subscription on Timer event and not on Thing start, I would expect that it will wait the preconfigured time before event is fired).

Sometimes event is fired (at least the service in subscription says so) after 4 minutes instead of 1minute, and some time it is not fired at all.

What am I missing, is there any safe sequence how to start stop timer.

Is there more elegant way how to start some service, wait and start another?

Some non-blocking background style flow???

Thanks a lot

[mvonhasselbach]

Hi Tomas,

I had a similar challenge: I had an asynchronous backend system that didn't support callbacks. The backend system had REST services to start a job. This service returned a job number. Then there was another service to ask for the status for the job number. It returned a status (NEW, QUEUED, RUNNING, SUCCESS, FAILED) and in case of success you could get the result set with another service. There was no way to actively get a feedback message from the backend system when a job was done - I had to poll for the results from Thingworx.

My solution approach was a bit different than yours though:

• I used a combination of a Timer and a Thing derived from Stream (that I called JobQueuer) to capture jobs and their status and with services to add jobs and to check the status of jobs
  • The AddJobs service simply adds in entry to the Stream. It is called immediately after the job has been created. It uses a datashape with the following fields: jobName (logical name of the job), jobId, jobStatus and jobProcessorThingName. The later is the Thing that has the services to call the backend system. You may want to add additional fields, e.g. jobResultServiceName and jobResultServiceArguments as well as descriptive properties like jobDescription or the jobCreationTimestamp and jobFinishTimestamp.
  • The CheckStatus service is called by the timer in a fixed interval. It queries all non-finished jobs from the Stream with a query where jobStatus = [ NEW | QUEUED | RUNNING ]. For each of the returned jobs it calls the backend system by invoking the jobResultServiceName of the Thing with the name jobProcessorThingName with arguments jobId and potentially other args from the Stream entry. The status value of the backend system is checked and the Stream entries are updated accordingly if needed. If the status is SUCCESS or FAILED an event with name <jobName>_SUCCESS or <jobName>_FAILED and the jobId and other args is called on the Thing with the name jobProcessorThingName. I check if an event with this name exists before calling it. This event can then have subscriptions to it that can do error handling or call a next service and kick of a next job.

This design completely decouples the repeated polling of the jobs and the what should be done when a certain status is reached. What to do is completely implemented in the JobProcessorThing that creates the job, adds it to the Stream and has the events that get called back from the JobQueuer. The JobQueuer itself can be used with any (and multiple) JobProcessorThings. And as an extra feature you will get a Job Monitor as well: Simply look at the JobQueuer Stream content and you can see which jobs are running and how long they ran.

There is for sure room for improvement - I'm not sure what happens if the Timer interval is shorter than the time it takes to query the status of all running jobs but that's something I let you discover and handle if it causes issues.

Cheers, moritz

[tcoufal]

Interesting approach, but it wont help me in this case.

3rd party system does not return anything back (only "process started") since process can ran as long as 3 to 10 minutes I need to wait at least 10 minutes before I can call the parsing function, not sooner and it should not be later than that.

One question: is server side javascript blocking or non-blocking?

[ttielebein]

I am a bit confused here, first of all, are you using Workflow Builder (from ThingWorx Utilities) or do you just mean a workflow in general? Secondly, you say that an event is firing at the same time as the timer is active... why is this a problem? The event turns the timer off, is that the problem? Can you break down what you are trying to do with more detail?

Here is my understanding so far: you have some service that you want to run, and when it is done running, you want another service to run and process the results? Why not just have the first service trigger an event at the end and then subscribe to that event with the second service? Why use a timer at all?

[tcoufal]

Hi,

first of all when I say Workflow I mean it in general. (We dont have TW Utilities).

Second, external service cannot trigger anything, it can be only started remotely (via RestCall) and service (its a specialized test) can take anything between 2 and 10 minutes to complete. I cannot be informed when its done.

So What I need to do, is to start this service -> wait for at least 10 minutes -> parse the result (via RestCall), thats it.

I thought that I would create this "waiting period" with Timer, which I would be Turning On and Off. The Timer is set to fire each 10 minutes, I have my subscription which should run the parsing service, but when I activate the Timer, event is sometimes fired immediately (it does not wait 10m), thats a problem, I need to be sure that service is called with the delay, not sooner.

I hope I made myself bit more clear now.

Tomas

[ttielebein]

Ok, this helps a bit, yeah. So what is this external application? Is this something you own? You could have your external application update some property on the ThingWorx server (via REST API call to ThingWorx), and then subscribe to that property via DataChange event, thereby eliminating your dependence on timers.

If this does not suit your use case, then yes, timers it will have to be! So, now that I understand the context, let's clarify on the issue you are seeing with timers. Why not just have a timer run every 10 minutes which checks to see if the results are done, and if they are, parses them? This way you don't have to worry about turning it on and off.

Oh, or are you saying that you only need the timer to be active once you trigger the external service call? If that is the case, then what you are doing (activating the timer) makes sense to me. Are you saying that when you activate the timer, sometimes the timer event is triggering your subscription to process the external data before 10 minutes has passed? That is definitely not correct. If you tell me your version of ThingWorx, I can investigate this or bugs.

[tcoufal]

Hoy Tori,

That external application is made by our partner, cannot be changed easily. Does not make any calls to TW platform (for security reasons).

Target system does network scan and creates several json files on the fly, those files can be consumed by REST api which is built on top of that system.

I need to ensure that the scan is really through (finished, done...) I can still consume partially formed json files, but I have to avoid that.

I cannot run timer every 10minutes, that is why:

I manually start my remote test at time 6 and it takes 8 minutes to finish (only than I can consume the output via REST call).

And My timer fires at time 10 and tries to consume unfinished results because they will be ready on time 14.... There is no flag which indicates that test is done, because they are always previous results (they dont interest me).

About that Timer issue,

I have my timer,

Then I have my Thing which has a subscription to that Timer

When I activate this timerThing via script it fires TimerEvent immediately, any other after predefined Time. That is the whole problem.

So I am working around that a bit, by adding property to my TimerThing called iterations (default value = 0), Timer is subscribed to itself and it increments this property.

Any other Thing which is subscribed to that timer also checks for number of iterations:

if ([Timer].iterations) {

    [Timer].iterations = 0;

    [Timer].disable();

     myDelayedService();

}

I am using 6.5.3 and 6.6.2

[ttielebein]

Yeah, the timers fire immediately upon being activated (or upon server restart for already active timers), and I did confirm that this is the expected behavior, but I think your workaround sounds good.

[tcoufal]

Ok, got it.

Thanks..

Tomas

[CarlesColl · Accepted solution]

Hi Tomas,

You can let Timer run every 1 minute, and have a property which you set when you want to start counting down. Something like this:

• countdown = -1 --> doesn't counts down
• countdown >0 --> countdown-- ( when event it's triggered )
• countdown = 0 --> throw a custom event which will be the one you will subscribe to do your custom parsing.

[tcoufal]

Thanks Carles,

helpful as always, I've added a queueing mechanism so now all my things can use it. Queue is type of Infotable which contains ThingName to which the notification belongs and initial countdown (delay period). In subscription a simple comparison is used to find out if that event belongs to that thing.

I want to add a dynamic subscription option which would delete itself after event being handled, so there is no subscription in my system that does not need to exist, but I dont know if that will make any significant savings.

Thanks