detach a long running process from the trigger context that started it

[mrump]

Hi all,

I'm looking for a idea how to prevent my integrity server from being "blocked" by along running process during a scheduled trigger.

Use case:

I have a schedule IM trigger that runs every midnight. It shall checkpoint a certain number of LARGE SI projects, so our developers can at least "step back" on a daily basis.

Currently I use the SI project bean to create the checkpoint, which generally works fine; but

as long as the checkpoint process is running, no other IM trigger can be processed .

So i wonder if there is a way to "fork" the checkpoint process from the trigger script.

I do not need any response value, as I simply trust in Integrity to do it's job .

I've tried to start a separate OS process and using the command shell API to to the job.

Unfortunately this requires an installed Integrity Client on my Server machine.

I appreciate any idea.

[matt_giltaji]

Hi Matthias,

I haven't actually tried it, but it looks like you can run the checkpoint command from a webservice, so that wouldn't need either a scheduled trigger or the installed integrity client on your server machine.

You would still need to set up something to make that webservice call nightly.

I'm sure you also know that even though using some other call might make it so your IM triggers are not stuck, running those checkpoints will likely put a strain on the DB so the IM triggers which run while the checkpoint is being taken may process very slowly.

Also, I heard long ago that there is an RFC to allow taking a checkpoint of the past state of the SI project, so you may want to get added on that RFC as it would also solve your issue. Unfortunately I don't remember any RFC number for it

Hope that helps,

Matt

[mrump]

Hello Matt,

thanks's for the quick response.

I thinks give it a try like this:

1 implement a small java app using the WSDL that fires a webservice request for a checkpoint and simply ignores the response

2 add the class to the classpath that can be used within the trigger scripts

3 inside a JS trigger script instantiate the class and run it

Did I miss a step?

I promise to post my result here 😉

Regard Matthias

[matt_giltaji]

Hi Matthias,

Your approach sounds correct.

I didn't even think of using the IM trigger to make the webservice call, I was thinking similarly to Heike where a scheduled task in the OS would kick off the app to make the webservice call.

I am looking forward to your results

Regards,

Matt

[hsprang]

Hello Matthias,

when reading your use case I wonder why you use a *trigger* for creating the chekpoints and not the command line interface.

If creating the checkpoint in the command line interface you have to create a schedlued task in your operating system that starts the script, and you have to install the Integrity Client on the server, but this to me is the only disadvantage to that solution. Such a script is even much more easy to maintain and debug than triggers and don't affect system performance too much.

Regards

Heike

[mrump]

Hello Heike,

thanks's for the quick response.

You are right, that a schedule OS task would do the job. We tried several things in the past and all come with pro & contra.

But in general I do not like mixing client and server on the same machine (some bad experiences in the past).

Furthermore you still have to maintain

1. the scheduled OS task

2. the script you run

3. the client installation

no matter where you install the client.

To be honest the scheduled trigger is not the only use case I'm thinking about:

e.g. I could imagine a automatic checkpoint of a source project when the referencing IM project backing item reaches a certain state (or "process gate") based on a rule based IM trigger.

Things like this require the ability to detach the trigger from the process it call's as you can never be sure how long it might take.

Regards Matthias

[mrump · Accepted solution]

I finally found a maintainable solution that offers

- quick response for the Server-Trigger Script

- a stable environment

- flexible batch files.

I 've set up a Jenkins CI server that performs the actual operations (no matter how "long runnnig" or  "resource hungry" they are ).

The Integrity trigger script simply analyses the event, gathers the necessary parameters and triggers the Jenkins Server's appropiate Job via a http request.

Using this approach I can run batch operations that may take a hour to proceed while the Integrity trigger terminates in milliseconds.

Thanks to the Jenkins I do not have to worry about disk space (workspace cleanup) and I also can review the Log for all Job run.

[efc4]

I'm also interested to send http request (POST) to another server, but I don't find the right way.

What are you using to send the request ? XMLHttpRequest is unknown...

[mrump]

Hi @efc4 ,
sorry for the late response.

I for this specific case I used Powershell to create a WebClient Object (needed for the authorization inside our network) and then the DownloadFile method to trigger the Jenkins.

"powershell.exe -command \"$client = new-object System.Net.WebClient;$client.Credentials = New-Object System.Net.Networkcredential('BUILD_USER','PASSWORD');$client.DownloadFile('"+urlString+"' , '"+downfile+"' );

Note that the temporary downloadfile name needs to be unique for each trigger, to avoid conflicts with parallel trigger execution.