Jenkins Integration Plugin and checkpoint with custom labels

[spliefke]

I started to use Jenkins to impement a nightly build process for our software components. Therefore I created a Jenkins Pipeline item that checksout, builds and checkins the build-results. The final step would be to checkpoint the complete project incl. the build result files. Is there a way to customize the checkpoint label (that consists of date and build numbers) and a way of applying the label to all members.

[kthierer · Accepted solution]

Surprisingly for 'si checkpoint' there is no documented flag for "Apply Label to all Members"

like the corresponding dialog from the gui.

If I would not have that flag in the gui I would try

1.) Checkpoint the project eg. with label 2016_12_06_BUILD1

2.) Determine the the Project revision that was created (the one with label 2016_12_06_BUILD1)

3.a.) Retarget the sandbox to that project revision (this only works if your checkpoint source was from a root sandbox)

or

3.b.) View/Open the created build project from the project history view

4.a.) Apply Member->Properties->Add Label to the retargeted build sandbox.

or

4.b.) Apply Member->Properties->Add Label to the opened Build Project

The b. Variant is more generic but I guess it's harder to translate into cli commands.

(you will likely have to deal with exact repository location resp. configuration paths on the server side)

From a sandbox perspective my approach for 1.) (without concern for change package issues) would be:

    c:\prj1_sandbox>si checkpoint --label=2016_12_07_B111 --description=JENKINS_BUILD_111

This will create labels only on subprojects.

For a retargeted build sandbox my approach for 4.a.) would be:

    c:\prj1_sandbox>si addlabel --label=2016_12_07_B111 --recurse

This will add labels to the revisons of the corresponding build project too.

Regards

   Juergen

BTW: If you find a solution how to translate this all into cli commands in the context of the jenkins configuration, I would be interested too

[kthierer]

It just came to my mind that beyond the technical challenge of this problem the workflow might just not be "compatible" with the philosophy of Integrity.

This is especially a problem if you consider (or have the paranoia) that while you are checking out other people may check in, change member revisions ...

If you can't exclude this for a nightly build the workflow should definitly start with checkpointing

and the build should then take place on that checkpointed build project resp. build sandbox.

Even if you can exclude the nightly build danger, it could be an issue if builds are triggered manually from

the Jenkins web interfcae.

[KaelLizak]

The information Klaus Thierer provided above is accurate, and should provides what you have stated you are looking for.  I'd like to know why, though: What is the purpose behind this process that you're using? 

Also, What do you mean by "build result"?  Do you mean you're actually checking in binaries?  What's the business case for that?

It depends on the build you're using, but why don't you create a build sandbox as of (it's functionality that's in the CLI as of 10.6).  See CS86259:  Create Integrity Source build sandbox as of a certain date/time without a Checkpoint‌.  If that build sandbox passes, you can then do a checkpoint as of.  It might save you some pain, and avoid someone checking something in while you're in the process of making your sandbox.

You should probably also be made aware that labelling members is a very poorly performing command.

Regards,
Kael

[spliefke]

Thx for the information already given to me. You maybe noticed I'm new to the topic and it is difficult get information about connecting PTC and Jenkins.

The integartion responsible collegues are aware of the issue that maybe checkout and checkpoint could potential differ when someone commits in between. Currently we are avoiding that by starting the workflow at 2-3am in the morning. It take only couple of minutes to finish it.

I guess I will rethink that paradime and migrage to the process of checkpoint while checkout. This is also because I'm planning to start some algorithm test scripts which maybe take a while to process. At the end it will increase the risk of getting out on sync. It's the most safest solution and is provided by the jenkins PTC integration. The down side would be that all build results (binaries - mostly integration of the component into a development framework tools and test reports) will be not part of the checkpoint. Any Idea how I could link these results best to the checkpoint version I created at the beginning? Maybe applying the same label knowing that labels can be moved later on. If so how could I readout the label that was set by the checkout cmd at beginning and apply it via siaddprojectlabel when I'm chekcking in the resulting files.

Regards

  Sebi

[kthierer]

You might use cli commands to generate INTEGRITY context information for that build in an extra file eg:

    c:\myproject_sb\dev>si sandboxinfo > %JENKINS_ARTEFACTS_DIR%\INTEGRITY_SB_INFO.txt

    c:\myproject_sb\dev>si viewsandbox --recurse >> %JENKINS_ARTEFACTS_DIR%\INTEGRITY_SB_INFO.txt

(You could also add other useful information like the build log, map file, lint checks ... to the artefacts directory.)

Still if you now like the idea of precheckpointing it can cause confusion if the binary would be checked in within the directory tree of dev.

Maybe it would be good to seprarate the jenkings build process from the checkin of the binary resp jenkings artefacts.

Also after thinking about Kaels concerns about checkin in nightly build results, I would at least put a human factor before

the checkin.

One idea could be to dynamically generate and check in a batch file

  c:\myproject_sb\test\get_latest_jenkings_artefacts.bat

which downloads the latest jenkins artefacts eg:

  jenkins_server:8080/job/my_project_dev/123/artifact/jenkins/*zip*/jenkins.zip

and extraxcts them to the folder jenkins_123 (123 == jenkings build number)

The artefacts (binaries...) could then also be checked in/added the next or another day by a tester

after he resynchronizes c:\myproject_sb\test\get_latest_jenkings_artefacts.bat, executes it does the specified tests

and adds the test reports.

For this approach it would make sense for jenkings to create two sandboxes within the nightly build.

1.) a regular like c:\myproject_sb\test\project.pj

2.) a build sandbox like c:\myproject_sb\dev\projext.pj (1.2)

BTW:

I never tried such a procedure, its just brainstorm

For me the frequent use of labeling members in the context of checkpointing is not only a performance issue.

(Thanks to Kael I know know that it can be)

Some times I feel like it falsifies the picture, since members with small activity will accumulate

many labels on one revision making them look more interesting as they probably are.

From the member history view perspective it is often more helpful to configure the view to display

the checkpoint labels (a list of labeled checkpoints the revision participates in).

But even that may feel overpopulated for your case after a year of nightly builds

You would still have the possibility to delete uninteresting project labels but you should do that recursive (with subprojects)

in the context of the checkpoint that created them (checkpointing labels are always applied recursively) and as far I know 'si deleteprojectlabel' does not support this.

You would have to find a way to automate this or do it manually.

Regards

     Juergen