Abort WVS custom jobs from WVS Job Scheduler Administration

[MH_10463676]

Hi,

we think about generating custom jobs to create viewables for latest items that don't have one. Alone the "geneate jobs-job" runs a couple of days (testsystem) and would block users from productive work as the cad worker runs full.

Is it possible to abort these jobs?

Our plan would then be something like:

1. Stop publishing queues
2. Run latest-no-viewables-Job
3. Abort latest-no-viewable-Job (after e.g. generation of 5k jobs in the queue)
4. Filter queue (remove .drw, .asm etc.)
5. Start publishing queues

Regards

[avillanueva]

Do you have these custom jobs implemented?

https://www.ptc.com/en/support/article/CS211115?source=search

These are great for assisting with what you are asking. You should be able to change filters until you get to a point where you can get the drawings and assemblies done. This will avoid too much of manually removing jobs already queued up. It is really easy however to delete jobs using provided table filters. however there is a table limit of what is displayed. 5000 would far surpass that. You can also take the approach of product by product scheduled jobs to batch things. There is also a CSV file if you need to tailor it using a defined list.

I myself made use of the last option with publish actions on the search results. This way I can search for latest items and define via a search query, Select only the number I want to publish in that session and submit them. I then control what is publish and how many will queue up. 

[MH_10463676]

Hi,

thanks, I bookmarked the article. But we currently try to achieve a few things ootb (<-> without customization).

Regards

[avillanueva]

In that case, to your original question, it should be possible the way you described. You will see the queue jobs fill up but nothing will be processed until you turn them back on. I am not 100% sure how responsive it will to your request to abort the generation of jobs. Any idea of how many you will get? It will only increase your effort in reducing the jobs to a neat, nightly batch. Those additional jobs are really useful but I cannot find fault with your approach. 

[MH_10463676]

In at least one context we speak about 120k+ items. Queuing-job (test-system) is running since 5 days and ongoing. So that's not really handy in the prod-system 😉 

[avillanueva]

I see your problem. If you have the resources, you can consider scaling up the number of workers and queues for the overnight period. Anything that you can do to boost your publishing rates.

[mmeadows-3]

Scaling up is a huge part of the recommended solution for 120k jobs.  There are only so many jobs one publisher can push through in a day.  It has been running 5 days.  How many of the 120k jobs were processed in the 5 days?  Or (if keeping the successful publish job logs - wt.queue.removeCompleted.PublisherQueue#=false) what is the average time to publish a part, assembly, and drawing?

Maybe I missed it, what is the CAD system you are publishing: Creo, Solidworks, Inventor, ...?

Guessing your company has already limited publishing according to what the company needs? For example, don't publish everything on check in.  Only publish at specific life cycle states and/or on-demand.  Leverage positioning assemblies and extended positioning assemblies to speed up assembly publishing.

If you don't have the resources for additional workers and this is a one-time publish task, consider also tweaking the properties for job priority.  Batch submission jobs remain low priority.  Manual submissions become high priority.  Other forms of submission become medium priority.  The queues may be overloaded for a few weeks, but it won't impact daily submissions (assuming more jobs are being published than are submitted on a daily basis).

publish.publishqueue.priorities.<jobtype>.<jobsource>=<H, M or L>

The possible Job types are:

0-default/unknown type

1-publish of CAD data from an EPMDocument

2-publish of a file from a WTDocument

3-publish of a WTPart structure

4-publish of an EPMDocument from client side viewables

5-create representation from a local CAD file

6-create representationfrom a local WTDocument

7-create representation from preconverted local data

8-create representation from the visualization collection

9-create representation from loader using ticket file (not used in this context)

10-publishjob to perform only mark out of date for an EPMDocument

11-publish job to perform work package file sync for an EPMDocument

The possible Job Sources are:

0-default/unknown/any source

1-manual publish from the UI

2-event initiated publishing, eg from a WGM on checkin

3-publishing from a schedule publish job

How many workspaces have accumulated under your publishing account?  The more publishing workspaces that exist, the longer it takes to register the server for each job.  It can really have an impact if you have a couple hundred publishing workspaces.  Eventually publishing will fail due to taking too long to register the workspace.  We schedule daily/weekly removal of the CAD worker account's workspaces.

There are many more publishing performance tips depending on the CAD system and worker configuration, but these are the big ones.

[MH_10463676]

Hi,

CAD-system is Creo. We run our system, including the CAD-Worker, in the Azure-cloud - 3 server with currently 5 CAD-worker and 6 publishing queues. We might scale this up but would like to stick with the 3 server at the moment and optimize the system.

We currently don't publish - we just do a job-generation-test with the WVS Job Scheduler. This job is already running 10 days now and found 160k+ "Latest"-items without representations - in one of several contexts. We would only publish .prts which take in average 10sec/file.

This is a preparation for Position Assemblies. As soon as we decide to activate Position Assemblies, it would be very frustrating for the user if half of the assemblystructure is not visible.

I like the idea of lower priorization of the scheduled jobs. Anyhow, we need to filter out especially .asm and .drw. And I think that's either an manual task or customization, correct?

Is the priorization possible ootb or customization as well?

[mmeadows-3]

The properties based prioritization described above is OOTB.  I am not aware of another method to prioritize outside of customization (e.g. workflow robot or custom filter).

There is a job filter 'customization example' to filter by file extension that ships as part of Windchill.  It comes pre-compiled and should be available for use, even in PTC's cloud environment.

https://www.ptc.com/en/support/article/CS39455

Going this route, enable the filter and set it to just 'PRT'.  Then queue up all your CAD Document parts to the low priority queue.  When finished, remove the properties or set it back to 'ASM, DRW, PRT'.  Since the functionality is driven by properties, it will most likely require Windchill restarts.  I don't believe publish.cadfileextensionfilter outside of a Windchill restart.  PTC really should update this to be preference driven...

If restarting Windchill is not an option, Publish Rules can be used to filter out everything except *.prt files that are not skeleton models.  It can be accomplished by (regular expression) naming convention...

<condition name="Valid_Creo_Type">
	<and>
		<attribute name="epmdoc_CADName" regex="(.*\.(prt|PRT)"/>
		<not>
			<attribute name="epmdoc_CADName" regex=".*\_skel.(prt|PRT)"/>
		</not>
	</and>
</condition>

Or by CAD Document type and subType...

<!--  Creo objects that are allowed to be published all the time -->
<condition name="Valid_Creo_Type">
	<and>
		<attribute name="epmdoc_docType" value="CAD Part"/>
		<not>
			<attribute name="epmdoc_docSubType" value="SKEL_MODEL"/>
		</not>
	</and>
</condition>

Either of these should work.

We can also filter on additional attributes: epmdoc_lifeCycleState, epmdoc_docType, epmdoc_containerName, epmdoc_docSubType, epmdoc_containerName, epmdoc_folderingInfo, etc.

<attribute name="epmdoc_lifeCycleState" value="Released"/>

In either case, this should be done during off-hours to ensure end user submissions of assemblies and drawings aren't excluded from publishing.

[MH_10463676]

Interesting approach. Can this possibly be restricted to certain user-accounts? In that case we should be able to let it "just run" in the background even during office hours as normal users should still be able to publish and release as usual.

[mmeadows-3]

Publishing rules operate off of how a job is summitted, not who submitted it.

<if condition="Valid_Creo_Type">
	<!-- Publish on Check In -->
	<!-- <publish on="checkin" display-label="CREO Part, upon check in, with additional files" additional-files="parametric_part_files" /> -->
	<!-- Manually publish from details page -->
	<publish on="create-representation" display-label="CREO Part, manual submit for publish, with additional files" additional-files="parametric_part_files"/>
	<!-- Publish via publish scheduler -->
	<publish on="schedule" display-label="CREO Part, publish scheduler, with additional files" additional-files="parametric_part_files"/>
	<!-- Publish via workflow, state change, etc. -->
	<publish on="unknown-source" display-label="CREO Part, workflow or listener, with additional files" additional-files="parametric_part_files"/>
	<!--Enables UI pick to Post Publish from details page Representations listing -->
	<publish on="manual-post" display-label="CREO Part, manual content Post Publish, with additional files"/>
</if>

Comment one out to disable that form of submission (i.e. don't publish on check in).

The job priority is also set based on how the job was submitted (e.g. batch job vs. check in vs. manual submission).

So, define conditions for your batch submissions (e.g. parts only).  Then only publish batch submitted jobs (i.e. "schedule") using that condition.  Define a second set of conditions for non-batch submissions (e.g. parts, assemblies and drawings).  Publish each of these accordingly, excluding the batch submitted jobs.

[mmeadows-3]

To clarify, did you say it took the better part of a week to queue up 160k CAD Documents for publishing?  Or that is how many it has processed in a week?

If it took a week to queue up the jobs, there is a database performance issue.  Please take time to tune your test system.  It won't run as fast as production, but 160k objects should queue in a few hours at most.

I usually get 7-12 sec. per Creo publish job depending on complexity of the geometry, hardware configuration, Windchill/Creo configuration, etc.  I usually don't mention shaving off a second or two, but with 160k jobs, one second adds up to ~3.7 days across 5 CAD Workers.  Review the completed jobs to understand where they are spending their time.  A very simple part example:

[MH_10463676]

Yes, the time spend is only for queueing. We run the "PublishAllLatestEPMDocumentsNoRepresentation"-Job. I'll contact my colleagues if there is an issue with the database and if it can be speed up. Do you have a recommendation/idea where to start?

After all there is a huge difference between your expectation (couple of hours) and reality of 10days counting.

An average Creo-job is similar to yours in timing. More complex parts take a bit longer, over a couple of thousand items we end up at ~10sec avg.

[mmeadows-3]

SQL Server I'm assuming.  Run SQLPerf and fix everything it identifies.

https://www.ptc.com/en/support/article/CS299797

Oracle analysis is also possible.

https://www.ptc.com/en/support/article/CS271394

Then run an On Demand System Scan and continue fixing things that look important/relevant.

Instructions: https://www.ptc.com/en/support/article/CS299230

Upload site: https://www.ptc.com/en/support/on-demand-system-scan/

If it still runs poorly, please use the System Configuration Collector to gather performance related information and log a call with the PTC performance queue.

[MH_10463676]

Thanks for the infos! We'll give it a try and see if we can optimize our system based on the current infrastructure.

[mmeadows-3]

An update on epmdoc_docSubType...

Somewhere between Windchill 11.1 and 12.1, PTC excluded the docSubType from publishing.  To get it back, remove docSubType from the property edrload.propertyskiplist, set in wvs.properties.

edrload.propertyskiplist=iterationInfo classInfo cabinet conceptualClassname persistInfo attributeContainer entrySet lock versionIdentifier hasContents versionLineage class cabinetReference domainRef iterationIdentifier contentVector format typeDefinitionReference locked latestIteration inheritedDomain lifeCycleAtGate checkoutInfo branchIdentifier versionDisplayType oneOffVersionInfo oneOffVersionIdentifier displayIdentifier displayType type state identity extentsValid folderPath master versionDisplayIdentifier lifeCycleTemplate controlBranch primary teamIdentity teamTemplateIdentity generatedFields containerReference container organization organizationReference template creator modifier location masterReference organizationUniqueIdentifier organizationCodingSystem teamId teamTemplateId effVector federatableInfo httpVector indexerSet locker operation ownership view boxExtents docSubType versionDisplayIdentity eventSet iterationNote lockDate lockNote lockerEMail lockerFullName lockerName authoringAppVersion authoringAppVersionReference dbKeySize familyTableMaster familyTableMasterReference familyTableStatus featureValues parameterValues referenceControl parentFolder securityLabels securityLabelsCache rootItemReference interopInfo asStoredConfigReference asStoredConfig typeInfoWTPart roleBObjectId roleAObjectRef roleANonpersistentProxy roleBNonpersistentProxy roleAObjectId roleBObjectRef roleBObject roleAObject

Publish a sheet metal part to confirm the attribute is now visible to Creo view.

To use epmdoc_docSubType, the value set in the publish rules should match the display name.

For example:

<attribute name="epmdoc_docSubType" value="Sheet Metal" />

[HelesicPetr]

Hi @MH_10463676 

I can recommend you if you have 5 workers create 10-15 publish queues. 

I have experience that the number of queues should be 2 - 3 times more then workers for better performance.

There is huge performance progress. Because the jobs are better distributed to the third, fourth and fifth workers. 

if you check visualization statistics you realized that last worker processed less jobs then the first one. 

This is my long term experience with visualization and more workers. 

One time I use 12 workers with 25 queues to publish 680k visualizations with big assemblies 15k+ items in one assembly. 

It took 10 days to finish all jobs.. 

PetrH