Dedicated publishing queues for large files/assemblies

[Dobi]

Is there a way distribute publish jobs to queues based on size so that they could go to dedicated (and more capable) workers? 

[Ajit_Kulkarni]

Hi Dobi,

Very simple question , how you can decide the size before publishing the job ?

Do you have more capable and less capable workers ? 

You can have separate CAD Workers for High , medium and low priority job. You will need customization to change the priorities of the job. 

Below article will help you in understanding job priorities. 

https://www.ptc.com/en/support/article/CS28472

https://www.ptc.com/en/support/article/cs149619?source=ArticleViewerRelated

Regards

Ajit 

[Dobi]

Hi Ajit, 

For example let's say that I have 3 jobs going to workers: one is a single part (say a cylinder), one is a drawing of said cylinder and a third is an assembly of 300 parts. Is there a way to send the part and drawing to one queue while the assembly goes to a different queue? At point of publish rules being triggered, Windchill knows what is going into the queues (and to which queue based on source and type). Can size (either file size or assembly size by virtue of BOM size) be used to separate the files? 

As far as more or less capable workers: if we have a VM with limited resources and a workstation with high resources there is a way to dedicate workers to queues. The sizing question comes into play because I'd want to be able to send my large jobs to the more capable worker while the smaller jobs go to the limited resource workers. 

Dobi

[HelesicPetr]

Hi @Ajit_Kulkarni 

The point is that normal small parts are published by one worker and never be available for huge assemblies that may take over hours to publish.

So normal parts are published in good performance and huge assemblies do not case some lags in publishing small assemblies and parts.

For example you can have two background method servers, first cares about queues 1,2,3, and second one process queues 4,5,6.

With this knowledge I think there is possibility to write some custom code that recognize size of huge assemblies and send it to publish to correct queue. 

There is class com.ptc.wvs.server.publish.PublishJob that cares about the publishing.

I have idea how to cover this needs but I don't have opportunity to try it 😄 

PetrH

[jlecoz]

Size is not an obvious criteria for CAD.

If you have a drawing of a part or if you have a drawing of a top-level assembly it is still a drawing the drawing file size may be similar, but the conversion CPU and workload will greatly differ.

This is because opening and converting the assembly drawing will take much more time as all dependencies will have to be downloaded and loaded in a CAD session managed by the CAD worker..

If you are experiencing performance issues when converting large assemblies you can explore other approaches to improve this. For example, using positioning assemblies.

[HelesicPetr]

Hi @jlecoz 

The point of size is about BOM size not file size 😄

also drawing can be checked against ASM and if the asm is huge send it to the dedicated queue.. 

(sure I'm talking about customization...)

PetrH

[Dobi]

Fair point @jlecoz .

My question is unchanged though in terms of just capability. This bit talks about sizing the worker environment by estimating how many small, medium and large jobs (across parts, assemblies and drawings) there may be. Aside from looking at the jobs that go in and making educated guesses as to what size they may be, there's no way to send say large jobs (of any type) to a dedicated queue so that things like dowel pins don't wait for hours for a large job to finish. Better yet, could there be a sizing threshold so that jobs that are beyond it automatically get scheduled for after hours? 

At root of my question is a combination of how long something takes to open and publish on a worker (for one) and also how long it takes to transfer to the CAD worker (for two).

[joe_morton]

This exact topic came up in the PTC/User conference last week (maybe you were in the room and this is a follow-up?). One admin in the room said that his company had achieved this, but I'm not sure how. It does seem to be possible though.

We have a dedicated workers for drawings and models, but we haven't gone down the route of trying to divide by complexity. 

[Dobi]

Alas, I wasn't there last week. I'd love to know the how. 

Even separating 2D from 3D is beneficial I think. Did you go down the route of custom filters and queues through these methods: CS80629 and CS132318 ?

[jlecoz]

Dobi,

Use the  filter method publish.publishqueue.priorities.filtermethod described in CS132318

This is a good approach using filtering methods it is generally some of the easiest, The inputs and outputs are well defined.

Using filter is upgrade safe as the filters are never modifying or altering any of the OOTB code, and code to be written should reasonably simple.

[avillanueva]

I was at that session with Bridget from PTC. Excellent presentation and discussion. Please pass along the name of the admin.  I spoke to her offline and did not get the sense that filtering rules and dedicated queues would work the way we think unless there was more data added to the models. @Dobi and myself are looking for some automated way WVS can sense the "size" of the job which translates to how long it would publish in order to take advantage of those priority queues. A very good estimation in my opinion would be number of components which could be calculated by Windchill before sending job to queue. Perhaps PTC knows better the cost of that calculation?  Our conversation focused on those very large assemblies that are impossible to publish where we might want to use a position assembly. Its a similar problem in that I will see a failure or a user will inquire why its not publishing so we add it to the growing list of exceptions in the "Publish and Thumbnail Control" page.  I was asking for a rule which can let number of levels or "size" so we do not have to play whack a mole all the time. 

So how can we resolve @Dobi 's question or did we @Dobi ? Does not appear that there is a solution today for what he is asking correct? 

[joe_morton]

I didn't catch the admin's name, unfortunately. 

Our drawing/model split was done by my colleague, sorry I don't know more about he did it.

[Dobi]

Thank you @avillanueva and @joe_morton for the discussion. From what I'm gathering, there isn't anything OOTB that would allow Windchill to separate out jobs by size (component count) as things stand now. There are options to filter out jobs through publish.publishqueue.priorities.filtermethod described in CS132318.

And @d_graham, from what you're saying we could write custom code to get the number of components of an assembly (or perhaps a file size) and then use a listener to send jobs above a set threshold to dedicated queues.

What other data would be needed in the models for the filtering to work the way we think it ought to? 

[d_graham]

@Dobi 

You should be able to direct a publishing job to the desired publishing queue with a listener.

If you have a threshold for the assembly size (number of components) code could be written that is automatically fired which gets the number of components in the assembly. If the number is greater than or equal to your threshold the job would be moved to the appropriate publishing queue when it is moved from the H, M or L publishing queue.

[TomU]

There were presentations at past PTC/USER events where other customers did something similar.  Usually it required something to be done in the CAD software (Creo) to indicate which assemblies were considered 'Large' and then use that information during publishing to route to the appropriate worker.

The better solution (in my opinion) is to switch all assembly publishing to extended positioning assemblies and completely eliminate the delay that  traditional publishing causes.

I switched all of our assembly publishing to extended positioning assemblies many years ago and I would never want to go back.  Attached are a couple of presentations from 2013 where John Deere and Paccar both made the switch.  JLG had a similar presentation in 2014.

[Dobi]

This is helpful, @TomU ! I think we're doing positioning assemblies across the board but will go back through the settings to confirm. 

What I'm getting out of the Paccar PDF (and some more digging into our data) is perhaps some settings that may have gone amiss in setting up the workers in the cloud. The conversion of a file (large or small) takes a smaller percentage of time than the overall publish duration (in my case defined as the time spent in the worker). The download of data from the server to the worker is non-trivial, taking as much as 90%+ of the total job duration. For a whole truck model at Paccar it seems to be instant. I've had a dowel pin take 3 minutes...

So perhaps my question is morphing more into how do I get the VM settings right so that my download times are reasonable irrespective of job side. 

[jlecoz]

Hi Dobi,

Taking 3 mins for a dowel pin publication shows a performance issue.

I don't know your number of users and the architecture of your system, so my advice may not fit. But to investigate such an issue you always need to have a performance baseline.

To get a performance baseline I would test to relocate the worker on a Windchill test server and allocate processor or memory to it to cope with this. And then try your dowel pin conversion. From there you can look at the job summary and compare it with your prod environment and detect the bottlenecks.

I attached a log summary on a VM for a single part, you can notice that it takes only a few seconds.

[Dobi]

To get this updated with the latest info:

I have a case opened with PTC to troubleshoot slowness. 

First pass:

• UI based downloading on the CAD worker VM works as expected with 100MB+ taking a few seconds. 
• worker based downloading takes minutes (3-5 minutes for the same files as above). 
• there were residual publish workspaces under the wcadmin account that, when deleted, reduced the download time to ~1:30 minutes for the same files as above. For sense of scale, there were ~34000 workspaces from some prior installation/update that had gone undeleted. 
• WGM Logging tool (with PTC observing) collected data for the UI based session and these logs are being reviewed presently. However, the Logging tool collected no data for the worker based session and I'm now waiting on further direction on setup to collect the logs and see where the rest of the time is being spent. 

[jlecoz]

If this is the first publication of your assembly and all its components the number of files can differ according to the assembly structure and also if you are downloading all the required dependencies or only the one required to regenerate the assembly.

So CAD structure has an influence too as you can see below.

Here is a comparison example with only the required dependencies:

[Dobi]

@jlecoz I've been working with the same dataset for all of these to maintain at least some consistency in the testing. I understand there would be differences in dependencies later on but I'm trying to have that not be in the mix for right now.