Using PTC's standard Creo/Windchill performance test

[MikeLockwood]

Hello all,

Windchill is currently only used for CAD data management at Edwards (Windchill 10.2 M020, Creo 3.0 M060).

Windchill is on state of the art servers and user workstations are high end, but Windchill is very slow for some users some of the time.  In general, the network is very capable and fast.  No specific reason identified yet.

I've been asked to put measurement tools in place and methodically see what can be done.  We now have:

- PTC System Monitor installed and monitoring production

- PTC standard world car dataset in production

- A few key users lined up to do performance testing in order to trend / compare

I spent most of today on the available tools to automate the performance testing - with pretty good success.  Getting good support on this from EDC.  Able now to create / register workspace, import data, upload, checkin, checkout, export, etc., with log files recording the times.

http://support.ptc.com/WCMS/files/92341/en/WindchillSingleUserPerformanceTester.pdf

So, question - has anyone else used these tools to automate Creo user actions with Windchill using this approach?  I'd like very much to compare notes.  Happy to share what I've got with anyone interested.

It seems like the ultimate for this would be to automate the test, record the results using PSM, then analyze a PSM session exported for the single testing user.

[MikeLockwood]

Hey, I've got the automated test working perfectly now.  Zip file stored in Windchill along with Word doc with instructions.  Each user simply downloads and extracts the zip file, sets a command window on that folder and runs a batch file.  Results in a .txt file that I drop into a spreadsheet to summarize and compare to other tests.  It runs as a Windchill test user (Performance_Tester) and temporarily sets the .wf location, etc env variables, so doesn't mess up existing workspaces.  Happy to share w/anyone interested.

Haven't yet solved our performance problems with WC but have a way to measure very easily.

[TomU]

I would love to give this a try.  Thanks!

[byork]

Mike,

I would love to give it a try as well.

Thanks!

[MikeLockwood]

PTC folks in the EDC, would you like me to provide what I have back to you and then you post to the community?  I struggled a bit with the instructions and code files provided.  This might be a chance to update those somewhat.  I'll hold on this until Friday.

[GaryMansell]

Hi Mike,

I would also love to give this a try, so would very much appreciate a copy.

Rgds

Gary

[shumphrey]

I would also like any information you can share.

Sandra

[MikeLockwood]

Please see attached

[shumphrey]

Thank you Mike!!!!

[byork]

Yes thanks!

[BineshKumar1]

Thank you Mike

[TomU]

Ditto!

Just out of curiosity, have you been able to identify why two users were so much slower?  Is it just older hardware, or was the whole system more heavily loaded at the time they ran the test?

[MikeLockwood]

Reasons for much longer for some:

- Utah vs. California facility (no replication in place yet)

- One user testing same laptop using LAN and wireless connection (only ~ 3:1 longer which I was surprised with)

- Different workstations (all laptops)

- Within California facilities, one is ~ 5 miles away

[shumphrey]

Hi Mike,

We did have to make some adjustments on the security software to not scan any directories in the PTC structure.  Also, moving our users to a VM for running CAD tools seemed to help performance of not only Windchill downloads but for Creo not crashing. 

[GaryMansell]

Hi Mike,

Do you have any info you can share on using this tool, or a few pointers to using it perhaps?

Does the PTC World Car dataset need to exist in the target Windchill System before running the test and if so is there are specific product context and folder location that it needs to go in, or can it be anywhere searchable?

Or, does the PTC World Car dataset need to be available on the local machine (perhaps in this location: C:\PerfBenchmark\SPT\Datasets\Dataset_002)?

Does a specific user account need to be used to connect to the Windchill System, or can any account be used?

Rgds

Gary

[MikeLockwood]

So... Was not easy digging this info out of PTC's standard documentation but after fooling with it I got it to work well.  Answering questions specifically:

1. Yes, the data set (can be any data set) has to exist in the Windchill system as I've set it up. PTC provides a sequence in which it is uploaded and checked in as new data to the system, but I didn't see any way to do that repeatedly except to delete it after each run.  Seems to me that having it already in the system is more practical.  We chose to create a Library just for this purpose and named it "Performance_Test_Library."  The Team for this Library has just the Manage and Member Roles, and a single user (Performance_Tester) is mapped to the member Role.

2. The data set is in Windchill.  The test creates a workspace and downloads to the local machine.  It also includes both exporting from the workspace to the local machine, clearing the workspace and the importing back in (using the /Export folder).

3. The test requires having the username and password in plain text.  Because of this I created a Windchill test user Performance_Tester and all executions of the test are by this user.  Note: Environment variables are managed in the launch .bat file so that user workspace caches are not affected - very important.

I hope that the folks in PTC's EDC comment on this thread and update / clarify the documentation for all.

[James62]

Hi,

Does this work with Creo parametric 2.0? Seems like the libSPT.jar file is only for Creo 3.0.

[MikeLockwood]

There are some minor changes needed to run this for Creo 1 or 2.  Can see that in the downloads from PTC for these files and the related documentation

[James62]

Thanks for the reply. I just realized i might not have jlink installed. Will have to check.

Looks like the link to download the jar file for Creo.2.0 can be found here:

https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS56377&art_lang=en&posno=2&q=single%20user%20performance%20tes…

Just needs some minor setting

https://support.ptc.com/appserver/cs/view/solution.jsp?n=CS142419&art_lang=en&posno=1&q=single%20user%20performance%20te…

And there is also a newer version of the guide you posted above in the original post:

http://support.ptc.com/WCMS/files/166481/en/WindchillSingleUserPerformanceTester-R10.2.pdf

Can't wait to see this work!

[MikeLockwood]

I did not have to install JLINK - just extract the zip file.  Good luck.  Hope all who work w/this can improve on my original documentation.

[James62]

actually, here is the link with the jar for Creo 2.0

http://support.ptc.com/appserver/wcms/standards/fileothumbredirect.jsp?&im_dbkey=166480&icg_dbkey=522

[TomU]

I just noticed article CS140230 from PTC today on analyzing Windchill performance with PTC System Monitor.  Might be really helpful to use this approach with the automated test you've built.

[James62]

Turns out i had to use shortname as the path for proe inside jlink_SPT.bat:

PROE_DIR=C:\Applic\Creo2~1.0\COMMON~1\M090

It now fails on first operation:

[James62]

Any idea why is it updating only the top level assembly? I've tried to set <Object>Entire Workspace</Object> in the check in file but no go.

[MikeLockwood]

I could only get it to check out top level also, so we're living w/that.

[James62]

Ok, after few tests it looks like this tool can only check-in models that are already in Windchill, that is including Family Table Instances. Nothing completely new can it add to Windchill.

The file name that is supposed to be uploaded to Windchill has to be specifically set in the main input xml file.

Thanks for pointing out to this interesting tool, Mike.

[DanielO]

Excellent work Mike.

I will review the solution and see if we can implement something similar here.

BR

[DanielReid]

We are considering purchasing Fishbowl's LinkTuner, and still might as it does good monitoring and presentation of results for a pretty good price.  However, this could be a good supplement and we might be able to adapt it to more specific things we want to test (not covered in LinkTuner without customization).  I'll have to take a look at what you've done.  Thanks for sharing!