Skip to main content
M
melissa.alford1-Visitor
1-Visitor
May 24, 2012
Question

Automated Testing for Workflows

  • May 24, 2012
  • 7 replies
  • 2165 views

We are moving from Windchill 9.1 to 10, which means that all of our workflows will need to be tested again. In the past, we have done our workflow testing manually. We need a more reliable and structured method. I was wondering about best practices for testing workflows. Does anyone have it automated? Has anyone tried DynaTrace for this?

7 replies

P
ptc-14635261-Visitor
1-Visitor
May 24, 2012
in 9.1, we too do all manual workflow testing. But for our WC 10
implementation, we are digging very deep into the Rational Quality tool
set - quality manager and functional tester. Not sure yet how much
automation we can bring specifically to workflows, still in the
implementation stages with the Rational tools... But think it may be
worth a good look..



Rick Staggs
Enterprise PLM Support Leader
(Office) 812-377-2653
(Mobile) 812-343-6241
rick.staggs@cummins.com

P Please consider the environment before printing this e-mail - Go Green!





A
AL_ANDERSON12-Amethyst
12-Amethyst
May 24, 2012
We are migrating in production from 9.1 to 10.0 this weekend. We tested
our workflows manually.

However, about 10 years ago, when we went from Windchill 5.1 to 6.0, I
wrote a program that would launch workflow processes in a loop, and a
listener that would listen for work items with certain names to then
complete those tasks in a direction that I wanted the workflows to go.
Each task had to have a unique task name. The idea was to put the
workflow queues under a high concurrent load to see how they did. It took
a couple of weeks to iron it out, but if you have a decent programmer and
a fairly simple workflow to test, then you could automate it. Launching
the workflows might be easier using an object create API in a loop with a
workflow in the initial lifecycle state (i.e. Create ECR), but what I did
back in the day was to actually use the APIs to create a new WfProcess.
However, this API-driven approach would not help if you have javascript on
the UI to test, or visual customizations on the tasks, or complex logic
regarding roles and participants. Unfortunately, I no longer have any of
that code. We just have not found the time it would save by using an
automation for testing to outweigh to time it would take to write, debug,
and maintain that automation over time. Beside, there are a lot of
variables to test that are fairly easy to test manually. For example, if
you have 5 workflows with a total of 50 possible tasks across all the
workflows, then after conversion you will want to run each task more than
once from its current location to completion. You will also want to test
new workflows from stop to finish. You cannot test every possible
combination of every possible scenario, but with planning, you can get
enough covered to have confidence that both migrated and new workflows
will work normally in the new system. An example of where found 10.0 to
require some adjustment in the workflows included ECT task prioritization
that we had customized into 9.1, but has an out of the box way to do that
in 10.0 - so we decustomized to be more out of the box in that instance.
Most workflows, including all of the out of the box elements of our
workflows, work fine after migration to 10.0 without having to stop and
restart them, as we did in many prior major Windchll upgrades.

If all you need to do is verify that the workflows work, then manual
testing is probably faster and more accurate (since you can see the
interfaces and interpret what you see more easily than a program) than
automating unless you have very simple processes, or unless you need to do
load testing to quickly spin up 100s of concurrent workflows.

Al Anderson




P
PreetiGupta15-Moonstone
15-Moonstone
May 24, 2012
It is really a huge task to test workflows for any upgrade event. We started upgrade to Windchill 10 effort last year. Since then we had been rehearsing on these platforms


* Windchill 10 F000

* Windchill 10 M10

* Windchill 10.1 F000

Now we are planning to go to Windchill 10.1 M020 for some other reasons.

We had been testing workflow functionality and other functionality manually for each upgrade run. Believe me it is such a tedious job to run it through each loop of the workflow. Of course process manager is little bit time of saver.

Since these are auditable workflow processes so we need to generate test results documents as well. It adds time to the process.
D
ddemay1-Visitor
1-Visitor
May 25, 2012
I have thought about developing a system that is like Al's. Had a PTC
solution architect tell me when I was worked on NASA's IEC not possible
until I explained to him it was.



The only real way to do this is to understand the data model of the workflow
templates in the database and recreate their runtime counterparts with
Junit, HTTPUnit, JSUnit testing. You then query for the workitems and
compute all possible routing options as the user would see it. You monitor
auditing tables for secondary validation. Proper design of a workflow task
template also must be kept in mind so that way either you can parse out the
options -or- have them duplicately defined in a property file so you can
initialize values with test data. A separate program capable of validating
receipt of the e-mails generated and how they look would be ideal. You
might be able to get away with some of the browser plugins that let you
define what the UI would look like given certain conditions. It sounds
like a lot of work and I suppose for myself (and my expertise level) it
would be a good 3-4 month project, but you factor in how rarely this area of
Windchill changes, it's a decent investment. You also have to have the test
results generated into a report for non-technical management. The nice thing
is this replicates manual testing and ensures you omit human factors in
testing quality. No one has ever asked me to build it and I have
contemplated offering it as a tool suite available amongst many others
utilities I have ideas for. Other useful features of such technology is a
comprehensive report of saying this is what my workflow does and here are
all possible outcomes and faster completion of the tests. Long term I hope
to develop something to use at TRW before we have to upgrade to Windchill 11
in 4 years or so.



- Dave











From: Gupta,Preeti,IRVINE,R&D [
P
PreetiGupta15-Moonstone
15-Moonstone
May 25, 2012
That would be great David. Please try to offer this product to PTC so that we all can use it without worrying about maintaining it ourselves 🙂

Happy Memorial Day..

Thanks,
Preeti Gupta
Sr Engineer(Windchill Specialist)

15800 Alton Pkwy, Irvine,92618 USA
T +1 949-727-7516

[cid:image001.gif@01CD3A5B.917A51C0]

From: David DeMay [
M
22-Sapphire I
MikeLockwood22-Sapphire I
22-Sapphire I
May 25, 2012
A few more thoughts on this important subject...

Essentially all of our workflow templates in 9.1 use task form templates. With the tabbed UI for Tasks in 10.1, these are having to be dropped for the most part. For us, this is the heart of the testing - not so much the logic, branching, code, etc.

Also though, we found the first day of testing 10.0 that Authentication with username (both Checkbox for Authentication in individual Activities) plus the property to require username) resulted in garbage in the UI and was non-functional. This resulted in an SPR and got fixed in 10.0 M010.

It really is necessary to exercise one each of absolutely every function.

By the way, we're still super disappointed that keyboard shortcuts for workflow templates do not work in 10.x (still not added back in 10.1 F000). Ctlr V / C / S / X, A, etc. are so helpful. Clearly no one at PTC ever actually tried to work on a complex template in 10.x before releasing.

From: Gupta,Preeti,IRVINE,R&D [
D
ddemay1-Visitor
1-Visitor
May 25, 2012
From what I heard working with Creo was priority #1 for release dates for
windchill software, all other functionality was second tier. I'm sure thing
swill get added back in. Hopefully the growing pains of switching task
templates could also be addressed. There are test software suites that can
address the front end fairly well. They just do not always come cheap.
Selenium is not too hard to use. (
Terms & ConditionsCookie settingsAccessibility statement