I'm in a situation where I work for an organization using ModelCheck. However, in my humble opinion, it's being used incorrectly. While I believe I have the credentials and experience to make this judgment call, I wanted feedback from the greater PTC Community so I thought I'd pose the question here and see what the responses are.
This particular organization has three prongs to their implementation of ModelCheck:
In my personal opinion (I get a bit heated here), ModelCheck operating in Save Mode is among the top 5 dumbest ideas PTC has ever floated. They recommend it - but they don't recognize performance inefficiencies because they do not live with them. Their careers do not depend upon getting product out the door using Creo.
The organization feels:
In my experience and opinion, this restrictive implementation is hurting our schedule, budget, productivity, accuracy, and morale. Soliciting experiences from other professionals and organizations may help. If I were able to prove that the majority of companies do not use "ModelCheck Save Mode", this would eliminate part of the problem. If I could further prove that easing restrictions during a design phase was commonplace - and that more restrictive standards are more appropriate just prior to release, that might also help.
What is your experience running and using ModelCheck in your enterprise? Do you run it at every save? How do you mitigate crashes? Do you have multiple levels of error-checking restrictions (concept more permissive, release-ready more restrictive)? And finally, what is your experience with user's "waiting until the last minute" then needing relief?
In my opinion, the concerns can be addressed with training and proper project management. If designers are reminded that they must allow time to clean up their models (and supervisors are aware this is part of the designer's job) no one should be caught off guard. If so, the designer will learn that hard lesson ONE TIME and in the future, they'll be wiser. I believe people will take the easiest path in most situations. They will learn to work with the tool as long as it does not represent a significant performance and productivity hit.
What do you think??
Thanks for posting this, Brian. I'm eager to hear others' responses to your questions.
Though I'm not in your situation, and therefore cannot comment from the customer perspective, I tend to agree with you that there must be a better way. ModelCHECK is tricky in that you want to ensure quality models, but in such a way that it isn't a prohibitive burden for users. It does seem like it might be beneficial to use warnings instead of errors earlier in the design process. That way, users can see the issues they'll need to fix later on, but don't need to worry about fixing them immediately. I can understand the frustration of having to wait for ModelCHECK to run after every save, but the alternative would be to leave it to users to run ModelCHECK in Interactive mode, which is a manual step that might get missed.
So in my view, the best path forward isn't straightforward, but hopefully we'll get good responses from other folks.
I agree that the ModelCheck step might get missed. And I agree that there are good reasons to use the tool. My issue is that this particular organization has dialed up a bad combination of restrictions which equal lost work, lost productivity, less frequent saves, less frequent (or completely avoided) check-ins, and no strategy whatsoever for mitigating the problems being caused. Instead, it's "the designer's problem".
Someone else caused a problem with a file and it's impacting YOUR work? Tough. Delete your work! Do it again a second time later! Or just figure it out and quit whining!
It's assumed the designers would prefer to check in crappy models. It's assuming everyone will shirk responsibilities. Or, worse, it's assuming most people will do their job properly - but a few bad apples will not so therefore everyone must suffer equally.
It's just a really ridiculous stance - but IT management sees no problem with it at all! Hopefully we'll get some good responses. Maybe it'll help!
I tend to agree with what you wrote. We are currently not running ModelCheck on a very large project and that is simply because I haev not had time to set it up. I have setup and configured ModelCheck at 2 other companies, so I do have some experience with it. I do have another system that does have ModelCheck enabled but I have only marginally tweaked that system configuration a few times in the last 11 years.
I would not rely on a user setting a parameter to flag which ModelCheck rules to execute. They will leave them at the minimal check if at all possible, unless you have someone review the ModelCheck result files as part of the release process.
Having ModelCheck run at save and flagging errors may be a time consumer, but it is minimal. Also it might be easier to fix a flagged error earlier in the design process than later. ModelCheck rules should be reviewed with the users and management to be sure that they are checking the important things and not something mundane that doesn't matter. Do your designers review the ModelCheck results after each save to be sure they have clean models as they go along?
From your comments, it looks like PTC consultants set up ModelCheck per management 'goals' and did not consult with the end users about all of the issues it can cause. Setting the Windchill gatekeeper to not allow a check-in due to a single ModelCheck error is asking for trouble. Maybe set the threshold at 10 or 25 errors, so the file can be checked-in, but then it MUST be fixed. Marking a check to be an error versus a warning is critical to using the tool effectively.
Thanks for your reply. You're totally right - having different codes to trigger different checks is a mess. It was implemented specifically so we could have the most restrictive environment possible. For instance, suppressed features are disallowed (why?) but in some tools like AFX (Advanced Framework Extension), a suppressed feature is part of the tool. Suppressed features are used as part of the AFX module. If you disallow those features, many beams and components generated by AFX cannot be checked in. So there's a code to allow a suppressed feature for AFX. But if you know that code (and can avoid other errors), you can use the AFX code to skirt the checks. It's a bad rabbit hole to go down.
Note that there are several ways this could be fixed:
In this case, PTC did not set up ModelCheck. I have to be careful what I say because this will likely make its way to the very people I am taking about so I'm torn to explain how it got this way. Our own people set it up. They did not consult users... "the lowly stupid user doesn't know what's good for him". It's a bad combination of egos, inexperience, misguided goals, misunderstanding of the tool, lack of confidence in the designers, poor training, poor management all around, and so many chips on so many shoulders it looks like a forest.
I do have to strongly disagree with you that ModelCheck running at each save is "not much productivity hit". In this environment I wait an average about 45 seconds per save and up to 3 minutes. If I save every 15 minutes, I lose about 5 minutes an hour (and sometimes more). At the end of every 12-hr day (and I work MANY of these) I have lost one hour to simply saving my model and waiting for ModelCheck. At four hours per week per designer and 25 designers - that's 100 hours a week - 400 hours a month - and nearly 5000 hours a year for nothing. We could hire 2.5 more engineers, make all of them checkers, and have the better results.
The simple fact is... people do fix their models as they go along. But sometimes things happen! Someone else will remove a feature from a part - let's say they delete a hole and re-place a new one somewhere else rather than simply MOVING it. You lose a placement reference and your model assembly model is now "red" with feature errors and component placement errors. You cannot suppress the failed component while you go back to address the hole issue with the part designer. You cannot fix the failed component into position. Your only option - is to delete the failed feature. Otherwise, you cannot check your model in. Period.
We are checking mundane errors and relevant ones. But all are treated as dire and serious. Some model "codes" which can be applied allow one or two specific errors through so people "code shop" trying to find any code which will allow them to save and check in their work. No one cares what code you use - we're just trying to get files checked in so we can collaborate. There must be 15 different codes and all of them are too restrictive.
I think if you worked with this system for a week, you'd be appalled and trying to alert someone that this combination of restrictions is hampering work. Your experience with the tool would be noted - and ignored. It's like saying:
"Thank you for your irrelevant opinion!" You're only complaining because you like to make bad models and we are stopping you! (See the adversarial way this plays out? You, the designer, are TRYING to cause problems and ModelCheck is forcing you to do your job - because unless you're forced, you're a moron who will purposely perpetuate bad models.)
You made the comment "If ModelCheck crashes your model and all work is lost..."
I want to make sure you're aware of the config (SAVE_MC_PRE) that will cause ModelCHECK to run AFTER the save happens, rather than running ModelCHECK before the save and thus risking data loss.
Of course, the problem with this config is that if the model is saved before ModelCHECK runs, then the parameters from ModelCHECK don't get saved after ModelCHECK runs.
So I'm not really sure if this would be too helpful for you, but I at least wanted to let you know it exists, in you weren't already aware.
First, we have not totally implemented Modelcheck as I would like but here is my philosophy and where I'm trying to drive my company. First, we setup a button to make running Interactive Modelcheck readily accessible in the Quick Access Toolbar. We are NOT using gatekeeper to enforce absolutely error free models on checkin. There are a couple of reasons for this, first we don't want to discourage people from checking in and there simply are times that you can't completely eliminate a few specific errors. What we do have is the ability to force the user to run Modelcheck as the very last thing they do prior to doing a save if they are going to be checking in the model. So the user can save without running modelcheck but if they are going to check in the model then they MUST run modelcheck LAST. When MC is run it sets a parameter called MC_VERIFIED to YES and if any other change is made after that it the parameter changes to NO. This at least puts the results of the MC in front of the user on an ongoing basis. We then use Release Validators to prevent models with MC_ERRORS from being released. One major drawback is that there is still the issue of some errors that simply can't be fixed so our Configuration Managers can override the issue which of course can tempt them when deadlines approach.
There is simply no foolproof method to give the users the flexibility they need to get their work done but also force the discipline needed for good modeling. This is simply the balance I have tried to strike.
I could not agree with this more if I tried.
A balance is what is needed. ModelCheck is the THIRD button on our Quick Access Toolbar. I put it there intentionally so people could access the tool quickly and often during their design work. The approach was going to be encouraging manual usage of the tool and making minimally restrictive rules for GateKeeper (for those who don't know, this is the tool which controls whether models can/cannot be checked in unless ModelCheck has been run).
Unfortunately, this approach was abandoned in favor of "fix all errors - even minor ones at once, all the time - no exceptions!" Sadly, it's all ego. I recognize "ego" - because I have one myself but I always try to listen to differing feedback. I will admit a mistake and I will change my opinion when confronted with reality. In this case, the decision-makers are either completely inexperienced with Creo or are so insulated from the problems being caused that they do not see an issue. They're more invested in staying the course because any change might be taken as an admission a mistake was made. There are a dozen ways to mitigate the problem without damaging anyone's credibility but there's no interest in doing so.
I fully support the approach you shared. I support both running ModelCheck just prior to checkin (or prior to release - I am flexible on this point). I also support resolving all serious errors prior to release as long as there's an override capability for situations that cannot easily be resolved. I'm not saying everyone gets an easy PASS... I'm saying there are times when it's warranted to allow something to slide (imported model, COTS item, etc) and an administrator should have an override tool available.
In general though, I'm just trying to advocate for a balance but I am getting nowhere. Thank you for your feedback. This is the kind of reasonable approach I hoped to hear about.
I'm strictly a user, I have never set up Modelcheck. I have had input on the setup. I've worked at 3 companies that use modelcheck in various ways. Company 1 set it up for the user to run (I doubt anyone ever ran it after the first few times). Company 2 had a much more restrictive setup and only ran when the user ran it AND they had a manual check afterwards to verify all errors were corrected. Company 3 (current company) is also doing the user run with a verification via an attached report by our documentation people.
I agree that if left to the user to run, Modelcheck would never be run. I would skip it everytime.
I also agree that an every save Modelcheck run is absolutely a waste of time and would directly result in fewer saves and more "lost work" due to crashes. I also feel that gatekeeper would also result in lost work due to the reluctance of checking in and the "work" it would cause.
From my standpoint, my current company implemented modelcheck about 5 years ago. We have models that date back 20 years, maybe more. When I run modelcheck on my large assembly, it can take an hour or more and generate a lot of errors. Our current method of enforcing Modelcheck "fixes" is upon completion of the drawing, model check is run and a report is generated and attached to our EN. Our release documentation people verify that the report is attached for each drawing to be released and that there are no errors. We also have limited exceptions for no modelcheck requirements on some drawings that would take a long time run modelcheck.
I think you've done a good job of explaining what I would call a "Common sense approach" to using ModelCheck. In my experience, this is how most companies operate. Thank you for your feedback! I agree with points you raised. I wish I had more peers with ModelCheck experience to join forces and say "This is wrong" but there are too few of us to make a difference.
We implemented model check this year and it is only implemented for the things that actually help users. It only checks:
We have large assemblies and having it run on every save or regen would be very problematic.
I agree, Chris... checking for a few really important things would be much better than what we have now. Running the tool constantly has become a real burden with no benefits.
For our company, it’s worked well to have a group of superusers that represent various internal user groups, locations, etc. that can work with your Creo/Windchill admins to provide feedback before and after implementing changes.
ModelCheck at our organization:
ModelCheck isn’t perfect but it’s helped us enforce some basic company standards. For Windchill relevant things like missing parameters or allowed drawing formats, I wish you could configure Windchill products and libraries to require the parameters or drawing formats there. Currently, I believe you can only set required parameters at the Org or Site level using the Type and Attribute Manager.
Thank you for your detailed response, Luke. I appreciate you providing the benefits of your experience. Personally, I still hate SAVE mode - because when you do have a crash situation, there is no remedy whatsoever. The configuration is set to run off he user's local hard drive but still takes between 5 seconds and 2 or 3 minutes to run with no indication of what's happening or why. If it truly ran in only 1 second, I might be willing to live with the potential crash issue - but I feel very strongly there should be an emergency save button (perhaps a toolkit function) which will bypass MC and allow the the save regardless.
I don't believe checking incessantly for parameters and layers is a good use of time - but I can bend on these things. They're minor. I liken it to checking the tire pressure on your car every time you go for a drive. Heading to the store? Check the tires. Heading home from the same store? Check the tires. Engine on fire? Sorry, we don't check for THAT but the tires are 100% perfect!!
Alone, I do not have a problem with ModelCheck, GateKeeper, or the notion of fixing problems as you go. The problem I have is an extremely restrictive set of checks coupled with no alternatives. ModelCheck must run or else. Gatekeeper allows zero errors. And there are a list of errors as long as your arm. Warnings are everywhere, too. Everything demands to be resolved immediately. It's the lack of balance that troubles me so greatly.
For example... there are extremely valid reasons why someone might suppress a component during the rapid iterations of a design phase. I'm not saying I routinely junk up my model with a bunch of suppressed components - but there are occasionally very reasonable reasons to suppress something in targeted situations. This is completely disallowed. Sure, you can use a simplified representation to turn things off/on but on a massive project with thousands of assemblies, no one knows all those simp reps. So people open the master rep and are then confused by what they see.
In our case, rules were created by fiat with no plan to slowly and methodically adopt new checks as they became necessary. Your choices are to live with it as-is and smile or be deemed a "ModelCheck denier" who simply wants to pollute the Windchill database with junk. 🙂 I'm being a little funny... but it's sadly not far from the truth.
Thanks again for your guidance! Take care!
Thanks, Tom... that's simply me doing too many things at the same time. 🙂 I knew better.
Oh, and I'm super thrilled that after achieving Diamond status and (for awhile) being the third most proliferate provider of solutions on the old Community Board - that I'm back to being "dirt" level or whatever it's called now. I'm one step up from "manure" level. LOL
At least no one has mistaken my name for a rock - I'm really sorry Luke!! My brain has already left my head for the weekend, I apologize and I have corrected my post.