cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

We are happy to announce the new Windchill Customization board! Learn more.

Vault fill up

wharaden1
6-Contributor

Vault fill up

I am using Windchill PDMLink Release 11.0 and Datecode with CPS M030-CPS09

We are finding in Windchill production that we are filling the vault directory up and it seems like the size seems big. We are seeing where the vault fills in two weeks and last time it filled the PTC person said it was documents. I was also wondering if re publishing the thumbnails would cause this vault to fill up. I would like to know how to minimize the grow and was wondering if republishing tumbnails would cause this. Let me know.

1 ACCEPTED SOLUTION

Accepted Solutions
mmeadows-3
13-Aquamarine
(To:wharaden1)

This is a common issue with multiple possible sources.

 

1. Has anyone removed unreferenced files lately?  When a content object is deleted from Windchill, it leaves behind the content file in the vaults.  These should be removed regularly based on file vault growth.  Personally, I 'move' the unreferenced files, confirm none of them were referenced (windchill wt.fv.tools.WContentVerify), and then remove them from disk.

 

2. If the vaults are cleaned up regularly, are some of your users checking in more frequently than they need to?  In the distant past, we trained CAD users to check in daily, sometimes multiple times a day.  Today we prefer to upload regularly and check in when the content needs to be shared with others.  Setting CAD to upload on save will grow the file vaults quickly, but this is manageable in conjunction with remove unreferenced files.

 

3. What are the largest content file sizes and how frequently do they iterate?  Sort the folders by file size or query the database for the large files and frequency of upload.  I once encountered a drawing that was over 1 GB because it had multiple photos embedded in it.  Five iterations of the drawing due to drawing note changes added 5 GB to the database.  The company quickly changed their documentation procedure in this case and purge helped clean up the unneeded iterations.

 

4. What is being published?  Visualization representation files (OL, PVS, etc.) are small, but the additional file types (STEP, IGES, DXF, etc.) are not.  One company was seeing rapid vaults usage growth because they were publishing STEP files upon check in for every Creo part in Windchill.  When questioned about the policy, they only needed a small fraction of the STEP files.  They were publishing just to have them in case someone wanted them.  We changed the policy to allow publishing on the fly of STEP files and upon release.  Then we removed a hundred GB of STEP files from Windchill.

View solution in original post

16 REPLIES 16
mmeadows-3
13-Aquamarine
(To:wharaden1)

This is a common issue with multiple possible sources.

 

1. Has anyone removed unreferenced files lately?  When a content object is deleted from Windchill, it leaves behind the content file in the vaults.  These should be removed regularly based on file vault growth.  Personally, I 'move' the unreferenced files, confirm none of them were referenced (windchill wt.fv.tools.WContentVerify), and then remove them from disk.

 

2. If the vaults are cleaned up regularly, are some of your users checking in more frequently than they need to?  In the distant past, we trained CAD users to check in daily, sometimes multiple times a day.  Today we prefer to upload regularly and check in when the content needs to be shared with others.  Setting CAD to upload on save will grow the file vaults quickly, but this is manageable in conjunction with remove unreferenced files.

 

3. What are the largest content file sizes and how frequently do they iterate?  Sort the folders by file size or query the database for the large files and frequency of upload.  I once encountered a drawing that was over 1 GB because it had multiple photos embedded in it.  Five iterations of the drawing due to drawing note changes added 5 GB to the database.  The company quickly changed their documentation procedure in this case and purge helped clean up the unneeded iterations.

 

4. What is being published?  Visualization representation files (OL, PVS, etc.) are small, but the additional file types (STEP, IGES, DXF, etc.) are not.  One company was seeing rapid vaults usage growth because they were publishing STEP files upon check in for every Creo part in Windchill.  When questioned about the policy, they only needed a small fraction of the STEP files.  They were publishing just to have them in case someone wanted them.  We changed the policy to allow publishing on the fly of STEP files and upon release.  Then we removed a hundred GB of STEP files from Windchill.

wharaden1
6-Contributor
(To:mmeadows-3)

Hi,

Thanks for the great in site. This is laid out well and explains nicely.

Can I ask how I can purge let say the E: drive or D:drive so I reduce duplicate and really all we need is the last 3-4 iterations.

Let me know and thanks.

Regards,

William (Bill) Haraden

Design Engineer responsible for CAD/IT Systems

Certified PTC Instructor

Engine Power, Inc.

Direct: (262) 560- 3217

Phone (800)-457-7056 ext 217

Cell: (704) 622-2307

upload_-aW1hZ2UwMDEucG5n-2760704510695206585..pngupload_-aW1hZ2UwMDIucG5n-6418990028815332338..png

bill.haraden@enginepower.com

upload_-aW1hZ2UwMDMucG5n-401845642197213352..png

mmeadows-3
13-Aquamarine
(To:wharaden1)

@avillanueva did a presentation on purge at the PTC/USER meetings last October.  His presentation can be downloaded from the PTC/USER app.

Thanks @mmeadows-3 , to be clear, I showed some SQL scripted that would identify contexts that had large amounts of purgeable data so you know where you need to target. Also, might want to try looking for extremely large file uploads for a quick win.

wharaden1
6-Contributor
(To:avillanueva)

Sorry, I did not receive the script .. can you please resend.

Regards,

William (Bill) Haraden

Design Engineer responsible for CAD/IT Systems

Certified PTC Instructor

Engine Power, Inc.

Direct: (262) 560- 3217

Phone (800)-457-7056 ext 217

Cell: (704) 622-2307

upload_-aW1hZ2UwMDEucG5n-6179290694800329288..pngupload_-aW1hZ2UwMDIucG5n-7165508088052919389..png

bill.haraden@enginepower.com

upload_-aW1hZ2UwMDMucG5n-6729938272951764213..png

avillanueva
22-Sapphire I
(To:wharaden1)

Purge Report - Calculate which Products have extra iterations

SELECT P.IDA2A2 CONTEXT_ID, P.NAMECONTAINERINFO CONTEXT_NAME, COUNT(*) TOTAL_PURGABLE_MASTERS, SUM(ITERATIONS) TOTAL_PURGABLE_ITERATIONS, (SUM(ITERATIONS) - 2*(COUNT(*))) BLOAT, ROUND((SUM(ITERATIONS) - 2*(COUNT(*)))*100/(SUM(ITERATIONS)),2) BLOAT_PERCENT
FROM 
( SELECT M.DOCUMENTNUMBER, M.IDA2A2 MASTERID, E1.BRANCHIDITERATIONINFO BRANCH, COUNT(M.DOCUMENTNUMBER) ITERATIONS
 FROM EPMDOCUMENT E1, EPMDOCUMENTMASTER M
 WHERE 
 E1.IDA3MASTERREFERENCE = M.IDA2A2
 GROUP BY M.DOCUMENTNUMBER, M.IDA2A2, E1.BRANCHIDITERATIONINFO) ITER_COUNT, 
 EPMDOCUMENT E2, PDMLINKPRODUCT P
WHERE
E2.IDA3MASTERREFERENCE = ITER_COUNT.MASTERID AND
E2.LATESTITERATIONINFO = 1 AND
E2.BRANCHIDITERATIONINFO = ITER_COUNT.BRANCH AND
E2.STATESTATE = 'RELEASED' AND
E2.IDA3CONTAINERREFERENCE = P.IDA2A2 AND
ITER_COUNT.ITERATIONS > 2
GROUP BY P.IDA2A2, P.NAMECONTAINERINFO

Variation with Files and Filesizes

 

SELECT P.IDA2A2 CONTEXT_ID, P.NAMECONTAINERINFO CONTEXT_NAME, COUNT(*) TOTAL_PURGABLE_FILES, ROUND((SUM(A.FILESIZE)/1048576),2) TOTAL_BLOAT_FILESIZE_IN_MB
FROM 
( SELECT M.DOCUMENTNUMBER, M.IDA2A2 MASTERID, E1.IDA2A2 ITER_ID, E1.ITERATIONIDA2ITERATIONINFO ITERATION, E1.BRANCHIDITERATIONINFO BRANCH
 FROM EPMDOCUMENT E1, EPMDOCUMENTMASTER M
 WHERE 
 E1.IDA3MASTERREFERENCE = M.IDA2A2) ITERATIONS, 
 EPMDOCUMENT E2, PDMLINKPRODUCT P, APPLICATIONDATA A, HOLDERTOCONTENT H
WHERE
E2.IDA3MASTERREFERENCE = ITERATIONS.MASTERID AND
E2.LATESTITERATIONINFO = 1 AND
E2.BRANCHIDITERATIONINFO = ITERATIONS.BRANCH AND
E2.STATESTATE = 'RELEASED' AND
E2.IDA3CONTAINERREFERENCE = P.IDA2A2 AND
ITERATIONS.ITERATION < (E2.ITERATIONIDA2ITERATIONINFO - 1) AND
H.IDA3A5 = ITERATIONS.ITER_ID AND
H.IDA3B5 = A.IDA2A2
GROUP BY P.IDA2A2, P.NAMECONTAINERINFO

 

wharaden1
6-Contributor
(To:mmeadows-3)

Nice,  I am running it currently after hours and good so far.  Let you know when I finalize but good so far.

 

Thanks Bill H

 

BenLoosli
23-Emerald II
(To:wharaden1)

Do you have your vaults set to automatic rollover so a new vault is created when one fills up?

What is your vault size set to? We use a rollover at 50,000 files in a vault.

Do you have the system set to Upload files when they are Saved? This does add extra files, but is a good insurance policy against lost work.

Do you Remove Unreferenced Files on a regular basis? This is the easiest way to get rid of excess files that are no longer needed.

 

Hey @BenLoosli , when you say rollover and create a new Vault, are you talking new folder creation or do you mean a new network drive/attached storage?

 

50,000 files seems more like a vault folder not the entire File Vault. 

 

Does anyone have a process that will rollover to a newly attached storage location? Say our Drives are 5TB and we need a new 5TB storage location. 

 

Thanks,

AndyH

what @BenLoosli is referring to is folders within the same vault. If you setup it up that way, when it hits the limit per folder, a new folder will be created under the vault and files will start to be written there. Seems like you are more concerned with the total space in the vaults not the number of files right? Yes, you need to monitor growth rate. Purging old iterations especially after release will make files unreferenced. Remember, deleting something in Windchill does not remove the files associated from the vault.  You need to run remove unreferenced to truly delete them from vaults.

 

If you need to shift from one attached storage to another, it should be as simple as creating a new vault on the new device, changing rules to publish content to that new vault. System will starting moving data (or is it copying) to the new vault and eventually the old vault can be removed. If you are doing to have two attached devices of the same size, that would not work since I do not think a single vault can span two devices with a common mount point. Field help me out here. I would recommend your new storage device be larger and totally replace the first one. Or look to do some clean up and avoid moving all together.

 

Hi

It rolls over automatically and creates a new vault folder in the vault

Regards,

William (Bill) Haraden

Design Engineer responsible for CAD/IT Systems

Certified PTC Instructor

Engine Power, Inc.

Direct: (262) 560- 3217

Phone (800)-457-7056 ext 217

Cell: (704) 622-2307

upload_-aW1hZ2UwMDEucG5n-144560550370345465..pngupload_-aW1hZ2UwMDIucG5n-7769737816436942895..png

bill.haraden@enginepower.com

upload_-aW1hZ2UwMDMucG5n-7646446689470437975..png

wharaden1
6-Contributor
(To:BenLoosli)

Yes I remove unreferenced files every two weeks. I was doing every week but it was not changing much.

Ty for your response. Also, I don’t believe they have ever used a Purge.bat to cut the size down. How best is this done.

Regards,

William (Bill) Haraden

Design Engineer responsible for CAD/IT Systems

Certified PTC Instructor

Engine Power, Inc.

Direct: (262) 560- 3217

Phone (800)-457-7056 ext 217

Cell: (704) 622-2307

upload_-aW1hZ2UwMDEucG5n-315650211732911885..pngupload_-aW1hZ2UwMDIucG5n-6690634818124060782..png

bill.haraden@enginepower.com

upload_-aW1hZ2UwMDMucG5n-4878061609852469341..png

If you want to locate the location of files in the vaults, their name, storage context, and size (so you can figure out which files are the largest and the number of iterations saved) you can use these reports.

 

I use them regularly to prompt users who have large files to keep only the last iteration if the previous ones are not needed (e.g., multiple versions of the same layout in a production department)

 

How to prepare a report which shows file vault information (file path and file name) that represents EPMDocuments

 

Report listing the size of content in file vaults (in MB)

 

How to get the Name and Context from any object based on the file vault file name

 

Report that can fetch vault location of all iteration of an EPMDocument

 

 

Marco
wharaden1
6-Contributor
(To:Marco_Tosin)

Hi,

Thanks for sending all this information. I will run these from the Windchill folder on the server. Where will the report be placed?

Thanks

Bill Haraden

The query builder reports are run from the Windchill interface.

You can import them as described in this article

 

How to import QML into report management in Windchill

 

In some cases you may have problems importing reports because of the rights required or if your Windchill installation is on the cloud

 

Importing or saving a report template as a non-Site Administrator user fails with error "WTPropertyVetoException: Access is restricted for this user"

Cannot upload or import a report in Windchill

 

 

Marco
cstein
4-Participant
(To:wharaden1)

We saw an issue with one of the 11.0 MO30 cps patches, can't recall which one.  But the vaults were closing with only sometimes ~90 objects in them and then moving to the next vault folder.  We were burning through vault folders very quickly which became a maintenance issue.  But one of the later CPS's @~20 something resolved that issue.  The vaults should fill up to ~50K objects before they go "read only" and move on to the next vault.  However, if you are seeing lots of objects, you might view the size of the objects to get a feel for how large the objects are.  Lots of very small objects could be the thumbnails as you suspect.  Very large objects might indicate that someone is uploading .avi files which can be several hundred GIG is size.  But, the web browser upload by default is limited to 2G, so . . . that's maybe not causing your storage consumption. 

Top Tags