9.1 m070 re-vault issues

[WayneBeck]

When our system was setup there was no limit set on item
folder count. We have one in place now but our folder1 has 2.6 million items in
it. We are trying to upgrade to 10.1 as well as changing platforms. We cannot
get the vault to copy from a linux format to a windows format. It dies around
the 2 million mark. So we are trying to re-vault to make the folder size more
manageable. I have followed PTC's re-vault document but I run into a error. My
first question is can users be in the system working when a re-vault is going
on. And if so the error I receive says that there is no default vault set when
wt.fv.forceContentToVault=false is set to false. The second I switch this back
to true system runs fine. Any help or input would be great.

Thanks,

Wayne

[patrickchin2000]

Hi Wayne,

No software can copymillions offiles. The only solution I would recommend is block copy which is SAN "SNAPS". Rsync or CP commands will not work. Warning, DON"T do a MOVE which will result in a loss of data. Another thing, you are trying to move your ext# (Linux) file system to a NTFS file system. Try a samba mount instead after a SNAP. Actuially, you can still use AuFS with samba mounts. Hopefully, your vaults are in a type fo SAN and not in a local hard drive. If it is in a local hard drive:, I suggest:

1. mirror the drvie (with an exact duplicate block wise)


2. break the mirror.


3. backup the original


4. use the mirror as drive A for system 2.


5. replace the original mirror with a new drive


6. remount again with drive A for system 2 with another new mirror drive

Even if you set your wt.fv.forceContentToVault=false and turn on autofolderCreation with a limit of 50,000 per folders, you are still have to copy millions of files to another location/file system.

Can I ask why are you switching Linux to Windows? Better test the performance first.

Good luck,

Patrick

[mmeadows]

Wayne,

There are two approaches to addressing the single massive folder.

1) PTC recommends revaulting like you are attempting. It may take a week or more to get the data revaulted. You will have to run it multiple times until it revaults '0 of 0'. Then you call Tech Supt to get help moving the remaining files that just won't revault.

Revaulting is a background process so it can be done while users are in the system. Also, try setting the original vault to read-only and change the new vault to be default target for site. This might get rid of your error.

2) Instead of revaulting, configure the existing vault to be a root folder, split the contents of the master folder across the sub-folders of the root folder and redirect the database file pointers to the new vault folder locations. No second vault, no revaulting headaches, and you can do it in Linux or Windows. It 'should' behave like a background process and not impact users but I would do it during off-hours and bring Windchill down just to be certain.

I created it in Windows Powershell (mostly for performance reasons) by referencing this knowledge base article:

https://www.ptc.com/appserver/cs/view/solution.jsp?n=CS121641

[patrickchin2000]

Hi Wayne,

If you really need to copy your files to a Windows NTFS system, I reccommend doing the following:

• wt.fv.forceContentToVault=true


• wt.fv.revaultOnCommit=true


• wt.fv.uploadtocache.revaultOnCommit=true


• wt.fv.useVaultsForAllContent=true


• wt.fv.useFvFileThreshold=true


• wt.fv.purgeUnreferencedFvItemsOlderThan=5(make sure you are above 9.1 M050 SPR1954314)

If you use FVFileThreshold then revault, the default number of files per subfolder under your root folder is 50,000. Thus, for 2 million files, you should have 40 subfolders. Then create ascript file with 40 cp -r commands in separate lines to copy each folder# from your ext# file system to a samba NTFS mounted system (New Windows server) on your Linux machine. Make sure you run each command as a background processso you can log off or exit out of the terminal window.

This should take about 6 hours depending on your CPU and memory speed of both Linux and target Windows machine. If you have more cores, you can separate the script file into multiple scripts. Your linux CP commands shouldn't fail now. I never trusted windows copy.

Try this out on your test server first because you still have to becareful of the SPR1954314.

Good Luck

Parrick

In Reply to Wayne Beck:

When our system was setup there was no limit set on item
folder count. We have one in place now but our folder1 has 2.6 million items in
it. We are trying to upgrade to 10.1 as well as changing platforms. We cannot
get the vault to copy from a linux format to a windows format. It dies around
the 2 million mark. So we are trying to re-vault to make the folder size more
manageable. I have followed PTC's re-vault document but I run into a error. My
first question is can users be in the system working when a re-vault is going
on. And if so the error I receive says that there is no default vault set when
wt.fv.forceContentToVault=false is set to false. The second I switch this back
to true system runs fine. Any help or input would be great.

Thanks,

Wayne