Performance - # Objects per Folder

[thomas_hogan_99]

All,

I am migrating from Pro/INTRALINK to PDMLink 9.1 M010.

I am being told that for performance reasons, the target PDMLink environment needs to be limited to 3000 objects per folder, an object being defined as an iteration. The target PDMLink environment will be very large and many users worldwide, most being in the US.

The proposed solution is to automatically create numbered folders undereach productbased uponobject count. The proposed solution basically does away with using the folder structure to structure the data. This will require the users to search for their data rather than browse CS for it. As you can imagine, this will make migration very difficult and is guaranteed to make my users a ** little ** annoyed.

Questions :

(1) Is there a performance impact when there are many object iterations in a folder?
I have folders in my Pro/I environment with 50-80k versions in them.

(2) If there is a performance impact, how have you dealt with this problem?

(3) Is there a best practice out there for managing large data sets in PDMLink?

Thanks in advance!

Tom Hogan

[MikeLockwood]

Be careful with terms. There are considerations for folders in a Product or Library seen by users in a Browser - which you address below. There are also operating system folders in the vault. These both need planning. For the vault, see separate write-ups and guidelines - there is a significant performance impact depending on the quantity per OS folder. There is no relationship between the two types of Folders.


Not sure how others handle this, but we don't use any context folders and we have 1.4 TB of data (almost 1,000,000 Pro/E files) in about 45 Product and Library contexts. Before we migrated from Intralink (Feb 07) we had a complex and extensive set of folders set up as recommended by GSO, but we kept questioning the purpose/need and finally took them all away. We migrated all data to the root (/Default) level of each context - this has worked very well with no performance impact.

Users are quite productive always using search, never browsing thru folders (and/or navigation via Windchill relationships from the starting object that resulted from Search). Browsing by Folder is not feasable once you have any significant quantity of data. Advice: don't need to use any folders except if you need to apply different permissions.

[ptc-571495]

Tim,

I concur with Mike Lockwood 100%. In my observation across many
implementations for different clients; the more successful (and happier)
users are the ones where the data is segregated by Context (i.e. Products
and Libraries) for access control and team membership purposes. Within a
context, the more you can minimize folders the better. For everything but a
small data volume, using Search will be faster than browsing folders.

Here are some other tips for finding information efficiently:

- Users can find items they've already worked on recently from their Home
Page (Updates or Checked-Out tables). This almost eliminates the need to
browse *or *Search.
- If you use workflows for Promotions, Change processes, etc; most
documentation that the user needs to work on will be delivered to their Home
page in the Assignments table; again, practically eliminating the need to
browse or Search.
- Build relationships as a part of the on-going data management process
to allow you to quickly find or collect related objects.. Mike mentioned
this navigation from your starting part/document.
- Create and use Saved Searches to quickly find data meeting specific
criteria. An example is a Manager who uses a simple saved search to find a
data created or modified within the last seven days where the only variable
is the user name.
- Use Folders to separate by Object Type (e.g. Folders for CAD Parts,
Documents, Promotion Requests, etc). You can define in the OIRs the folder
location for each Object Type. This saves the user the few seconds
necessary to select a folder location. As the object will go into the
desired folder upon creation.

Hope this helps.


H. Lewis Kennebrew, II
Vice-President, Operations
ProductSpace Solutions, Inc.
"Your Products to the Nth Power"

2021 Midwest Road, Suite 200
Oak Brook, IL 60523
630-495-2999x8101


> layouts, etc. and allow Revise and Modify access only to lead users.
>
>

[ejhlti]

We're currently undergoing evaluation of 9.0, we've tried to limit all our folders to under 1000 objects. Only issue so far we've noticed with large folders is the time it takes to manually browse the folders if the display is set to all folder objects. What is instant in Intralink 3.x takes about 20 seconds with PDMlink. Most folks here are familiar with saved searches so manually browsing should not be an issue.

What is an issue for us is the display of real/decimal attributes. See my separate post a few post down in this group.

[prashantkisanpa]

Tom,

Here are answers to your questions

(1) Is there a performance impact when there are many object iterations in a
folder?
I have folders in my Pro/I environment with 50-80k versions in them.

*Prashant:* It is better to keep the folders logically partition in case you
want to set different ACL's depending on your requirements, it should not
create any performance impact as such, but might slightly decrease your
folder refresh time.

(2) If there is a performance impact, how have you dealt with this problem?

(3) Is there a best practice out there for managing large data sets in
PDMLink?

*Prashant: The vault folder limitation you can set it to 40000, as files for
above 65000 or so in a single folder in Windows OS, it causes significant
drop in performance while searching or retrieving.*

Regards,

Prashant

On Mon, Mar 23, 2009 at 1:11 PM, Lockwood,Mike,IRVINE,R&D <
mike.lockwood@alconlabs.com> wrote:

> Be careful with terms. There are considerations for folders in a Product
> or Library seen by users in a Browser - which you address below. There are
> also operating system folders in the vault. These both need planning. For
> the vault, see separate write-ups and guidelines - there is a significant
> performance impact depending on the quantity per OS folder. There is no
> relationship between the two types of Folders.
>
>
> Not sure how others handle this, but we don't use any context folders and
> we have 1.4 TB of data (almost 1,000,000 Pro/E files) in about 45 Product
> and Library contexts. Before we migrated from Intralink (Feb 07) we had a
> complex and extensive set of folders set up as recommended by GSO, but we
> kept questioning the purpose/need and finally took them all away. We
> migrated all data to the root (/Default) level of each context - this has
> worked very well with no performance impact.
>
> Users are quite productive always using search, never browsing thru folders
> (and/or navigation via Windchill relationships from the starting object that
> resulted from Search). Browsing by Folder is not feasable once you have any
> significant quantity of data. Advice: don't need to use any folders except
> if you need to apply different permissions.
>
> Note: We do have about 1/2 dozen folders in each context with
> more-restrictive permissions. We use these to "protect" Pro/E skeletons,
> layouts, etc. and allow Revise and Modify access only to lead users.
>
>

[vignesh]

Hi Tom,


Having too many files in Vault folder will definetly a performance issue. We had 250K files in single folder and we split them into 25k folders. This is a back end process and will not affect users. Vault folders is referenced by the database and NOT by the users. So, Users will never know that their files are stored at different locations. I am not sure whether you can mount on different partitions, in our case we have mounted on single 1 TB RAID-5 disk.

[wpalm_99]

Also, don't forget to do disable 8.3 naming. This will eliminate any
performance problems with writing files to your vaults due to the number of
objects in a single folder. You still should maintain a reasonable number
of files per folder so you don't lock up a machine when browsing to a folder
with a large number of files.

From the performance tuning guide page E-9 (for wc 8.0)...

Migrating Data to a Single Vault
When performing a data migration with a large number of files that are
migrated
to a single Windchill vault on a Windows server, the performance of the
FileTransfer loader may degrade as the number of files increases. This is a
result
of Windows creating an "8.3" compliant file name for each file loaded in the
vault.
Since the vault file names are 14 characters long and start with a series of
zeros,
the time it takes for Windows to formulate a unique "8.3" file name
lengthens
considerably as it processes a large number of files, eventually taking
several
seconds per file.
To improve the performance of the loading process, disable the creation of
"8.3"
compliant file names. There is a utility in Windows 2003 called "fsutil"
that can
be used to disable this behavior. The command line is as follows:
fsutil behavior set disable8dot3 1
You must reboot your system for this change to take effect.