Skip to main content
J
JonathanHodgson13-Aquamarine
13-Aquamarine
February 11, 2013
Solved

Mechanica memory allocation vs actual usage

  • February 11, 2013
  • 3 replies
  • 8071 views

We've established here that setting Mechanica's memory allocation to around half your available memory is still a pretty good guideline, even though that's more likely to mean 6 GB than the 128 MB default.

In my team we're getting some amazing performance using a RAM drive to supplement WF4's 8 GB Mechanica limit, and it's a very useful (and usable) tool these days.

However, this number doesn't seem to be a hard limit - to copy from a recent .rpt file:

Memory and Disk Usage:

Machine Type: Windows XP 64 Bit Edition

RAM Allocation for Solver (megabytes): 8192.0

Total Elapsed Time (seconds): 5162.19

Total CPU Time (seconds): 3089.44

Maximum Memory Usage (kilobytes): 19861061

Working Directory Disk Usage (kilobytes): 18542428

Results Directory Size (kilobytes):

5172630 i:\casing_dyn_3rd_3mounts_3casings

Very small components usually do only use around 8 GB, but as the run gets bigger so does the memory take by msengine.exe, and I suspect that this recent run started hitting the Windows swapfile (20 GB Mechanica + 6 GB RAM drive + xtop + Windows on a 24 GB machine...) - which can't be a good thing!

What's the exact relationship between the memory allocation setting and what msengine.exe actually uses? By trial and error we'll probably turn it down to 6144 next time, but as each model is different it's a little hard to predict for a new run.

Thanks!


This thread is inactive and closed by the PTC Community Management Team. If you would like to provide a reply and re-open this thread, please notify the moderator and reference the thread. You may also use "Start a topic" button to ask a new question. Please be sure to include what version of the PTC product you are using so another community member knowledgeable about your version may be able to assist.
Best answer by JonathanHodgson

OK, I'm now re-reading Tad Doxsee's reply to my original post.

http://communities.ptc.com/thread/33467?start=0&tstart=0#160574

If I've understood it all correctly, my theory is this:

msengine.exe (Mechanica) has at least two elements to its memory usage: the global stiffness matrix, which it will fit into SOLRAM if it can and page to disk if it can't; and a general 'database'.

For very small models and adequate SOLRAM, SOLRAM is bigger than everything, so it claims that amount of memory (plus a tiny bit for the database) but probably doesn't use it all.

For slightly larger models, the database is big enough to add a noticeable amount to the memory use - but still small in relation to SOLRAM. At some point the matrix may become too large for SOLRAM, and start to page to disk.

For very large models (and with the WF4 8 GB SOLRAM limit), the database can be as big as, or bigger than, SOLRAM. Now the memory usage can exceed 2× SOLRAM, and the matrix is almost certainly already paging to disk. This is probably only possible with a 64 bit operating system!

So, the summary is: to analyse very large models, you may need to reduce SOLRAM to allow space for the other data that Mechanica needs to keep in memory. There's probably no way to predict this; just be aware of the possibility, and be ready to reduce SOLRAM if your computer gets close to running out of memory during a run.

Presumably this means that it's possible to create an analysis where the database takes up or even exceeds all the available RAM, leaving no space for SOLRAM at all... what then? Is that the limiting factor for model size / complexity?

3 replies

I
integrisbrad1-Visitor
1-Visitor
February 11, 2013

Hello Jonathan,

I believe this was answered in an older thread that you were involved in: http://communities.ptc.com/message/161358#161358

As far as mem allocation, my understanding is that it's the size of chunks of memory that Mechanica will take when it needs it. Thus, when we used to get the "insufficient memory" error on 32-bit machines, you could sometimes make the run go by reducing memory allocation, which is counterintuitive. But the reason it worked is that it was taking smaller bites of the memory to get closer to exactly what the run needed instead of throwing more than enough RAM at it. So most often, I would expect that you would see less Maximum Memory Usage with smaller memory allocation.

Smaller bites = more writing to disk, larger bites = less writing to disk. In general, you can see a decent improvement in run speed by taking larger bites, but my experience was that it was model and/or machine dependent as to how much speed benefit you see. Put SSD into the picture and it might be irrelevant.

Brad Green

J
JonathanHodgson13-AquamarineAuthor
13-Aquamarine
February 11, 2013

The "chunks" explanation doesn't hold water for me. We now use 8192 as our standard SOLRAM value, for everything. With that setting, I've seen memory 'takes' of around 8 GB, 10 GB, 13 GB, 16 GB, 20 GB... there's no sign that it uses multiples of SOLRAM. It just seems to vary as it wants.

Running my own quick test:

SOLRAM Max Mem Ratio E.T.

512 668852 1.28 3.84

1024 1168884 1.11 3.07

2048 4093364 1.95 2.78

3072 5093364 1.62 3.63

4096 6093364 1.45 3.17

8192 10093364 1.20 3.13

(Also, how bad is the font and tab management in this text editor?!?)

So, from that I'd conclude that for any given model, larger SOLRAM means larger memory take, but not in any sensible or linear fashion.

Mostly I'm just a bit concerned that it doesn't stop at a ratio of 2.0 (my original example went to 2.37× the SOLRAM value) which isn't helpful when you're trying to make best use of finite RAM.

This is only really a problem when we're running really big models - 8192 works for all the smaller ones - but that's potentially when it has the biggest impact.

S
SylvainA.1-Visitor
1-Visitor
February 11, 2013

The "093364" string looks strange...

Keep in mind that 1Mb = 1024 kb.

S
SylvainA.1-Visitor
1-Visitor
February 11, 2013

My 2 cents:

I recently saw that the memory usage reported by mechanica depends on the SOLRAM setting more than on the matrix size.

As an example, the analysis of a little rod with solram set to 500Mb:

Machine Type: Windows XP 64 Bit Edition
RAM Allocation for Solver (megabytes): 500.0

Total Elapsed Time (seconds): 0.90
Total CPU Time (seconds): 0.33
Maximum Memory Usage (kilobytes): 553.282

with solram set to 1000Mb:

Machine Type: Windows XP 64 Bit Edition
RAM Allocation for Solver (megabytes): 1000.0

Total Elapsed Time (seconds): 2.25
Total CPU Time (seconds): 0.94
Maximum Memory Usage (kilobytes): 1.101.019
Working Directory Disk Usage (kilobytes): 0

with solram set to 2000Mb:

Machine Type: Windows XP 64 Bit Edition
RAM Allocation for Solver (megabytes): 2000.0

Total Elapsed Time (seconds): 1.15
Total CPU Time (seconds): 0.33
Maximum Memory Usage (kilobytes): 4.016.333
Working Directory Disk Usage (kilobytes): 0

But in all cases, the computer memory usage increased by ~30Mb.

I also found this in the *.pas file:

Size of global matrix profile (kb): 249.888

Number of terms in global matrix profile: 31236

Minimum recommended solram for direct solver: 2

and this little advice:

If you set solram too high, performance will usually suffer,

even on machines with very large RAM, because there will not

be enough machine RAM for other important data.

For example, Mechanica allocates many large, non-solver

memory areas that will cause excessive swapping unless you

leave enough spare machine RAM.

J
JonathanHodgson13-AquamarineAuthor
13-Aquamarine
February 11, 2013

Yes, the "may case excessive swapping" warning is exactly what I'm concerned about.

For small models it seems that there's no significant effect on speed (I don't really care whether a model takes 3.1 or 3.8 seconds to solve!) but with larger runs there's definitely a penalty if you set the value too low, certainly when using a mechanical hard drive for the working directory.

Interestingly, all my test runs in my other post had the same working directory disk usage (11573).

J
JonathanHodgson13-AquamarineAuthorAnswer
13-Aquamarine
February 11, 2013

OK, I'm now re-reading Tad Doxsee's reply to my original post.

http://communities.ptc.com/thread/33467?start=0&tstart=0#160574

If I've understood it all correctly, my theory is this:

msengine.exe (Mechanica) has at least two elements to its memory usage: the global stiffness matrix, which it will fit into SOLRAM if it can and page to disk if it can't; and a general 'database'.

For very small models and adequate SOLRAM, SOLRAM is bigger than everything, so it claims that amount of memory (plus a tiny bit for the database) but probably doesn't use it all.

For slightly larger models, the database is big enough to add a noticeable amount to the memory use - but still small in relation to SOLRAM. At some point the matrix may become too large for SOLRAM, and start to page to disk.

For very large models (and with the WF4 8 GB SOLRAM limit), the database can be as big as, or bigger than, SOLRAM. Now the memory usage can exceed 2× SOLRAM, and the matrix is almost certainly already paging to disk. This is probably only possible with a 64 bit operating system!

So, the summary is: to analyse very large models, you may need to reduce SOLRAM to allow space for the other data that Mechanica needs to keep in memory. There's probably no way to predict this; just be aware of the possibility, and be ready to reduce SOLRAM if your computer gets close to running out of memory during a run.

Presumably this means that it's possible to create an analysis where the database takes up or even exceeds all the available RAM, leaving no space for SOLRAM at all... what then? Is that the limiting factor for model size / complexity?

S
SylvainA.1-Visitor
1-Visitor
February 12, 2013

Jonathan Hodgson wrote:

Presumably this means that it's possible to create an analysis where the database takes up or even exceeds all the available RAM, leaving no space for SOLRAM at all... what then? Is that the limiting factor for model size / complexity?

I think that swapping to disk will occur, performances will decrease, but it would run fine (assuming that the OS virtual memory is big enough).

One would keep in mind that the memory usage reported by mechanica isn't the actual used RAM. That's just the amount of memory allocated to mechanica and things can go wrong if the global stiffness matrix is "too big".

I mean, if we set solram to, say, 10Gb on a machine with 4Gb RAM all will be fine until the 4Gb RAM will be entirely filled.

Tad Doxsee wrote:

For example, suppose you have a machine with 4 GB of RAM and 4 GB of disk allocated to swap space.You run an analysis which needs 1 GB for the global stiffness matrix, K, and 2 GB for everything else, which I'll call DB. If you set solram to 1.5 GB, then, ignoring the RAM used by the operating system and other applications, the memory usage looks like this.

Available:

RAM swap

|--------------------------------|--------------------------------|

Used by Mechanica:

DB K
****************(########----) Ideal
solram



DB + solram < RAM good (no OS swapping)
K < solram good (no matrix equation swapping)

In the above, the memory used by DB is shown as ****, the memory used by K is shown as ###, and the memory allocated to solram is inside parentheses (###--). Because K is smaller than solram, there is some memory that is allocated to solram that is unused, shown as ----. This is an ideal situation because the K < solram and DB + solram < RAM and hence, no swapping will occur.

But in my example the maximum memory usage reported by Mechanica would be ~10Gb or more.

I'm curious to see what's the global matrix size of the analysis you ran above as a quick test.

Could you have a look at it?

J
JonathanHodgson13-AquamarineAuthor
13-Aquamarine
February 12, 2013

The small run that I tried at several values of SOLRAM reports: Size of global matrix profile (mb): 9.51816

The run that hit 20 GB: Size of global matrix profile (gb): 13.8528

Terms & ConditionsAccessibility statement