Out of Memory Problem

[maurice]

I have a code I use for processing a large quantity of image files. Each image file is 1600 x 1200 ASCII, about 10 MB. I read each file inside a for loop using READFILE and process it with an end result that's a 400 element vector. I then read in the next file, etc. I was very careful in constructing the code to use function calls so that memory wouldn't be occupied needlessly. My code never has more than 2 image files in memory at any one time. Nevertheless, if I try to process more than about 40 files the code terminates with a "not enough memory" issue. This seems strange both because I shouldn't be taking up memory unnecessarily, but also because when I've observed the memory usage using the Task Manager on my computer, it appears that over time the amount of memory being consumed by MathCAD just slowly creeps up until the program crashes. And according to the Task manager, this is occurring even though I'm using less than 2 GB of RAM (I have 4 GB).

Is there a bug in the way MathCAD allocates and releases memory? Are there any decent workarounds?

[Ninetrees]

It's ironic that I just found a similar problem in MATLAB. The problem there seems to be that garbage collection is the responsibility of Java, and Java would not schedule a garbage collection unless I forced a change in focus in the MATLAB environment. So I wrapped my code to the function calls that do the processing in a while loop that made a function call and then looped, alternately checking free memory and issuing commands to change the UI focus, until /free/ memory went above 2GB (for me). It works, though it is barely defensible, and I am not a Java programmer.

I don't know how to do the equivalent in Mathcad except maybe to click on different items in the UI as your code is executing.

...and, yes, if this is a problem in Mathcad, then it needs to be fixed here, too.

Rich
http://www.downeastengineering.com/

[ptc-1368288]

As it reads, no matter if you process only two images at the time. As the processing goes, the previous images called are still residing in full occupancy. You should clear them by making them = 0. If you have so much memory available from fresh start, it might be wise to call them all in a single read code.

jmG

[ptc-1368288]

On 4/9/2010 5:54:02 PM, maurice wrote:
...
>Each image is ... about 10 MB.<<br> _________________________
If you mean 10 millions pixels, that's a huge equivalent screen size
SQRT(10 000 000) = 3.162E+3 pixel square screen.

What's the size of the camera sensor ?

jmG

[maurice]

The camera is 1600 x 1200, or ~2 MPx. But each data point is 12 or 14 bits deep.

[ptc-1368288]

On 4/9/2010 8:36:50 PM, maurice wrote:
>The camera is 1600 x 1200, or
>~2 MPx. But each data point
>is 12 or 14 bits deep.
______________________________________

Something I don't understand. It looks a cheap camera for kids. Maybe they are *.JPEG images and except for little things, no processing applies as *.JPEG has destroyed it. The sharp "crater" [READRAW] 256 x 256 *.pbm occupies 12288 bytes of disk size. If the processing you are doing does not concern the "public" in one way or another, Photoshop might be better tool.

jmG

[TomGutman]

It makes no diifference how much RAM you have installed. Each application (like Mathcad) has a 2GB address space and all its memory requirements have to fit within that. Often you run out of memory well before that due to memory fragmentation. But from your report you are pretty much filling up the 2GB address space.
__________________
� � � � Tom Gutman

[ptc-1368288]

On 4/9/2010 5:54:02 PM, maurice wrote:
>...
>Is there a bug in the way
>MathCAD allocates and releases
>memory? Are there any decent
>workarounds?
_______________________________

What does work in version up to 11 and the versions after might differ or not work in the latest versions. The simplest workaround is to allow Mathcad the "pause syndicale". Every so often you restart. But before that and because a work sheet saturates which is equivalent of consuming unrecoverable memory, you can postpone the restart by working on a new sheet. Also you can have many and very many Mathcad instances opened and work for a while in one instance then close and keep working on the next instance and so on. Microsoft is like the Rubic cube, in 5 sec or never. Unfortunately there is no way to arrange in advance a variable structure like a work sheet. As the day goes and the work keeps going, what's left in the memory are only pointers, up to the point the box will not even add 1 + 1. Mathcad 11 and Millenium were excellent companions ... near saturation, the menu letters of Windows and more items would turn default font [ugly Terminal], closing some applications would free enough path to allow restarting the PC. XP does not work this way, it goes as far as the gaz tank allows then runs dry "Run time error", then zap Mathcad.

jmG

[ptc-1368288]

... also, it could be that some Improc functions are not "self-clear", i.e: not scalar functions. Then the all bits and bytes keep piling down instead of clearing themselves. Many user functions of such behaviour have been encountered in the collab. One of such pathology is the Mathcad 8 "de Casteljau" curve: that function must be renamed for each use, though she is constructed scalar and looks re-usable, but it is not. Maybe that's why they have removed it from the qs.

jmG