On Sun, 26 Nov 2000, Russ Whitaker wrote:

On Sat, 25 Nov 2000, David James wrote:

...

The application has been running for some 21 hours now, and I've been taking occasional snapshots both of status and maps.

status shows VmSize and VmRSS growing.

status

Name: webmake.e

Now that you've identified a program that grows, suggest looking for something simple: like an object created by new(), perhaps later saved to disk, but does not have a coresponding dispose() to remove it from memory.

Shot in the dark, but it's an easy mistake to make.

Another shot in the dark: up to now we can't exclude a garbage collection problem inside the memory management behind new and dispose. Under certain conditions pieces of memory freed with dispose may not be reusable for (even slightly) bigger ones. Suitable new and dispose sequences hence may produce lots of unusable memory blocks, which would eat up all available memory.

Ernst-Ludwig

Hope it helps Russ

On Mon, 27 Nov 2000, Martin Liddle wrote:

In article Pine.HPX.4.02.10011271431440.19419-100000@dirac.desy.de, Ernst-Ludwig Bohnen bohnen@mail.desy.de writes

...

Another shot in the dark: up to now we can't exclude a garbage collection problem inside the memory management behind new and dispose. Under certain conditions pieces of memory freed with dispose may not be reusable for (even slightly) bigger ones. Suitable new and dispose sequences hence may produce lots of unusable memory blocks, which would eat up all available memory.

Can you be any more specific about what conditions might apply. The worst affected program starts off using ~5Mb of RAM and after 24 hours can be using more than 100Mb. Fortunately the server has 768 Mb RAM so the problem is manageable, although not pleasant.

A special sequence of new() and dispose() may produce memory fragmentation. The example below doesn't take into account additional bytes needed for pointers or byte counts, etc, and ignores modulo 4 effects:

1) Get 1000 records each with length of 1000 bytes numbered from 0 to 999. We use 1,000,000 bytes taken from the heap.

2) Now free all odd records and then get another 1000 records each with length of 1001 bytes. Now we use 500*1000+1000*1001= 1501000 bytes, but really took 1000*1000+1000*1001=2001000 bytes from heap because the old odd 1000 byte records can't be reused for 1001 byte records.

Assuming a complex get/put application may break the heap into unusable small memory fragments this way and calling new() will eat up fresh memory with sufficient size.

But the numbers you mention above do not indicate to me that this story causes the problem. Continuous memory leakage proportional with time is more probably caused by missing dispose calls, for instance forgotten to dispose daughter records before disposing their mother record, or ...

Ernst-Ludwig

-- Martin Liddle, Tynemouth Computer Services, 27 Garforth Close, Cramlington, Northumberland, England, NE23 6EW. Phone: 01670-712624. Fax: 01670-717324.

Martin Liddle wrote:

Now that our most urgent deadline has passed we will have a little more time to look at the debugging data we have gathered and to follow Frank's suggestion to upgrade to the latest snapshot of the compiler. Thank you for your interest in our problem.

That's not exactly what I said. I said I could try to add support for this in the RTS if you like, and you'd have to upgrade *then*...

Frank

Once upon a time, Martin Liddle wrote:

In article 4F1A9BDB.20001129032709.FOO-2554.frank@g-n-u.de, Frank Heckenbach frank@g-n-u.de writes

...

Martin Liddle wrote:

...

Now that our most urgent deadline has passed we will have a little more time to look at the debugging data we have gathered and to follow Frank's suggestion to upgrade to the latest snapshot of the compiler. Thank you for your interest in our problem.

That's not exactly what I said. I said I could try to add support for this in the RTS if you like, and you'd have to upgrade *then*...

OK. I accept that but you have also previously suggested upgrading to fix the recurrent problem we see with random failure to initialise null sets.

I'm now adding somewhat improved support to trace memory leaks (will be uploaded soon).

Simply using the unit HeapMon should show some report about non-released pointers at the end of the program. The unit also provides a function to generate such a report at an arbitrary point during a program run and to an arbitrary file.

The report contains the caller address (which can be turned to source lines using addr2line) which might help find the leaks.

I suppose this is not currently am important thing to you, but when it will become a problem again (to you or anyone else), this should now help debugging.

Frank

Martin Liddle wrote:

...

I suppose this is not currently am important thing to you, but when it will become a problem again (to you or anyone else), this should now help debugging.

We haven't yet located the problem so this will be very useful. Thank you very much.

OK (but you'll have to wait for the next update, perhaps today, or in the next few days).

I'm also adding a shell script (gpc-run) that calls addr2line to translate the addresses into line numbers (provided the program was compiled with debug info).

Frank