Fog Creek Software
Discussion Board




Need a link


I've searched Google and found a few, but I am not so satisfied. Any good reading on "stacks and heaps" about how and when memory is allocated from the heap? I think the intelligent readers of JoS will be able to give me more relevant links than I can search by myself.


Wednesday, June 02, 2004

The answer is: it depends mostly on the language and/or environment that you're using. For instance, you can't answer this question without qualifying it by saying "In traditional C++, ..." or something similar.

Brad Wilson (dotnetguy.techieswithcats.com)
Thursday, June 03, 2004


I was going to add just that actually, "in a traditional environment like C", but it soon hit me that it happens even in Garbage Collection with Java or .NET, so it quite generic to any programming environment.

Estudiantin
Thursday, June 03, 2004


Geee....        ;-0

Estudiantin
Thursday, June 03, 2004

Good topic.

This is much more complex topic than you might imagine.  It so complicated that MOST C/C++ developers don't understand it.  Trust me.  I recommend Richter's Advanced Windows programming if you want to get an understanding of the way the heap and stack work on windows.  Linux is actually pretty similar.

Basically the stack and heap are both allocated out of Virtual Memory.  If pointers blow your mind the double indirection of Virtual Memory managers will doubly blow your mind ;)

In win32 allocating memory is a two step process even though it is a one step process in most languages.  The language runtime must reserve Address space and commit physical memory.  A process has a 32 bit address space, but not all of it is commited to physical memory.  If it were you'd only be able to run one process, and certainly windows would die a horrible death.   

Because of this you can run out stack space in two ways.

1) you run out of address space (ie deep recursion).

2) the OS can not commit physical stack space due to the lack of resources. 

While address space might be allocated for the stack by the OS, only 1meg is commited to physical memory by default on Windows.  You have to play around with the linker switches to change that. 

If either fails a SEH is raised, and generally all hell breaks loose.  In C++ the exception model gets completely screwed up even if you map SEHs to C++ exceptions. 

What does it mean if calling foo() fails while allocating the stack frame?  It is undefined by the specification. 

The really, really odd thing is malloc and new could fail in very similar way to calling a function (remember the stack is just Virtual Memory like the heap). 

That's why I say checking the return values of malloc and new is a waste of time as the next function call you make could fail as well, and when it does, you are pretty much screwed anyway. 

Checking the return value makes you feel like you've done something of value although you really haven't.  This is a huge falacy in modern C++ environments. 

To make a long story short that is why multithreading and dynamic allocation in servers is evil.  You are just waiting for something bad to happen. 

This is EXACTLY the reason I started writing my HTTP validation server and proxy (http://www.summitsage.com/) It allocates all its memory at startup, and uses only one thread.  No way for free to fail or the stack to overflow while under load. 

If you REALLY, REALLY care about uptime, I suggest you study memory management in detail.  It is the #1 cause of software failures. 

Have fun.  It is a huge topic. 

christopher baus (www.baus.net)
Thursday, June 03, 2004

I just reviewed my notes on this.  On Win32 1 meg of Virtual Address space is allocated to a thread (or the main thread of a process) by default.  And only 4k is commited at startup.

That means the OS is dynamically commiting physical stack space much in the same way malloc and new commit heap, in all but the most trivial applicaitons that use only 4k of stack space. 

Again this is a really interesting topic as no one, and I mean virtually no one, addresses what happens in low memory situations like these.  Post an article on comp.lang.C++ on this topic and see what response it brings.

The stack may over flow even if less that 1 meg of stack is used.  Want to know why an applications is dying in low memory situations, even though you've checked the return value of malloc religiously?  Here is a major reason.

 

christopher baus (www.baus.net)
Thursday, June 03, 2004

christopher, spot on.

Although in Symbian programming, we do cope carefully with the equiv to failing heap allocations and where possible recover from them.  Whereas we try to avoid situations where stack is exhausted, since we don't typically have such big stacks (8KB, and iirc stack on Symbian is always commited if I understand your description properly).

i like i
Thursday, June 03, 2004

Did you also searched on groups.google.com?

Also for really technical stuff use this searchengine: http://citeseer.ist.psu.edu/cs

Somorone
Thursday, June 03, 2004

*  Recent Topics

*  Fog Creek Home