The Old Joel on Software Forum: Part 4 (of 5) - Out of memory--what should I do?

Out of memory--what should I do?

Ok, my library's churning along just fine. It's moving things around, making lots of noise, and spitting stuff out, when all of the sudden, it asks the heap for just a tiny bit more memory—and the heap says no.

What *should* it do? If a call to malloc() or realloc() fails, what's the best practices for library routines? My first instinct is to just return NULL or 0, but then you end up with situation where every function has to check the return value of almost every other function, leaving you with overly verbose syntax to make sure the stack unwinds correctly:

HTTP_Request *http_get_request(SOCKET socket)
{
HTTP_Buffer *buffer = http_new_buffer();
if (buffer == NULL)
return NULL;
...
}

...

HTTP_Buffer *http_new_buffer(void)
{
HTTP_Buffer *buffer = (HTTP_Buffer *) malloc(sizeof(HTTP_Buffer));
if (buffer == NULL)
return NULL;

buffer->type = ...
}

I know some big, important libraries (like libghttp) just ignore this problem all together and don't even bother to check malloc() or realloc() return values, and other libraries just exit() or abort() when malloc() or realloc() fail.

This got me thinking. If each routine returns NULL/0 when there's no more memory available, then you've set up a potential point of failure for almost every single routine, and if the user actually does their part and bothers to check each and every return value, then what exactly can they do to handle the error _besides_ exiting or aborting the process?

My library isn't graphical, and it handles allocation and deallocation through custom routines. It also allows for user-defined error handlers, so a real attractive solution to me would be to have these custom memory routines call the appropriate handles when allocation fails, letting the user abort or whatnot, and then just ignore the return values of each allocation function called within the library.

Suggestions? Comments? Anecdotes?

A drifter lost in Philadelphia
Sunday, March 28, 2004

"call the appropriate handles" should have been "call the appropriate handlers."

A drifter lost in Philadelphia
Sunday, March 28, 2004

Throw an out-of-heap exception.

Compact the heap and try again.

See also "Static Allocation Pattern" and "Pool Allocation Pattern" at http://www.informit.com/articles/article.asp?p=30309

Christopher Wells
Sunday, March 28, 2004

This is the age of virtual memory. Unless your library's running in an embedded system, or you know you'll be using truly massive amounts of memory in regular use, I wouldn't bother dealing win out-of-memory, cause it just isn't going to happen.

Chris Tavares
Sunday, March 28, 2004

The way you deal with this without having explicit checks everywhere is with exceptions.

Dan Maas
Monday, March 29, 2004

If you are writing a general purpose, you should probably return NULL and let the caller decide. The library should ideally implement mechanism without policy.

runtime
Monday, March 29, 2004

In Symbian apps, you (should) write your app to use strategies when memory is low;

for example, an app might open several documents; when memory is low, documents not on top are delta-ed back to disk if diskspace allows.

i like i
Monday, March 29, 2004

Wonderful how this question comes up so soon after a rehash of the exceptions debate. If I read Joel's take correctly, he might recommend sprinkling out-of-memory checks everywhere you ask for memory, lest you have too many possible exit points.

http://www.joelonsoftware.com/items/2003/10/13.html

I wouldn't take Chris Tavares' above advice for anything but throwaway code. Virtual memory has (and should have) limits too, otherwise a runaway operation could fill up your disk and bring your entire system to its knees. That's the kind of carelessness that makes IIS vulnerabilities a monthly worry or makes rockets crash.

Dan Maas tells it true. You've described a textbook example recommending exceptions, and a nice demonstration of why they're so very excellent. You can pass the fact up using an exception to wherever is best to decide a response, and the bulk of your code will look as uncomplicated as if you took Chris Tavares' advice. You'll also have runtime's desire for mechanism without policy.

veal
Monday, March 29, 2004

Veal

You say :

>>That's the kind of carelessness that makes IIS >>vulnerabilities a monthly worry

What month was the last IIS vulnerability ?

Damian
Monday, March 29, 2004

How to handle an OOM condition may be a moot point on different platforms. Linux (and I believe, FreeBSD from some googling), will always return a non-NULL pointer from malloc() [or more properly, from brk()]. They use an overcommitting strategy; pages of RAM are allocated when you access them. If there's no system memory left at that point, something bad will happen (depending on configuration of the OOM killer, your process may be killed, or the process using the most memory, etc.).

David M. Cooke
Monday, March 29, 2004

The way we do it is the following:

o By default, we use malloc and don't check memory
errors. Unwinding would be generally too troublesome
and would add too much code - we have a set of API's used
in an embedded system.

o For users of our API's who need to catch memory errors,
we allow them to register callbacks which we use instead
of malloc(). These callbacks allow files to be
closed, etc, and are expected to do a longjmp out of our
world and clean up memory. After this, they can reinit
our API.

This approach isn't pretty, but customers have generally
been happy with it. Like all things, it's a compromise.

x
Monday, March 29, 2004

Exceptions alone are not the solution. You also need a strategy for cleaning up resources consumed while unwinding the exception.

C++ has RAII, and resource freeing is done automatically in destructors as the stack is unwound. With Java or C# you don't care, as the garbage collector will get around to things anyway.

The OP didn't specify his runtime environment, but if it's plain old C on an embedded system, then "exceptions" (i.e. setjmp/longjmp) aren't going to help.

David Jones
Monday, March 29, 2004

brk() doesn't always return more memory. It can fail: in the case of being unable to allocate more VM resources. You may, at that point, be out of process address space (often limited to 1,2 or 3 Gb depending on OS), out of system address space or out of page table entries.

Those are YUK situations, and frankly, reaching them at all is very very norty...

Katie Lucas
Monday, March 29, 2004

The library is a general purpose C library.

A drifter lost in Philadelphia
Monday, March 29, 2004

David Jones is, of course, correct that exceptions are not the solution in languages that do not have them.

In Java and C# you do, however, care about releasing non-memory resources like open connections. Relying upon the owning object to be collected and finalized as a resource release mechanism is not wise, since you have no basis to think the object will be collected any time soon. Fortunately we have a very simple guarding idiom to ensure our resource allocations unwind nicely.

acquire x
try {
work with x
} finally { release x }

veal
Monday, March 29, 2004

If you can, implement the library in a language that offers exception handling, then at the API level, catch any exceptions and consistently return error codes across all functions.

If the user of your API is using a language that supports exceptions, they can wrap the API calls and convert error codes back to exceptions, thus never have to worry about checking return codes.

Big B
Monday, March 29, 2004

> If each routine returns NULL/0 when
> there's no more memory available,
> then you've set up a potential point
> of failure for almost every single
> routine

So what you're saying is that since almost every routine can fail, every routine needs to inform its caller of the failure somehow. Sounds sensible to me.

> what exactly can they do to handle
> the error _besides_ exiting or
> aborting the process?

You can't possibly know, so don't even try to guess.

I worked for many years on the script engines, and you know what? The way IE handles an out-of-memory error is VERY different from the way ASP handles an out-of-memory error.

You're writing a general-purpose library, so by definition, you don't know the scenarios in which it will be used. You'd better believe we wrote the script engines to handle out-of-memory, out-of-stack, all that stuff, because you never know whether its going to be used on an overstressed server box or a client with a tiny amount of physical memory.

In the first build of the script engines, there was ONE place where we forgot to check the return value of malloc. It was less than a month before we got a crash dump from an ASP user who had run out of memory at that exact place and crashed the process because we didn't handle the error. Murphy's Law -- if it can go wrong it will -- is practically a mathematical certainty when writing general-purpose software.

You CANNOT assume that malloc will succeed, and you CANNOT assume that you know better than the caller what to do about it.

Now, that's not to say that its necessarily the case that return values are the right way to implement error handling. I like your idea of an immediately called "on error" method which can do the caller's work right there.

In the script engines we implemented both. When an error occurs, the engine calls the host back on an "on error" routine. The host then tells us whether to ignore the error and continue, or record the error information and clean up. (This is how you can get that "a script error occurred -- ignore it?" dialog in IE.) If the host wants us to record the error info and abort, then we start returning a special "a recorded error occurred" message through return values.

The reason we have to have both return values and the handler is because there are situations in which we cannot call the handler -- an "out of stack" error, for example, but there are other, more obscure cases as well.

Eric Lippert
Tuesday, March 30, 2004

Thanks for all of the helpful replies, especially Eric, Katie, and Veal. I've decided to just stick with the NULL/0 combo, as verbose as it might be.

I really wish C itself provided much cleaner, neater ways to handle errors. Not even exceptions, just some Perl-like statement modifiers to save you from haveing to write awful looking stuff like "if (!kill_port_moniter()) return NULL" (e.g., "kill_port_moniter() or return NULL", or "return NULL unless kill_port_moniter()").

A drifter lost in Philadelphia
Wednesday, March 31, 2004