The Old Joel on Software Forum: Part 3 (of 5)

Debug vs Release build

Many of us have been bitten by an app behaving differently under debug and release builds. What are common tips/tricks that the JOS audience can suggest to avoid the release build problems?

Let me start: MS VC++ initializes local variables to 0 in the debug build, but not in release. Other problems/gotchas, anyone?

Thanks!

Floridian
Wednesday, December 3, 2003

Regular "Release" builds on clean machines.

If you have a build schedule, make a release build and test it on a nice clean machine (VMWare and Virtual PC can help).

Gp
Wednesday, December 3, 2003

http://www.codeproject.com/debug/releasemode.asp

.
Wednesday, December 3, 2003

For debugging, I highly recommend Microsoft's free "Windows Application Verifier". It can detect all sorts of runtime errors like heap corruption (PageHeap), lock misuse, bad handles. It's like Purify or BoundsChecker, but it uses debug features in Windows memory management.

http://www.microsoft.com/windows/appcompatibility/appverifier.mspx

runtime
Wednesday, December 3, 2003

ASSERTS are not active in release builds....so in case the side effect is important use VERIFY instead

e.g ASSERT ( f() ) which checks to make sure that the function f() returns a non-zero value. In release mode f() will not be called at all. To avoid this use VERIFY ( f() )

Code Monkey
Wednesday, December 3, 2003

Working with a lot of different codes and computers, I see this all the time. I do however not work in Windows. All compilers have problems like that. Optimizations levels can bring in problems, a bit like Debug/Release. The code can be perfectly correct and still be too much for the compiler. There are no shortcuts. QA is important.
Automated unit testing, or something will help. It depends on the program how you can set it up, but the easier it is to test the code the more frequent it will be done. Add timing and you will also be able to have a history of how computeintensive part change.

What may help is to have a clean code, not too big source files, no extraordinary strange loop structures or strange references. Depends a bit on the language you are working with. Sometimes a restructuring of the code is the only thing that works. Even a simple print statement can get the compiler on track, then you know where to restructure.

Often you find a simpler solution at the end anyway, so it is worth the extra work restructuring.

Fredrik Svensson
Wednesday, December 3, 2003

Another ASSERT variation that's a "classic" debug vs release issue:

if(blah)
ASSERT(something);

nextThing();

In release, the ASSERT gets stripped out, obviously, and nextThing() only gets called if blah is true...

A PS2 Programmer
Wednesday, December 3, 2003

To A PS2 Programmer,

That is incorrect.

danpop
Wednesday, December 3, 2003

The famous one in MFC-land is that incorrect signatures in message map entries work fine in Debug, but crash in Release. This is devilishly hard to find the first time it happens to you.

Here's an example of a correct declaration:

//{{AFX_MSG(CMaintBatchDetailTree)
afx_msg LRESULT OnMumbleFoobar(WPARAM wParam, LPARAM lParam);
//}}AFX_MSG

If you incorrcetly declare (and elsewhere define) it like this:

//{{AFX_MSG(CMaintBatchDetailTree)
afx_msg LRESULT OnMumbleFoobar( );
//}}AFX_MSG

It will very probably work just fine in Debug mode, but crash in Release mode. I've been bitten by this mostly when I cut and paste message map entries.

Grumpy Old-Timer
Wednesday, December 3, 2003

Also a Vsiaul C++ short article regarding this is available @ http://codeguru.earthweb.com/debug/ReleaseMode.shtml

Code Monkey
Wednesday, December 3, 2003

danpop - It's not nice to say "That is incorrect" without any further explanation. I happen to agree with you in this case, but if you're posting to help someone rather than just feel good that you're smarter than them, then you should give an explanation.

I think your reasoning is:
"ASSERT(something);" will evaluate to ";" in release, and so it will work right. Perhaps PS2 was thinking of where ASSERT(x) is defined to "<something>;", and you write "ASSERT(something)" in your code?

Exception guy
Wednesday, December 3, 2003

Make SURE you are compiling with the highest warning level (which is 4 at the moment). This will point out some things that might cause different behavior between the two.

With ASSERTS, test values, not "complex" statements with side effects. Example:

ASSERT((fp = fopen("foobar", "r"))); // BAD

fp = fopen("foobar", "r");
ASSERT(fp); // Good

njkayaker
Wednesday, December 3, 2003

"Perhaps PS2 was thinking of where ASSERT(x) is defined to '<something>;', and you write "ASSERT(something)" in your code?"

That's why you should never write this:

if (foo)
bar;

... but should always do this instead:

if (foo) {
bar;
}

How often do we get to say "always" and "never", anyway?

Grumpy Old-Timer
Wednesday, December 3, 2003

Um.. never? :)

sgf
Wednesday, December 3, 2003

Exception guy: yes, you’re right. I should’ve explained myself.

Speaking of ASSERTs: many people do not know how to use them. An ASSERT should express a fact (logic, business, etc), rather than an error, as it is so commonly done. For instance, in the previous example, opening a file COULD fail so it is nothing to ASSERT, it is an error that should be dealt with.
Here is something that I recently used where I think an ASSERT is properly used:

Input: std::vector<T> v; v is supposed to be sorted…. (is_sorted<T> is a function that verifies whether an array is sorted)

…….
ASSERT(is_sorted(v)); // verify that v is sorted
do_something(v);
ASSERT(is_sorted(v)); // verify that v is still sorted
………..

danpop
Wednesday, December 3, 2003

Right, that's why you should *never* use the VERIFY macro. Use ASSERT for assertions in debug builds, and real honest-to-goodness error-checking in release builds.

For most people, VERIFY is a subtle way to avoid doing error-checking.

Grumpy Old-Timer
Wednesday, December 3, 2003

A higher-level method we used to use is to have a pre-release build just before the release, with enough time to handle small-to-medium problems before the release build (varied from program to program, usually 1-2 days, but one big program used 4).

I liked the concept; if there weren't any problems generated from the pre-release, we used the time to catch up or get a head start on whatever's next. Gave people some slow time to catch their breath.

van pelt
Wednesday, December 3, 2003

Thanks to all for the info! Now I'm armed to find my problems!

Floridian
Wednesday, December 3, 2003

Yes, one would not use ASSERT on an fopen. It was just a quick illustration of the side effect problem.

I tend to think of ASSERTS as guides to the programmer or very-early testers. They are a way to document that "this is not supposed to happen" or "you are using this incorrectly" as code instead of as a mere comment. Thus, they say "fix this now" when they report an error.

They really don't make a program more reliable in production because they go away in the release version. If testing is done on the release version, they are no help. If you use ASSERTS, you may also need release error reporting.

njkayaker
Wednesday, December 3, 2003

My solution is to not have a different debug
and release build code base. It's the same
code that runs all the time. Test what you
ship and ship what you test.

What you would compile out make settable
by level or efficient enough it can always run.

son of parnas
Wednesday, December 3, 2003

Son, you are missing something. The problem, at least for my team, is that we do test and release the release version, but often a bug is reported, we go to debug the code, and it works fine. I think that's the issue being raised in this thread.

GML
Wednesday, December 3, 2003

GML, you are missing something. Son of Parnas is saying that you should not waste your time debugging the debug builds. As you pointed out, it has not helped you find these (particular) bugs. He is recommending that you only debug your release builds.

runtime
Wednesday, December 3, 2003

I guess I can throw in my missive about overuse of asserts in general. Overuse? Yes.

RUN-TIME CHECKING, with extensive, tunable run-time logging, can help find the most devilish problems and provide an audit trail for your support team.

My simplified example is:
...
do {
if ((rc = TableOpen(...)) != 0) {
Log("Couldn't open table");
break;
}
bTableOpen = TRUE;
if ((rc = TableLock(...)) != 0) {
Log("Couldn't lock table");
break;
}
bTableLocked = TRUE;
if ((rc = RecordLock(...)) != 0) {
Log("Couldn't lock record");
break;
}
bRecordLocked = TRUE;
if ((rc = RecordUpdate(...) != 0) {
Log("Couldn't update record");
break;
}
} while (0);
if (bRecordLocked) {
if ((rc1 = RecordUnlock()) != 0 && !rc) {
Log("Couldn't unlock record");
rc = rc1;
}
}
if (bTableLocked) {
if ((rc1 = TableUnlock()) != 0 && !rc) {

Log("Couldn't unlock table");
rc = rc1;
}
}
if (bTableOpened) {
if ((rc1 = TableClose(...)) != 0 && !rc) {
Log("Couldn't close table");
rc = rc1;
}
}
return (rc);

IOW, everything is checked _at run-time_. Period. Everything can be tracked, depending upon the trace level of logging.

My key finding over 20+ years: few applications are so performance-intensive that you can't afford checking whether a pointer is NULL or not or verifying a return-code came back the right way.

IMO, a big reason most software is unreliable is that "go-naked" tools like ASSERTs are overused (so called, because is in RELEASE mode, you're going nekkid :-).

doug badblue com
Wednesday, December 3, 2003

One problem with release builds is that they aren't very debuggable. If your program crashes, the stack trace is usually useless. Even if you disable frame pointer omission you can't tell what the variables are without recourse to the (obfuscated) disassembly. What a pain!

Some people say you shouldn't use debuggers. I disagree, but if you don't like them then it's true that you may as well not bother with debugging builds.

(It's been my experience that you can program without introducing any bugs that appear only in release builds. It just takes a bit of practice. This will of course skew my opinion.)

Insert half smiley here.
Wednesday, December 3, 2003

Our core product has:

ASSERT(ptr != NULL);
ptr->Foo();

in over 3,000 places...

Skagg
Wednesday, December 3, 2003

>>
Let me start: MS VC++ initializes local variables to 0 in the debug build, but not in release. Other problems/gotchas, anyone?
<<

Uh, no it doesn't. In debug builds, it initializes local variables to 0xCC to make unitialized values obvious. It also initializes dynamically allocated and freed memory to similar values to likewise make them obvious.

I have a vague memory of what you said being true in the past but it certainly isn't the case for version 6 with the latest SP and version 7.

SomeBody
Wednesday, December 3, 2003

"doug badblue", you are my kind of programmer. Ours is the one true path, because you and I can fix any bug in an *instant*. Here's an interesting article on the subject:

http://www.windevnet.com/documents/win0302a/0302a.htm

Unrewarded Genius
Wednesday, December 3, 2003

I've used this approach in the past. It is especially good when you can recover from errors like unexpected NULL pointers

#define MYASSERT(x) if(!(x)) { throw MyAssertException(__FILE__,__LINE__);}

(writing the above from memory - please don't get sidetracked by syntax errors...)

Then I catch MyAssertException at a high level in the code and act appropriately.

A real example: The code I was writing was a plugin to another program that exported that program's documents in a custom format. I wrapped the main export process in a try...catch block. If there was an assertion failure in the export I didn't have to bring the program down. I could pop up a dialog suggesting the user save their file (to a different name!!) and restart the program.

This discussion really reminds me why I much prefer Java these days.

AndrewR
Wednesday, December 3, 2003

well, one app our developers worked on (previous employer by several generations) was somewhat non-deterministic -- how it worked depended just a bit too much on timing of certain events in the app, which meant that program flow ended up depending upon execution speed.

the net result of it was, we in my team (not the developers) discovered somewhat to our surprise one day, was that our company's 'released' code was in fact the 'debug' code.

when we asked about it, the head-honcho developer basically told us that the app didn't work right unless it was compiled as a debug build. the timing was wrong otherwise, and it just wouldn't work. they hadn't figured out why, so they'd just gone ahead and released the debug build.

ouch. it's a good place to be gone from.

horseWithNoName
Wednesday, December 3, 2003

.. just to add another one.

with VC++ uninitialized automatic variables are often set to 0 in debug mode.

void foo()
{
int a;
printf("%d",a); // gets you 0 in debug mode. Ouch!
// good source of release mode crashes.

}

Michael Moser
Thursday, December 4, 2003

You should get 0xCCCCCCCC in your auto variables in a debug build. Make sure you are specifying /GZ on the command line. It's very handy -- this sticks out a mile in the debugger, and if you use floats you'll quickly get to recognise its float equivalent. The VC++.NET compiler has even better options.

Insert half smiley here.
Thursday, December 4, 2003

Grumpy Old timer said :

>Right, that's why you should *never* use the VERIFY macro. >Use ASSERT for assertions in debug builds, and real >honest-to-goodness error-checking in release builds.

I don't think you know how the VERIFY macro operates...VERIFY is not a way to show a message in release mode...in release mode VERIFY just evaluates its arguments and does nothing no matter what the result is.

>For most people, VERIFY is a subtle way to avoid doing >error-checking.

Wonder where I can meet these "most" people :-) Around here ASSERTS are used to check conditions which you know are true or which rarely ever fail but which you still want to cross check just in case Mr Murphy intervenes.

So you would check for a fopen call with a a normal error messsage but it is perfectly normal to check for the corresponding close with an ASSERT since it rarely if ever fails if you are closing a valid file opened earlier.

Show me a programmer who checks each and every function call for each and every error and I will show you a programmer who is writing a "hello world" program :-)

Code Monkey
Thursday, December 4, 2003

"Show me a programmer who checks each and every function call for each and every error and I will show you a programmer who is writing a "hello world" program"

Show me one who doesn't, and I'll show you an amateur hack.

I check virtually every return code (not the silly ones, of course, like the return value for printf, but everything else). This makes the debugging of problems trivial, because I always know exactly what went wrong, on which line of code. (Can you say that?) My logging class automatically logs all available error numbers and messages for every kind of error and exception.

So, what takes most people hours or days to find and fix, I can do in minutes. I never spend time wondering how this function could have possibly failed: when a (fatal) error occurs, my classes and macros capture the values of local variables, the call stack, and (when I want it to), a dump of the values in specific whole "branches" in the Registry. And there's a lot more.

This kind of thing, BTW, is why the top 5% of programmers are 20 times as productive as the other 95% - I spend zero time *hunting* for bugs. Bragging? Yup - it took me a long time, and a lot of hard work, to get here.

Chipper-Dipper
Thursday, December 4, 2003

hi
I want to write a program with VC++ about read and write from PS2 or USB what can I do?

bita
Monday, May 10, 2004