Fog Creek Software
Discussion Board




Rewriting Code Is Better Than Working Around Flaws

Interesting tidbits from this ( http://www.gamasutra.com/features/20030710/hargreaves_01.shtml (I think you need to register (free), sorry)) article:

"In a code postmortem at the end of the first MotoGP project, we came up with the theory that rewriting flawed code always saves time in the long run, and is better than trying to work around existing design flaws. "

"Rewriting your entire codebase in two months might sound like a tall order, but that is exactly what we did. Starting with such basic elements as our text rendering system, we rewrote every single UI screen, every speedometer, lap counter, and position indicator overlay, and every bit of game mode logic."

"A month into the project I was worried that we had bitten off far too much, and were never going to get it all working in time. But after six weeks it was working well enough that we were able to start moving screens around and experimenting with different layouts, taking advantage of a flexibility we never had before. After seven weeks, we shipped what one gamer called "the best example of what an Xbox Live game can be". Today, we have a solid codebase that was a pleasure to work with as we moved on to MotoGP version 2."

"I can now confidently state that rewriting huge swathes of code takes a lot less time than you might expect, and the benefits can be enormous."

Just me (Sir to you)
Tuesday, July 15, 2003

Certainly true of the flaws are so fundamental. Should make you think how it got so flawed in the first place, though. To avoid repeating the same mistakes, of course.

But fundamental flaws almost certainly mean there is something wrong with the design, not necessarily individual algoritms or data structures.

Practical Geezer
Tuesday, July 15, 2003

This sounds either like they've actually been refactoring with a proper plan and schedule rather than rewriting, or that they set an unreasonable task of completely rewriting the code from scratch and got their superstar developers to work 24/7 fixing all the new bugs they'd created in the rewrite.

My gut feeling is they did the latter. And what's the bet that they think that since they got away with rewriting it once, they can do it again? Ha ha ha ha ha.

From my experience, it is usually better _not_ to rewrite code, but not necessarily always. If you have a small module that's written by a bozo in spaghetti, then there's a chance your rewrite will on average contain less bugs when it's finished, but even then it's not guaranteed. Bozos seem to be able to come up with the most elaborate coding subtleties and misleading / pointless / irrelevant comments.

Better Than Being Unemployed...
Tuesday, July 15, 2003


Only if you have a better design in mind and you have time to make and test all the changes.

I've rewritten huge amounts of my own code to obtain a better result.

It is worth it in the long run (easy maintenence, etc.) as long as it's a better design than the original.

~~~~~ ~~~~~~
Tuesday, July 15, 2003

That reminds me of a story I read about Ken Thompson.  I heard that he used to never just fix a bug in a function.  If there was a bug in a specific function, he would rewrite it from scratch, under the logic that if there was one bug there, there was probably another.

Incidentally he did this type of development on a line editor as well, and when shown a graphical editor (maybe it was vi?) he said (paraphrased): Thats neat, but I like to keep it all in my head...

Andrew Hurst
Tuesday, July 15, 2003

Most developers are not Ken Thompson, though.  Rewriting a function to remove one bug is a good recipe for introducing two more.  Maybe it works for him, and that's great, but I dispute whether it works for most people.

According to the original article, the existing code was completely inadequate for their current needs ("our UI system provided none of [the UI requirements we identified during initial analysis], and had already proven awkward to work with even for the simpler situations encountered in a single player game").  I'd support the assertion that the code needed to be rewritten, just like you may need to rewrite the code for the original Asteroids if you were developing it as a 3D multiplayer game.

However, this is not a scientific proof that rewriting code is better than the alternative, because the alternative was never tried.  Was it better?  We'll never know.

Brent P. Newhall
Tuesday, July 15, 2003

If they actually rewrote their application from scratch in two months, it's not much of an application.

Philo

Philo
Tuesday, July 15, 2003

There's no question that, in the long run, sometimes you're better off rewriting the pieces of code that make the design flawed rather than working around those flaws.  The question is when does it make sense?  You need to identify the flaws, figure out which code embodies those flaws, and then rewrite.  Sometimes this is 80% of your codebase, which makes it tough.  Sometimes you identify the wrong code and rewrite a bunch of stuff that didn't need it.  Sometimes it's just a few small but important glue-type pieces that can be rewritten in 6-8 weeks.  In the latter case, hell yeah- bite the bullet and do it ASAP.

Ken
Tuesday, July 15, 2003

Sure about that Philo? After all, you delivered an application in 6 weeks just recently, didn't you?

Some people, I swear
Tuesday, July 15, 2003

Just to remind people, I think that the main point of Joel's article which I presume prompted the original poster to start this thread was that your existing code embodies a zillion little lessons learned and bugs fixed that aren't captured in the project documentation, interface, or anywhere else. So if you re-write from scratch you'll have to learn almost all those lessons over again. And that's what's going to kill you.

Joel's article is at:
http://www.joelonsoftware.com/articles/fog0000000069.html

And I'd like to make another point that game development is quite a different environment from business software. And the lessons of one don't necessarily apply to another. For example, it's extremely rare for game software to be maintained and have significant new features added year after year after year, as is usually the case for business software.

Bill Tomlinson
Tuesday, July 15, 2003

Yeah, and it wasn't much of an application, esp. since most of that six weeks was conquering technologies instead of actually writing code. It's certainly not something I'd use to brag about rewriting from the ground up as an alternative to refactoring.

My EDI engine, which took eight months to write, is something I'd use to test the "rewrite is better than refactor" allegation, since it's got a bajillion* little nits and workarounds to support a myriad of business rules.

Philo

*term of art

Philo
Tuesday, July 15, 2003

To quote Joel's article, linked above by Bill:

Yes, I know, it's just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I'll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn't have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.

None of these are issues apply to the xbox.  The hardware is new, and constant.  Nobody out there has an xbox with half the standard ram.  The software doesn't have to run on anybodies Intellivision. 

The vast difference between coding games and business software was covered here:  http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=56172&ixReplies=35

Ged Byrne
Tuesday, July 15, 2003

They didn't rewrite the game, they rewrote the UI. The UI for a game like this is maybe 5-10% of the code.

xyz
Tuesday, July 15, 2003

>> For example, it's extremely rare for game software to be maintained and have significant new features added year after year after year, as is usually the case for business software. <<

Though I guess the definition of "extremely rare" is somewhat subjective, from my observations of gaming development this isn't at all true.  The Unreal Engine comes to mind as a codebase that has been used in countless games over the past five years, with new features added as different games required them or new hardware supported them.  There are other examples of this besides the Unreal Engine (such as annual updates to sports games), but the Unreal Engine is popular enough that I think it disproves "extremely rare" in and of itself.

I didn't register and read the article yet, but my guess is that they started writing without much direction, ended up with garbage for code, and had to do a rewrite to get it in order.  This isn't exactly uncommon in business software either.  The Mythical Man Month goes so far as to suggest "Plan to throw one away". 

Anyway, rewriting before the initial release is an entirely different thing than rewriting after version 4, which is what Joel is talking about in that article. 

SomeBody
Tuesday, July 15, 2003

Ok ok, hold your horses.
The reason I brought this article in the spotlight is not to take on a simple stance saying "Thou shalt dump and rewrite at every turn of the wheel".
The article just presents a point that shows that not all rewrites are disasters.

Let me qoute also the beloved Paul Graham. In another context he writes: "awkward systems yield meatier papers, because you can write about the obstacles you have to overcome in order to get things done. Nothing yields meaty problems like starting with the wrong assumptions"( http://www.paulgraham.com/hp.html ).
Now even though he is writing about science,  think we can safely transpose the gist of this to software systems.
You have to recognise when all the "hair" is truly a part of the intricacies of complex domain knowledge, or wether it is a fungus that just grew out of strating out from the wrong assumptions. In other words, not every big hairball is a sign of deep knowledge gained through blood, sweat and tears. Sometimes it is just a result of trying sculpt icecream with a chainsaw.

Just me (Sir to you)
Wednesday, July 16, 2003

I have done this in programming environments that truely sucks. The debugging error is not helpful, and nothing leads to anywhere no many how hard you try to cajole information out the run-time. You end up rewriting 500 lines of code. The best way to avoid this is to write unit tests for small modules of your code. But if you are in a rush that little script that had to be done an hour ago will never be done until next week.

Li-fan Chen
Wednesday, July 16, 2003


"You have to recognise when all the "hair" is truly a part of the intricacies of complex domain knowledge, or wether it is a fungus that just grew out of strating out from the wrong assumptions."

Sure, but you also have to question if is necessary to remove that hairball. As other person puts, "let the sleeping dogs lie."

Leonardo Herrera
Wednesday, July 16, 2003

The UI code for a game isn't going to have 27 other applications which depend on it...

Joe Cuervo
Wednesday, July 16, 2003

>Sure, but you also have to question if is necessary to >remove that hairball. As other person puts, "let the >sleeping dogs lie."

Yes sire, as goes the saying "Don't trouble trouble till trouble troubles you" :)

The One You Loved (TOYL)
Thursday, July 17, 2003

Some game projects are very poorly run, which is disastrous given the technical complexity that's often required.  Instead of developing a good architecture, and taking time to refine it, developers get pushed into adding band-aids all the time. This occurs where the company or project management lacks programming experience.

In this case, it sounds like they had complete junk to start with.

.
Friday, July 18, 2003

*  Recent Topics

*  Fog Creek Home