Fog Creek Software
Discussion Board

Data Access code and Refactoring


I was curious about your post.  Do you find that most of the time you write code that gets the job done, but then later come back and refactor which result in nice objects like the entity objects that you mentioned?

I find myself doing that although there's a little "code angel" that sits on my shoulder telling me to write the entity objects first.  I just find it more applicable to write code that solves a problem and then come back and make it pretty.  The solve problem->refactor coding cycle works well for me.  It's sorta like the "make it work then make it perform" approach.

Ideas or thoughts?

Richard Caetano
Friday, January 3, 2003

In almost every large project I've worked on, I've done just what you described. Both CityDesk and FogBUGZ started out as big hairballs. FogBUGZ got massively refactored last year (I wrote an article about it here somewhere); CityDesk is in much better shape but will undergo the same thing this year.

Joel Spolsky
Friday, January 3, 2003

You gain a lot greater understanding of the problem involved by plowing through it once first, even if the solution might be messy and/or fragile. The first attempt, focused primarily at just basically solving the problem serves almost as a working prototype one of whose primary purposes is to be as much a tool for analysis and requirements refinement as it is to satisfy the end objective.

This works in other fields of course, too. When working for an analytical agency some years ago doing Operations Research work, one of our chief analysts formalized what they called a 'Model-Test-Model' approach which was basically the thing you're talking about. There the 'model' they referred to was usually one of our force-on-force combat computer simulations. They'd do a simulation run (well, often a set of runs), get some answers, then refine the model or the parameters to clean up problems they found or to reflect understanding they gained, then more runs, then refine, etc.

This helped especially for problems that were particularly complex or somewhat nebulous for some reason or other (like Joe's 'hairballs'), where maybe they needed to learn more about what questions they needed to be delving into. As Sarek of Vulcan said on one of the ST movies 'It is hard to answer when one does not understand the question', or words to that effect.

M-T-M was not as necessary or efficient when the problems could be better defined initially and the particular phenomena were better understood--equivalent to when you're developing a smaller, simpler, or more familiar application where requirements analysis and design could be much more complete right out of the gate.

For my money, this points out why there are such problems with trying to stick to pure waterfall for big, complex, or ill-defined projects--you can only guestimate so much about the problems when all you have are static abstractions of the system.

In fact, I wonder if anybody really uses pure waterfall anymore for anything but smaller, very clear-cut problems?


Saturday, January 4, 2003

Oh - and from the perspective of testing, I generally follow a similar pattern. Of course it helps that our shop works iteratively on pretty much everything, because that tends to naturally reinforce me using this approach. In general, though, I'll normally start off with plowing thorough the main workflows, following the 'happy paths', seeking to understand the system and watching for branches in the flows and potential error conditions I'll want to come back and add tests for later.

This is especially true when we haven't been able to do as much up front analysis as we would have liked, and therefore I haven't been able to do as much preliminary test design as I would have liked.

Saturday, January 4, 2003

The article Joel wrote is here:

John Topley
Saturday, January 4, 2003

I find that it is a good approach if you have the opportunity to do both the barely-working way and the clean neat way before releasing it to production.

But if you don't have that luxury, I think it is better to do it clean the first time, instead of having someone else other than the original developer picking up the code 1 year later to try to figure out the old hodge-podge and risk breaking other code that was since built on top of it.

T. Norman
Saturday, January 4, 2003

THing is, after you two or three porjects, you realize that you are solving the same problems over and over.  That is how frameworks develop.

In the web for data entry it is almost always, search, fetch values, display values, get input from form, validate, store or report error.  The Form is in a client area of the screen, with navigation information around the edges.  You drill down into information heirarchically.  The parts of this system that should be easily config/findable are:  The values on a give form and the drilldown heirarchy.  I think I would map out how I was going to to solve these problems in the abstract before tackling another one of the type of projects.

Adam Young
Saturday, January 4, 2003

I'm with Richard and Joel.

I've tried it both ways and the way to do it is to go for the big ball of mud design pattern first and then refactor it. A good time to refactor a section is of course just before trying to add something to already working code.

The problem in architecting something that has never been done before is that until you have implemented it, you don't really know where the problems are. Come up with your mud ball first and see where the med is especially lumpy.

Years ago we realized that 'premature optimization is the root of all evil'. This is because we can not predict where the actual bottlenecks will be until we are finished and run the experiment of doing profiling. Everyone knows this is true now and no one reasonable optimizes before its time because you end up messing with the wrong stuff.

Today, we have a similar situation but on a higher level.

Here is my new wisdom for the modern era:

"Premature Architecture is the root of all evil." -- X. J. Scott

X. J. Scott
Saturday, January 4, 2003

BTW, I am a huge proponent of refactoring.  Fowler should get an honorary degree for what he did with that book.

Yes, I know it was the subject of his Doctoral work.

However, many a project is in desperate need of refactoring , with out enough time alloted to do it.  Many a project suffers from Architectural Arthritus (I made that term up, but feel free to use it) that you just have to deal with it.  And sometimes, you have already solved a problem, so there is no use in avoiding it just to get right into the solution.

Personally, what I like to do is to map out the objects/entities of a project right at the start, and then use that map as a guide.  It serves the role of the common Metaphor from XP.

Adam Young
Saturday, January 4, 2003

When I was a graduate student I once took a course in how to write papers. The idea was to start out with rough drafts and outlines until you get a general idea of what you want to say, and then just keep revising and re-writing until the paper is good enough (or you run out of time).
When I started programming I applied the ideas I had learned about writing papers (start with rough drafts and outlines, revise and revise). I assumed I was doing it all wrong, but I had learned to like working that way. Then I found out I was refactoring, and now it's considered a reasonable way to program.
Before graduate school I used to hate revising, and wrote papers once from beginning to end. Maybe refactoring, like revising, is a practice you have to learn to appreciate.

Saturday, January 4, 2003

My own experience with architecture/refactoring was in a dot com company. I designed a very nice set of objects that mapped a lot of common operations in a rather large website. It was cool, a lot of functionality got created in short development cycles. But then, something strange happened: the mission of the company actually changed two or three times in a short amount of time, so the web app had to change accordingly.

As a result, the programmers in this project started doing workarounds to solve the lack of functionality (instead of refactoring the objects), and in the end we got our own hairball, complete with strange wiring around the initial entities.

Now I'm minimalist :-)

Leonardo Herrera
Saturday, January 4, 2003

I believe this is similar to "bottom-up programming."

Sunday, January 5, 2003

When I think about refactoring, I am reminded of a quote that I read a few years back (unfortuantely I don't remember where I read it or who wrote it).  It goes something like this:

Perfection is achieved not through the perfect use of a pencil, but rather by the liberal use of an eraser.

Brian B
Monday, January 6, 2003

REGARDING: [[The problem in architecting something that has never been done before is that until you have implemented it, you don't really know where the problems are. Come up with your mud ball first and see where the med is especially lumpy...
"Premature Architecture is the root of all evil." -- X. J. Scott ]]

The problem is really that too many "architects" are not experienced enough for the job.  After doing development for enough years over enough projects, some are able to recognize when a new concrete problem is similar at an abstract level to ones they have solved before.  The longer one does software development the more one realizes that there is practically never "something that has never been done before", especially when one finally realizes that even if it is new to me, there are others out there that have encountered this and have shared their wisdom about it. (design patterns ring a bell?)

So when, to young turks, everything looks like a ball of mud, they create methodologies like XP that give them dispensation to code first, think later. When an architect actually has decent development experience (and knows to tap into the experience of others as well), picking an architecture, applying patterns, etc, is a huge productivity booster relative to reworking code over and over.

old fogey
Saturday, May 10, 2003

*  Recent Topics

*  Fog Creek Home