Fog Creek Software
Discussion Board




Just write the code...

I'll get you the database structure later.

A "senior" developer told me this today.  I found it more than a little amusing in a "hit head here" sort of way.  Mind you, the project is four weeks along and I've already created a well normalized database structure to write my code against.  But based on this discussion I'm suspecting that I'll be doing a lot of ripping out of code come tomorrow, when the Grand Unified Database Structure is released.

Clay Dowling
Tuesday, April 13, 2004

So you have been writing code in a vacuum?  Four weeks without a database schema? You had to make one up? Who is managing (or not) this project? Looks like a train wreck...

DJ
Tuesday, April 13, 2004

Thank God I am not the only one.

Sathyaish Chakravarthy
Tuesday, April 13, 2004

By the way, it's been over a year since I was let go from Camel, it's six months past their "it must be finished by" date, and I understand their pilot (web-based, against our recommendations) crashed and burned recently.

What a shame....

Philo

Philo
Tuesday, April 13, 2004

It's their first time with more than one developer on a project. I'm willing to wager that if my database design were used, it would also be the first one with a normalized database. My design won't be used though, because the "senior" developer has a real fondness for the second normal form.

Clay Dowling
Tuesday, April 13, 2004

You can always write an email to the manager and the developer and remind them that you cannot code against a schema which does not exist.  State that you have created a normalized schema so that you can start writing code but that it might all be wasted effort if the new schema is very different from the current one.  Therefore there are two options, use your schema and your code, or use the new schema and start over with the code.

I think you can present a relatively strong case for using your schema.

Of course this presumes that you've presented your schema to the other developer and tried to integrate with or replace his currently developing schema.

Lou
Tuesday, April 13, 2004

Is anyone else trying to develop against a schema that changes on an almost weekly basis?

Regular Poster Made Anonymous
Tuesday, April 13, 2004

My schema's got RSS feeds. ;)

Sathyaish Chakravarthy
Tuesday, April 13, 2004

My schema has been presented, and there's a chance that he's going to be reasonable about it.  I'm just not holding my breath, and I'm prepared to do a lot of code ripping after he's presented his schema.

Clay Dowling
Tuesday, April 13, 2004

"Is anyone else trying to develop against a schema that changes on an almost weekly basis?"

Sure. I've been doing that so long that I just assumed that it's the standard way of doing things. In fact, I'm convinced that the changes are actually submitted daily, but I only look at new stuff after I finish old stuff :)

You write a bit of generic "this will update a record" code based on a variety of parameters (connection string, table name, etc.) and keep that on hand. As you start getting schema info, you start writing some helper routines (connection strings, etc.) that simplify hitting your generic stuff. As things progress, you write more helper stuff to help you with the other helper stuff. Before long, you've got this nice plate of spaghetti, but the schema has probably settled down. At that point I try to do a bit of judicious refactoring and carry on. Mostly the spaghetti hangs around to haunt the maintenance programmer (usually me), but nobody ever allocates time to clean up something that works.

If you're feeling really brave, you can fool around with dynamic screens that depend on code that 'discovers' the schema, but every time I've tried that it was a big disaster :)

Ron Porter
Tuesday, April 13, 2004

Layer your system. Your domain objects shouldn't
be dependent on the schema so you should be able
to save that for last.

son of parnas
Tuesday, April 13, 2004

Not sure if this will be helpful, but it seems similar. I inherited an app which handled multiple users where each user had permissions to see and modify different attributes of the schema. Also, the schema changes frequently. Who can view/modify what changes frequently. The original app was pages of display and programming intertwined. It had grown and grown and no one liked to get inside. I rewrote parts of it. I created data structures. I have an objects called viewBuilder,formGenerator, and similar objects. When I get a change in the schema (adding a new field, taking one out, changing a field type, etc.) most of the time I just modify my data structure. The structure provides size, maxlength, type, id, etc. and whether required or not, display in relation to and things like that.  Changing what users can view and modify also only requires changes to a data structure, I don't have to write any code.  I don't see a lot of people using this approach.

Me
Tuesday, April 13, 2004

son of parnas got it right.  It is not unusual at all for the schema to be in flux while the coding is well underway.

Bill Rushmore
Tuesday, April 13, 2004

If any of our code would compile, I'm sure we'd find that it wouldn't match the database schema. :)

Is that what's called an impedance mismatch????

<cry>

Brian
Tuesday, April 13, 2004

Philo,

I trust you have the courtesy to write them and say!

David Roper
Tuesday, April 13, 2004

Son of Parnas' idea works fine in theory, but it has some real problems when it meets up with reality. When your database schema and your objects don't match well, what is required is a Super Human Integration Troube (SHIT).  Although I've produced more than my fair share of SHIT projects, I like to avoid it.

Domain objects have to get their data from somewhere.  The idea of writing custom data stores while I wait for a schema to be produced leads to more SHIT.  After going to all of the trouble to write my own custom data store, I'd have to think that there wouldn't be any point in using the database.

Certainly, some schema flux is expected and even desired. As applications grow the schema has to change, and during initial development its pretty much guaranteed as the real world needs become clear.  The only thing I have a beef against is the "you don't need to know the schema" mentality.

Clay Dowling
Tuesday, April 13, 2004

If you have a rough idea at least about what it's going to look like, you can make assumptions and err ... use views when the assumptions are wrong..

and promise yourself you'll come back and fix it.. yeah, yeah. That's what you should do.

Not that I've ever done that to anyone, of course.. just sayin'

deja vu
Tuesday, April 13, 2004

>Son of Parnas' idea works fine in theory, but it has some > real problems when it meets up with reality.

I am not a real theory guy, so while there are always
problems, in reality it works out. The alternative
of churning for every field change is far worse.
Schemas always change. Making your layers above
the domain objects aware of that is nuts.

son of parnas
Tuesday, April 13, 2004

This must be a major conceptual error on my part, but how do you code your domain objects without knowing where the data is coming from and where you are going to put it?

I've had people talk about xml and layers, etc... Seriously, how do you code for an unknown?  Just store the xml directly in the database?  Bad example, but that was suggested to me once, I won't go into all the ugliness that represents...

I generally code my domain objects to reflect the data structure (normalization, etc..) to ensure I know what goes where.  What do you do?

Ray Beckett
Tuesday, April 13, 2004

>I generally code my domain objects to reflect the
>data structure (normalization, etc..) to ensure
> I know what goes where.  What do you do?

You are creating DTOs (data transfer objects),
not domain objects. DTOs are just beans for
getting data in and out of the database.

Domain objects are about behaviours. Write your
application as if the database did not exist. Concern
yourself with the what is needed to implement your
requirements. These same objects should be usable
from unit tests, swing, html, cli, etc. If they are not then
they are not domain objects.

For testing you can just serialize objects into the file
system, or blob them into a database, or use a mapper
like hibernate to store your objects directly into the
database.

Then you can map your domain objects to a schema
when it has settled down. What this means depends
on your application and organization.

This may be hard, but at least every schema twitch
doesn't cause a verical earthquake.

None of this is easy or effortless.

son of parnas
Tuesday, April 13, 2004

"You are creating DTOs (data transfer objects),
not domain objects. DTOs are just beans for
getting data in and out of the database."

Hmm...  Sounds like you are developing the data manipulation/processing objects in isolation from the data source/storage.  How do you represent the data you are processing?  Are you using objects to encapsulate the data you anticipate working with, then map those objects/properties to the DTO's? 

Can you point me to any books that implement a similar methodology?  I'm all for minimizing how much code get's shaken up by a change.

Ray Beckett
Tuesday, April 13, 2004

> How do you represent the data you are processing? 

I am not processing any data. I have domain objects
that have attributes that support the behaviours
needed to carry out their requirements/use cases/user stories.

>Are you using objects to encapsulate the data you >anticipate working with

If you use the DTO approach you will need to get
down to that level. But the relationship between
DTOs and domain objects is arbitrary. I hate DTOs,
but they may be necessary if the schema is shared
so it doesn't map to your app.

>Can you point me to any books that implement a
> similar methodology

Nothing specific. Some general notes at
http://c2.com/cgi/wiki?search=layer

son of parnas
Tuesday, April 13, 2004

The general pattern for DTOs is described at http://java.sun.com/blueprints/corej2eepatterns/Patterns/DataAccessObject.html

This article is more concerned with changing the data source than the schema, but the general idea works -- there's some other class responsible for populating your business objects.  The business objects themselves just do what they're supposed to and you leave off coding the Populator class until things are "stable"  (whatever that might mean at your workplace).

Boofus McGoofus
Tuesday, April 13, 2004

Hi Clay,

Can you tell us more about this particular development project?

Question 1
Are you contracting with a very small business? Subcontracting with a small consulting firm? It sounds like the project team consists of just you and a senior developer.

Question 2
Were you told what the end product was and then instructed to start coding immediately? I am asking this question in order to better understand what prompted you to create a normalized database structure to write your code against. I am guessing that the senior developer was supposed to create a schema several weeks ago and somehow got sidetracked.

Inquiring Minds Want to Know
Wednesday, April 14, 2004

Don't worry you can just refactor later; it's the XP way - all the cool kids are doing it!

Mr Jack
Wednesday, April 14, 2004

Once again, I must be missing something...  Isn't a use case/story/behavior at it's most basic level data processing?  You have inputs, internal data, outputs.  If there is no input, the output is really just a constant.  Internal data may not be necessary, but usually exists.  If you have no output, there is no result, and hence no point to the process.

I'm guessing our respective environments/projects differ greatly in scope and need.  I'm primarily a RAD database guy.  I've used Access, VB6/SQL Server, now C#/SQL Server.  All my stuff has been database oriented, so that's the mindset I use for developing my apps.  Note:  I'm also the guy that usually designs the database.  The largest project I've been on had a total of 6 developers.

The link you pointed me to looks to be J2EE based, or very large projects, lots of people, lots of time.  What is your environment?

Ray Beckett
Wednesday, April 14, 2004

This firm is tiny. The project consists of the senior developer, who comes from a Java background, myself (who requires java in the morning to get started) and another part time developer who is just getting the basics of programming.

As for the schema that was supposed to be written by the senior developer, he vamoosed to Europe for three weeks just after I came on board.  I was given a very high level document about what the application was supposed to do and a UML diagram.  The grand unified schema was I was supposed to receive today is nothing more than the UML diagram.  So I updated my schema to match the UML diagram more closely (after we hashed out some of the things that needed to happen in the application) and submitted that.  I have high hopes that it will be accepted.

Son of Parnas, your ideas are definitely unique.  I don't see myself spending a lot of time serializing my data objects and managing a filesystem data store when I'm supposed to be writing against a database store.  This project doesn't have that kind of budget, and I don't have that kind of patience.  Maybe programming in Java gives you that luxury, but this application won't be in Java so long as I'm on the project.

Clay Dowling
Wednesday, April 14, 2004

>Son of Parnas, your ideas are definitely unique. 

I don't think they are. They are fairly common in OO.

>I don't see myself spending a lot of time serializing
>my data objects and managing a filesystem data store

Neither do i. Thankfully it is quite automatic.

> This project doesn't have that kind of budget,

For that evil java these tools are plentiful and free.

>and I don't have that kind of patience.

Getting old and set in your ways?

> Maybe programming in Java gives you that luxury,
> but this application won't be in Java so long as
> I'm on the project.

Getting old and set in your ways?

son of parnas
Thursday, April 15, 2004

"Getting old and set in your ways? "

What Getting?  Already there.

Clay Dowling
Thursday, April 15, 2004

> What Getting?  Already there.

Argue for your limitations and they are yours.

son of parnas
Thursday, April 15, 2004

*  Recent Topics

*  Fog Creek Home