Fog Creek Software
Discussion Board

Any thoughts on generic data models

After reading the discussion at the below URL, I thought I'd ask the JoS community.  What do you think of generic data models,  i.e. using database tables "objects", "attributes", and "object_attributes" to represent almost anything?  Do you think the flexibility gained offsets the complexity of the queries/scalability issues for certain applications?  Any alternative models that are both flexible and scalable?,F4950_P8_CRITERIA:10678084117056,

will s
Thursday, June 26, 2003

Unless you have a really pressing need for a generic data model, avoid them. To me it boils down to the 80/20 rule.  80% of the time will be spent in maintenance and debugging.  Generic models make application support more difficult because everything is dynamic. You cannot assume anything as the code flies.

Add to this the overall impact to resource time/cpu/memory and you begin to have a difficult time justifying the ease of expansion across the entire domain of support.

Some places it can work very well. Most I have seen overuse it to their detriment.

Mike Gamerland
Thursday, June 26, 2003

Its a well known bad idea.  You are replicating functionality already provided by the RDMS, and in doing so you just add a layer of inefficiency.


Peter McKenzie
Thursday, June 26, 2003

I actually started down that road once. I eventually got to the point where I was building screens to help me manage things. That's when I realized that all I was doing was rewriting Access in Access. So much for that brainstorm, but it was fun while it lasted. I also learned a great deal (keeping in mind that we learn the most from our mistakes <G>).

Ron Porter
Thursday, June 26, 2003

Generic data models, specifically those which convert everything into attribute-value representations, are a bad idea.

Their worst feature is that they obscure the semantics of the database. In the absence of suitable documentation, unless you can inspect the data you have no idea what attributes are stored, nor of their aggregations and relationships.

Further, this type of data structure makes it difficult to work with sets of attributes. For example, a query to "show me the orders where the estimated delivery date is greater than 90 days from the date of order placement" is damned difficult with this type of data structure, but trivial with a correctly normalised design.

Finally, the performance of applications with this type of design is atrocious.

Schemas with sparse data requirements, such as star schemas for data warehouses were not all dimension values apply to all other dimension values, are one of the few suitable candidates for this type of model.  Otherwise, don't bother!

Friday, June 27, 2003

The only time I've used a generic "other attributes" field is when it's used to mean "miscellaneous data we didn't know we needed but last minute feature creep meant we had to shoehorn it in here for this release because it was the least risk to retrofit it". Oh, also we're using a ten year old bespoke database engine where adding fields on the fly as part of an upgrade is a major programming effort.

A proper database design planned upfront with sensibly designed tables and relationships is the way to go. Don't make things too abstract "just in case", you'll probably end confusing yourself.

Better Than Being Unemployed...
Friday, June 27, 2003

Any thoughts?  IMHO, they suck

If you have more than a few records, a generic model as described in the original question will probably be unusably slow. It also means you can't write simple queries or use standard tools such as report generators, or set up constraints and so on the server (which is one thing normal databases excel at)

Lots of vertical applications however require user (or user's admin) configurable fields, relationships, tables.  For this I would suggest using the database as intended (avoiding the problems mentioned previously), and then some kind of meta data which your app queries to understand the fields/tables/relationships. 

Most modern databases do have some meta data built-in, but chances are this will be insufficient to the task, as it will simply give you table, field names/types/etc.  So therefore you need additional (or alternative) meta data describing relationships, application-specific information (e.g. name of field on the screen or what your app uses a particular field for, etc.).

S. Tanna
Sunday, June 29, 2003

*  Recent Topics

*  Fog Creek Home