Fog Creek Software
Discussion Board




Portable Business logic ?

Anyone attempted to encode business logic in a way which can be transferred between different layers of validation ? For example, we might want to validate our business layer somewhere in one of our middle tiers. We might want to implement a subset of that logic in the UI for some up-front sanity checks. We might want to implement a subset of it in the database for speed.

We don't want to code it more than once, and we can't necessarily share the code between all of these layers. One solution would be to have validation engines in each layer and pass the rules between them in the form of data.

Has anyone done anything like this ?

WoodenTongue
Monday, June 21, 2004

Could you elaborate?

Why validate and revalidate at each of several layers using the same rules?

Bob Riemersma
Monday, June 21, 2004

My experience with 'Business Logic' is that this is a euphemism for "All the STUFF we want to do, to implement what our company does, to produce MORE STUFF that we then sell for money."

It is the process of building the requirements for this, designing this, and coding this into a processing layer of an application that clarifies what is meant by 'Business Logic' in the first place.

Your question seems to assume that you can have this set of 'Business Logic' rules up-front, independent of an implementation platform.  Thus you should be able to implement it in such a way that it is 'portable' between these platforms.

In my experience, this assumption is never true.  If you try to 'force' it to be true, the result will be to implement changing 'Business Rules' multiple times -- at least once for each 'platform' you try to move it to.  Then you'll need to update the 'older' platforms with the new insights.

You have been warned.

AllanL5
Monday, June 21, 2004

Oh yes, and building a 'validation engine' for rules you don't have yet can also take much more effort than coding the rules in the first place. 

AllanL5
Monday, June 21, 2004

You can put settings/code for your business rules in a data dictionary.  Basically, that's just a way to store metadata in a way that can be accessed and used from any tier.  My work is in Delphi, a compiled language, which would ordinarily limit the type of changes you can make just by changing metadata at runtime.  But there are numerous "scripting tools" for Delphi that are lightweight and can be incorporated into a compiled program.  These allow for storing business rules into the data dictionary in the form of Delphi code that can be run by the scripter at runtime.

Don't know if this method would work for you, or whether there are appropriate scripting tools available for you if you're using a compiler, but this method looks like it's going to work well for me.  (Not fully implemented yet, but I don't see any insurmountable problems.)

For one resource on this data dictionary stuff, you could check out : http://www.geocities.com/SiliconValley/Lab/6888/top.htm

Herbert Sitz
Monday, June 21, 2004

"Why validate and revalidate at each of several layers using the same rules?"

I agree, I don't really see the need for this, either. 

My main reason for going the data dictionary route (see previous post by me) and making validation rules metadata that can be changed at runtime was to enable users to modify the validation rules themselves.  No need to change the rules in code, recompile, and distribute new executables.  I have also investigated incorporating the rules in a .dll that could be distributed separately to update programs with new sets of rules, but the data dictionary method seems preferable to me now.

Herbert Sitz
Monday, June 21, 2004

For Windows development, you could use the Scripting Object. It can execute arbitary VBScript code at runtime.

KayJay
Monday, June 21, 2004

Business logic (aka constraints) belong in the DBMS.

Captain McFly
Monday, June 21, 2004


Put the validation rules in as few places as possible.

Besides, if it validates at one level, why should you have to re-validate it *unless* you've done some sort of processing or operation on it.

Then you're not talking about the same thing, you're talking about the same thing, you're talking about validating the process.

KC
Monday, June 21, 2004

Often in a web-app you want to do validation client-side -- to avoid round-trip latency on errors, and reduce server load.

But the right place to do validation according to the purists (end-to-end argument) is in the database level (or model in the MVC pattern). For other views or controllers (e.g. batch) you may not have any need to do "client-side" validation.

The purists are right in a sense--if you are going to do validation, the only place you must have it is in the database. But often in a practical implementation you need it elsewhere too. So you end up with duplication and (over time) inconsistency. But it would be a mistake to plan for redundant validation before you know you need it for e.g. performance.

It would be nice to have a central set of meta-data rules. But you don't know a priori what the rules look like, so lazy evaluation of the rules is hard to do. A good compromise (which I have seen in practice) is to generate the platform specific validation code+data from a single generic specification. Which is I guess where you started.

This approach is relatively easy for database style constraints, but gets progressively harder when you start trying to "compile" stored procedures into e.g. JavaScript.

MugsGame
Monday, June 21, 2004

"Business logic (aka constraints) belong in the DBMS. "

And what to do when your DBMS doesn't support the types of constraints you need to impose? 

Even if a person agrees, in theory, that it's best to keep all business logic in the DBMS, in practice I'm guessing that's rarely going to be the chosen route.  At least not with currently-available and widely used DBMS's. 

Herbert Sitz
Monday, June 21, 2004

"And what to do when your DBMS doesn't support the types of constraints you need to impose? "

Get a better DBMS and/or complain loudly and clearly to your DBMS vendor.

I would be interested to hear of a business rule that cannot be implemented in predicate logic and set theory (provided the DBMS supports such mathematical constructs).

Captain McFly
Monday, June 21, 2004

Captain McFly -- Well, for example, constraints in SQL Server can't compare the entered data against values in tables or rows other than the row being checked.  That seems like a pretty big limitation.

You can work around that by implementing triggers to do constraint checking without those limitations.  But then you're getting even deeper into writing database specific code that's going to tie your application into the particular database you're using.  And it's going to put a heavier load on the database.

Portability is perhaps not an issue for some.  But if you want the ability to change back ends in the future,  you're going to want to keep most contraints/validation out of the database.  Keeping the constraints/validation checking outside the database but still centralized in some easy to access part of your application seems to me like it may be an acceptable tradeoff for portability.  If done correctly, it can also increase scalabity, e.g., you can scale up by running middleware servers (doing constraint checking and lots of other stuff) on multiple machines, all accessing the same db.  Scaling a bottleneck-db up, in contrast, basically requires improving the machine the db server runs on, don't have the option of spreading the db load across multiple machines.

I do see the appeal of having all the constraint checking centralized in the db.  I'm just not sold that it's always the ideal solution.

Herbert Sitz
Monday, June 21, 2004

“That seems like a pretty big limitation.”

It is. One that Microsoft should fix – but they won’t if you don’t ask for it. :)

“But then you're getting even deeper into writing database specific code that's going to tie your application into the particular database you're using.  And it's going to put a heavier load on the database.”

I would include a quote about premature optimization here but in “most” cases there is not a huge difference between putting the constraints in the DBMS (not database) and having someone else enforce them. Someone has to check the data, and generally the DBMS can optimize data paths far better than 3rd party clients which really execute SQL statements under-the-hood -- unless you’re distributing the data out to separate rules boxes.

Once you start distributing the data you basically are re-inventing a distributed DBMS – and why go through the overhead and complexity of that when you can just implement your vendor’s distributed DBMS scheme?

“If done correctly”

I suppose that is the rub. All too often it is *not* done correctly (which is why I get paid ;)) and the headaches are severe. When you implement rules away from the DBMS you run the risk of your data becoming inconsistent/incorrect because someone can circumvent your rules engine and access the data via the DBMS’s query language and the likelihood of a bug in your middleware is significantly higher than the DBMS product (provided the DBMS is quality, of course).

“don't have the option of spreading the db load across multiple machines.”

I’m not sure where that came from but there are obvious things like replication, clustering, high-availability, etc.

Captain McFly
Monday, June 21, 2004

> users to modify the validation rules themselves. 

Ahhhh! Run don't walk from this project. Can you imagine the mess you will get into if end users are modifiying meta data to set the validation rules.

Matthew Lock
Monday, June 21, 2004

Matthew -- I'm not too worried.  Users will mostly just setting a "Required" flag to true if the field is required (and even that only for certain fields), setting default values (I know, that's not even validation stuff), maybe a couple other little things.  Plus, since the metadata is stored in a data dictionary it's simple enough to revert to the metadata defaults.

But Captain McFly does have me wishing I were working on a large SQL Server project where there more free-flowing money, big-iron, and enterprise-class db stuff than you could shake a stick at.

Herbert Sitz
Monday, June 21, 2004

Isn't that what objects are for? Build the object and pass it around to whichever layer it needs to be in at the moment.

  --Josh

JWA
Monday, June 21, 2004

> Users will mostly just setting a "Required" flag to true if the
> field is required (and even that only for certain fields),
> setting default values (I know, that's not even validation
> stuff), maybe a couple other little things.  Plus, since the
> metadata is stored in a data dictionary it's simple enough to
> revert to the metadata defaults.

Just finished one like that this week. It starts off simple enough then the client says, can I make the default value "x", except if "y" = foo, but if "y" = foo and "z" = bar then it's not required... (except if it's not a working day)..

You get the picture. Pretty soon you will have something complicated enough to warrant a programming language.

Matthew Lock
Monday, June 21, 2004

Matthew -- Well, I'm making shrinkwrap so the client doesn't necessarily get what they want.  ;)  I'm definitely going to keep it simple.

My decision to include that particular functionality (i.e., to let user/admin decide whether some fields will be required or not) is basically driven by trying to match some of the functionality of the better competitors in the product space. 

Herbert Sitz
Monday, June 21, 2004

Oh that sounds better then. No pesky clients comming up with all their weird rules.

Matthew Lock
Monday, June 21, 2004

dbms should only contain rules that are valid for data but not for the actions. eg how do you enforce that orders cannot be fulfilled on Monday? it has nothing to do with the data

   
Tuesday, June 22, 2004

Bob Riemersma: "Why validate and revalidate at each of several layers using the same rules?"

We don't necessarily want to do that. There are cases where we might want to shift the validation to a different layer. We need to balance usability, flexibility and performance. Moving along the usability axis might mean putting the validation in the UI. Performance might mean putting the validation in the database, sometimes. Flexibility may mean putting it in one of the middle tiers.

On the other hand, there may be cases where we do want to validate the same thing twice. Perhaps we want to check some simple things in the UI and avoid going back to the middle tier, thereby avoiding use of the network. We might still want to perform that validation in the middle tier. That might be the case if we were providing an API didn't completely trust the calling UI.

AllanL5:  "the result will be to implement changing 'Business Rules' multiple times -- at least once for each 'platform' you try to move it to."

The whole point is to avoid that as much as possible.

Herbert Sitz: Interesting. I'll check out the link. Thanks. I also agree that customers should be able to modify the rules in some cases.

CaptainMcFly: Putting all of the business rules in the database has two major issues. 1) I can't afford to keep going back to the database. 2) Business rules tend to require complex logic and implementing those in the database tends to be inefficient.

"Once you start distributing the data you basically are re-inventing a distributed DBMS"

Correct. That is one of the problems when you have a punishing scaleability and performance requirement, you have to start pushing some of the work of the DBMS out into the surrounding functional 'cloud layers'.

I agree that users can bypass your distributed validation. There you need a good API set for them to use, good user training and also contractual rules that dissallow access by the backdoor route. Our approach is precisely that. The users *can* fiddle with the data directly, but if they get into a mess because of it, they invalidate their support SLAs.

Josh: "Isn't that what objects are for? Build the object and pass it around to whichever layer it needs to be in at the moment."

Yes but not all of the layers allow objects to be transferred smoothly. If we use a common object design across all platforms and layers and provide decent serialisation then I suppose that gives us an equivalent of our validation engine.












WoodenTongue
Tuesday, June 22, 2004

WoodenTongue,

Take a look at the NakedObjects framework.  You can get it working in both Java and .Net.

http://www.nakedobjects.org

If the framework doesn't fit your needs you might find the techniques used useful.

An application is defined in therms of behaviourly complete objects, from which the user interface and persistance layers are automatically created.

The business logic is almost completely contained within two types of method.  'Actions', which define what can be done with an object and 'Abouts', which define the constraints upon classes, objects and actions.

There is a major release due very soon, with significant enhancements.

Ged Byrne
Tuesday, June 22, 2004

*  Recent Topics

*  Fog Creek Home