Fog Creek Software
Discussion Board




Pattern(s) for Calculating Variable Costs

Hello,

I'm doing some preliminary work for a system that will calculate costs based on a number of formula that contain fixed and variable elements.

The tricky part is that these formula are not fixed, i.e. the system I'm working on will have to work out which formula to apply, when to apply it and how to combine it with other formula. Furthermore the formula are configurable by end users.

I'm wondering if any JOSers have worked on such a system and have any suggestion about appropriate design patterns. Any comments appreciated.

Walter Rumsby
Tuesday, April 08, 2003

I have done a similiar thing (scoring configurable by an ini file) in a hobby project.  I ended up chucking in a full expression evaluator.  You can likely pick up code for your choosen language of the web.

Nice
Tuesday, April 08, 2003

'Command' pattern for the different functions, and 'State' pattern for the switching.

big bob
Tuesday, April 08, 2003

Interpreter pattern & friends

Dino
Tuesday, April 08, 2003

Walter,

Since I've nearly finished that excellent book you recommended, I'd say this is a definate candidate for the Abstract Factory pattern, and possibly the bridge pattern.

We have a similar system (point of sale) here that is an excellent demonstration of how not to do it (The hundreds of case statements pattern).

Ged Byrne
Tuesday, April 08, 2003

Maybe a simple recursive decent expression evaluator would suffice?

JJ
Tuesday, April 08, 2003

http://www.singularsys.com/jep/

Philippe Back
Tuesday, April 08, 2003

Take a look at using a rule-based approach. 

If you are programming in Java, JESS is a good solution for this:

http://herzberg.ca.sandia.gov/jess/

jess user
Tuesday, April 08, 2003

If the end users can edit the formula, the Interpreter pattern (or just embedding an interpreter) is a good way to manage the actual calculations.

Deciding which formula to use is often done via a Factory - you pass the factory the criteria, it returns the appropriate formula object.

Chris Tavares
Tuesday, April 08, 2003

JScript, JScript .NET and VBScript all have methods which take in strings containing expressions and evaluate those expressions.  You might consider using the Script Control or VSA to cheaply host the engines.

These languages also have the important feature that they are sandboxable. I would strongly recommend a thorough design review from a security perspective of any system which allows users to customize formulas, _particularly_ if they can call third-party libraries via those formulas.  Evaluating "10 + DeleteMyHardDisk()" can ruin your whole day.

Eric

Eric Lippert
Tuesday, April 08, 2003

I have written several rules based systems. For various reasons they have been implemented slightly differently each time, but I'm getting better at it :)

You will need to build a parser.

You can either write something from scratch, which will involve understanding parsing and token substitution, or use an existing parser (as Eric suggests). I have used VB script to handle the parsing and am currently experimenting with XSLT.

Either way you'll need to encapsulate it in something that provides process flow, results processing and storage access etc.

You need to create a UI for people to build formulae. They also need to be able to use predefined functions (eg. Avg(), Sum() etc) and create user defined functions, possibly also including formulae (have a look at the expression editor in MS access). They also need to include variables (tokens). Use your parser to validate before storing the formulae. Allow users to provide sample values to validate the formulae.

The UI needs to allow the formulae to be presented as discrete units, which users can move round and reuse and provide process flow. I have done this using Visio and it works quite well. I have also used systems where the UI was done from scratch. They also work quite well, but it’s a lot of work.

You also need to consider the run time processing requirements. Does the system need to process hundreds of calculations a second (and store the results)?

You will need to be able to provide a generic way for the parser to access data. The users also need to be able to create modify these “data adaptors” – although possibly not in version 1.

The system will stand or fall on these 2 main things:
The DB access, particularly runtime variable population & storage of results;
The user interface.

There are lots of smaller issues to be considered, not least of which, as Eric points out are the security considerations.

You should allow several months development time. Building something that parses is easy enough - a few days work. Making it easy to use is tougher.

Justin
Tuesday, April 08, 2003

I'd second (or is it third?) the parser idea. I built one for a stockbroker to allow the market analysts to enter formulae for calculated items, and it worked pretty well. As someone said, though, getting a decent front end that end users are happy with is the hard part - I never managed it, and ended up putting the formulae in myself.

If you don't want to build a parser from scratch you could look at tools such as yacc - there's an example of quite a sophisticated calculator, hoc, built that way in Kernighan & Pike's "The Unix Programming Environment".

Andrew Simmons
Tuesday, April 08, 2003

If you're looking into parser generators like yacc/flex, try lemon also. In my limited reference it seems a bit poorly documented, but it's very capable and is supposed to have advantages over yacc (fewer global variables, among many others, iirc).

Mike Swieton
Tuesday, April 08, 2003

Don't go overboard. Remember the 80/20 rule. Make sure that you focus on having them do the 80% in a very comfortable and straightforward way, and only after that concentrate on making the other 20% possible. Unfortunately it is in the nature of most developers to want to concentrate on the "power" of the 20%. Resist!
E.g.: It would be extremely easy to just chuck in a textbox and a full Lisp interpreter or a spreadsheet control, and extremely powerfull, covering every possible exotic scenario, but would it help your users?

Just me (Sir to you)
Wednesday, April 09, 2003

You can write it in Lisp or implement a subset of lisp in  your language of choice  :)

Ros
Wednesday, April 09, 2003

Just me: You've got a point about the 80/20 rule.  But I think for something like this, while you may want to consider that for the UI, you generally want to have a powerful backend that can handle whatever you will want to do with it.  For example, he could put in a complete expression evaluator backend, but then taking into account the 80/20 rule make a simple UI for the most common tasks.  But since the backend is there, it is possible to do the harder tasks.

If you develop your backend with the simple case in mind, you may have to redesign the entire program to handle the harder case.  If you design it with the harder cases in mind, you often only have to put a simplification layer in place to ease users into the power you have available

Mike McNertney
Wednesday, April 09, 2003

Ros,

I thought all projects ended up developing a subset of LISP.

Ged Byrne
Wednesday, April 09, 2003

Zawinksi's Law: every program expands until it is capable of reading email.  (but there are bonus points for programs that, in the process, are also capable of interpreting a dialect of LISP).

http://www.catb.org/~esr/jargon/html/entry/Zawinski's-Law.html

Alyosha`
Wednesday, April 09, 2003

... Jamie Zawinski would know ... he worked on XEmacs, which does both ...

Alyosha`
Wednesday, April 09, 2003

So email apps follow an Inverse Zawinski curve ?

Justin
Thursday, April 10, 2003

Justin wrote "You will need to build a parser."

Does it make sense to reinvent the wheel?

You may take a look at USPExpress Parser,
http://www.unisoftplus.com/uspexpress/ .
There are UI samples available that may give you an idea of possible end-user UI solutions.

Besides, it supports user-defined functions, variable aliasing and is capable of processing hundreds of calculations per second.

Vladislav Safronov
Sunday, April 13, 2003

You're absolutely right, Vladislav, and this was exactly the point I made in the next paragraph when I suggested that doing so involved a lot of work and that there are several fine products already available.

The main thrust really was to suggest that one of these be used as an engine, encapsulated in something custom written that handled business processes, IO, UI etc.

Performance is the only good reason I can think of for writing completely from scratch.

I have not heard of USPRExpress, but I'll put the link into my 'things to look at' folder.

Justin
Tuesday, April 15, 2003

*  Recent Topics

*  Fog Creek Home