The ‘Holy Grail’ of Software Systems

In my opinion the ‘Holy Grail’ of systems is predictability. 

What its not:
• Predictability does not mean zero defects. 
• Predictability does not require a cumbersome process such as the Capability Maturity.
• Predictability does not require a programming methodology such as ‘Extreme Programming (XP)’. 

What it is:
• Predictability lessens the impact of defects.
• Predictability does require a simple process be well understood by the participants.
• Predictability does require an iterative methodology rather than the more traditional water-fall.


This gives rise to the question: “Sounds great.  How do we make our system ‘predictable’”?

While I do not pretend to have all of the answers, I do have some simple ideas (not all mine) that I have personally seen work.  They are:

1. Religiously track all hours spent on all tasks by all team members. 

a. Tells us where our time is spent and how much time we have (capacity).

b. Tells us ‘where we have been’ (estimates vs. actuals).

c. Provides for planning.

2. Prepare estimate, in hours, for all requirements.

a. This allows for planning (see how it works together?)
b. Estimates must be prepared by the engineers who will do the work (after all, they are the ones who are supposed to know how to do the work).

c. Management must track estimates vs. actuals to determine how good the engineers are at making estimates (even Michael Crighton has an editor).


3. Releases must have fixed dates. 
a. Fixed dates provide a fixed number of hours for the work to be done.
b. A fixed number of hours, coupled with the engineer’s estimates, tell us how much work we can do in the time allotted.
c. This also helps to prevent feature creep.
d. The date should be fixed on a day in some set pattern of occurrence, such as, the first Tuesday of every month, or every other Tuesday.
e. Every cycle includes scheduled ‘buffer’ time.

4. Automated regression testing – “You can never have too much testing”
a. ‘Nuff said.

5. Enforce a two-stage hand-off such that the details of the system must be communicated for the hand-off to be successful.
a. This enforces the spread of knowledge.
b. This also has the effect of decoupling one group’s schedule from the next such that a delay in one area does not always mean a delay for the product.

Scott Rogers
Wednesday, May 19, 2004

Is this a troll?

Mr Fancypants
Wednesday, May 19, 2004

No.  Maybe just the wrong forum.  Merely looking for feedback on the idea of striving for predictability in production software systems (back office, web, etc...) and my ideas for acheiving it. 

Moderator: Please feel free to delete if deemed in-appropriate.

Thanks!

Scott Rogers
Wednesday, May 19, 2004

No, they are all reasonable things - or at least things reasonable things reasonable people could discuss.  Just the tone was wrong.

It was like on American Idol "you just didn't connect with the audience".  Except I don't want American Idol (way to go Fantasia, that was awesome!).

too cowardly to admit I watch Idol
Wednesday, May 19, 2004

True predictability is a pipe dream. 

The idea that certain processes will bring about predictability in software is a lie told to trick managers into thinking that developers are a commodity item and if only you had the right process you could hire a bunch of low-skilled morons for low wages and use the money to give the CEO and other upper management a bigger bonus.

There are good developers and bad developers, and no amount of process is going to make a bad developer good, though it can make a good developer bad by hidering his or her natural workflow.

Mr Fancypants
Wednesday, May 19, 2004

Predictability is the Holy Grail of Program Managers and bean counters.  Fine if they want to spend their time "religiously" counting hours and such.  The people that write and test it have better things to do. 

A Thrusday afternoon 3 line status report should suffice.

hoser
Wednesday, May 19, 2004

I think what you said makes a lot of sense.

Karthik
Wednesday, May 19, 2004

There are bad processes and good processes. When any process hinders a good developer's workflow, it is "cargo cult software engineering" and therefore something that needs improvement.

Derek
Wednesday, May 19, 2004

Good points, all.  While I don't disagree with the opinions regarding TPS reports :) , I also still think there is validity to tracking time.

The idea is not to keep developers 'under my thumb'. 

Rather, the notion is to understand the capability of the team and plan accordingly.

That said, how many of your are engaged in maintaining 'Data Center' production software (websites, CRM, work flow... essentially not 'shrink-wrap') which requires constant change and is highly-integrated with other systems? 

I really am curious as I am unable to find forums, books, etc.. that focus on highly-available production software coupled with un-realistic deadlines and managment that does not understand the realities of software developement (takes time, good developers make all the difference, can't develop without requirements, etc).


Once again, thanks!

Scott Rogers
Wednesday, May 19, 2004

"There are bad processes and good processes."

True, but which is good and which is bad is variable, based on individual developers, the situation, and team dynamics.  Anyone trying to sell you "the end all be all process" that works for everyone and everything is full of shit.

MrFancypants
Wednesday, May 19, 2004

Fanypants, I agree.  Another item missing from my original post is, "This is not a one-size fits all".  Rather, it is a collection of things that I have observed in my experience that help organize work.

Thanks!

Scott Rogers
Wednesday, May 19, 2004

As someone who writes software that includes time management, I'm well versed in all of the arguments against it.

What I will say is this:

- Tracking your own time is the only way you can gage the quality of your estimates. You only /think/ you know how long you spend on a problem.

- Comparing time between yourself and others is the a great way to gage the strengths and weaknesses of each team member. If anything involving regular expressions takes you an average of 2x longer than anyone else then either you should not be tasked with them or should be provided training. Conversely if you take 2x less to complete it then you should be tasked with it early and often.

- You may not like it, you may be offended by it, but your employer has a right to know what you are doing with the time he/she is paying you. If you don't like it then open your own business.

I don't require the developers under me to use time management (very small team so it isn't very informative). But I use it for myself to track my own progress all the time and my estimates are 80-90% dead-on.

Marc LaFleur
Wednesday, May 19, 2004

I'd add a step zero to your list:

0.  Design your software.
a.  You cannot have a requirements list without a design.
b.  Without a requirements list you cannot estimate time requirements.
c.  Without a requirements list you cannot avoid feature creep - every feature is the product of creep.

Aaron F Stanton
Thursday, May 20, 2004

"a.  You cannot have a requirements list without a design."

What?

MrFancypants
Thursday, May 20, 2004

Mr Fancypants: "There are good developers and bad developers, and no amount of process is going to make a bad developer good, though it can make a good developer bad by hidering his or her natural workflow. "

That nails it, I think.

Although, having said that, I still hold by the triumvirate:

"Good, quick, cheap - pick any two".

Steve Jones (UK)
Thursday, May 20, 2004

Estimating, IMHO, is much more of an art than a science. As I think Joel himself has pointed out, you can only reliably estimate something you've already done several times and hence already know how long it takes. The idea, therefore, is to break down any job into tasks that you have done before and to build up the overall estimate from those smaller steps.

However, this assumes that you *can* break down the job into tasks off the bat. Very often, you need to do analysis, research, sometimes prototyping to know *how* you're going to do the job, without which you cannot derive tasks and hence a reasonable estimate.

Now, put yourself in the situation that you cannot start a project until you get budget approval, and to get budget approval you must submit a time and cost estimate. And you must not exceed the estimate or the project will be deemed a failure. You can't spend any time analysing requirements or doing any architectural thinking at all until the budget is approved, because all hours must be billable.

So, you submit your 'best guess' estimate and get a budget, but the project actually goes over budget by 25% and you nearly get fired. From then on, you apply the 'double it and add 50%' rule to all the guesstimates you give out, with the result that your projected costs and times always look unattractive and most of your projects don't get budget approval. You and most of your team get fired because there's no work for you to do.

Welcome to the world of my last job - and from what I hear, my experience is by no means unusual.

Until we sort out the bigger problem - the PHBs - no amount of methodology or 'best practice' will ever fix this industry.

Neil Hewitt
Thursday, May 20, 2004

  Recent Topics

  Fog Creek Home