The Old Joel on Software Forum: Part 4 (of 5) - bug reports

bug reports > actual source code (doing part)

I've been in a job about 7 months now.

The company does consultancy work (mainly NOT in software development) for other companies. They bill their clients monthly using using an in-house timesheet system constructed out of Microsoft Excel, Access, Lotus Notes, Visual Basic and LotusScript. They use both lotus Notes and Access to store timesheet records as they found they couldn't query data in Lotus Notes quick enough. So every time they need a query they shuffle it out of Lotus Notes either into Microsoft Access or a plain text file.

The timesheet system shuffles data from one part of the system to another using databaes ---> text file conversions, Lotus Notes ---> Access database conversions and some COM.

As data is shuffling around all over the place, conflicts between data entered in one part of the system and another arise.

I've tried to examine the source code. All I see is code shuffling data around and trying to handle conflicts. I haven't found the real part of the timesheet system yet!! It's not quite my job yet to fix this system and I'm working on two other software projects.

Today I looked at a discussion thread (stored in Lotus Notes) about their internally written timesheet system. It consists of tens (probably in the hundreds) of posts about bugs, problems, crashes and conflicts along with many possibly resolutions to all the data conflicts. Obviously these resolutions never worked as the more problem reports always follow.

I thought today that the amount of time people spent in this company submitting bug reports surely must far exceed the amount of time it would take to rewrite this whole system from scratch.

I don't think the people at work are idiots, but I'm staggered that nobody seems to have realised that if they kept the data IN ONE DATABASE and didn't shuffle it around all over the place they wouldn't have had to write some much bug riddled conflict handling code that they never got right.

I just had to share this. I can't believe nobody scrapped this pile of junk. Maybe lots of people thought it but nobody said it.

Worse despite suffering for months with bug riddled conflict handling resulting from dynamically shuffling data between Lotus Notes and Microsoft Access, they still haven't got the message and are giving a variant of the system to a client. The sort of system is like a virus and I fear it is spreading to upset another company!

Savage
Tuesday, April 13, 2004

OK....I did not quite get your point...so you want to rewrite the whole system....fine...do you have a strategy for data migration from the old system? Do you have any handle on how much cost and resources it will take? Whether such resources are currently available? If not it is just a expression of the NIH syndrome. Show me one good engineer who does not feel like throwing an old system away and start afresh?

>I don't think the people at work are idiots, but I'm staggered that nobody seems to have realised that if they kept the data IN ONE DATABASE

Well think about it...may be it is not possible to have one database! Perhaps many of the folks are mobile or located at other (client) work locations from which they cannot access the single database. Perhaps they enter time data on their local Excel when it is convenient to do so rather than have to wait for connectivity to the maon database. Perhaps variopus clients have various time sheet reporting requirements (not unuusual in a consulting environment) which need to be reconciled later...there can be many reasons and it behooves you as an engineer to investigate them before you pronounce such judgements

Code Monkey
Tuesday, April 13, 2004

My first reaction isn't "rewrite it and make it work right" - it's "there _has_ to be an off-the-shelf app that will do the same thing (and cheaper)".

My current employeer had a previous CIO who always chose the "make" in a make v/s buy decision. We now have tons of quickly written in-house crap that has never worked right and is a nightmare to maintain.

The current CIO is slowly by surly replacing this mess with a combination of purchased packages and well-designed/written in-house software. The main questions asked are 1) can we find a package that meets our needs? and 2) would writing it in-house provide a competitive advantage?

In your case, I doubt if a billing system provides a competitive advantage and there are many professional off-the-shelf billing systems to chose from. Why re-invent the wheel?

RocketJeff
Tuesday, April 13, 2004

Your database will just become another
source of data. Someone will come along later
and wonder why your data isn't in some other
database.

son of parnas
Tuesday, April 13, 2004

You also have to understand something about large companies.

This is Katie's Rule of Business Analysis #2.

"Employee time is free to burn - it's already been paid for."

It doesn't matter how much time has been wasted dealing with the duff datasystem because that time has been spread thinly across many years and many projects and many employees.

However having someone FIX it, involves putting time in one place and then it's noticeable.

This happens for two main reasons:

Generally the threshold at which this gets noticed is the timesheeting interval - since you often can't timesheet 5 minute lumps of time, you can't record five minutes of "wasted" time. So the clients end up paying for it in their billed hours. This gets modified by the usual rules "All hours must be billable to a client/project code" and "Regular joe employees cannot create client/project codes" This means that you don't know how much your inefficiencies cost, and makes the employees collude in charging them to clients.

The second reason is that employee time is almost infinitely compressible.

If it takes someone two hours to do a one hour task because the systems are broken, you don't think about fixing the systems. In fact **THEY** won't think about fixing the systems. They'll do an hours unpaid overtime instead. So you don't notice the failure. A useful addition here is the timesheet rule "Only 37.5/40/whatever hours per week may be timesheeted" which nicely hides the fact you're doing this.

Effectively: Reason #1 makes the clients pay for the inefficiencies. Reason #2 makes the employees pay for the inefficiencies.. So why would your CTO want to pay to get rid of those inefficiencies?

Finally, I could be cynical and make you wonder: "Why would your current CTO want to have the boardroom arguments to budget time/money to fix something to make his successors' departments more efficient?"

Katie Lucas
Tuesday, April 13, 2004

You're pretty smart Katie Lucas.

Braid_ged
Thursday, April 15, 2004

Sadly not smart enough to not be a software engineer..

Katie Lucas
Thursday, April 15, 2004

Thank you for your comments Code Money.

What has been implemented here for the timesheet system is a mixture of home grown replication code that mediates between storing the data in both Microsoft Access and Lotus Notes. They are using a bit of this and a bit of that.

They have practically implemented something vaguely similar to how Microsoft Access replicates itself in home grown Visual Basic code.

They produced a bug riddled mess that people find new bugs in every day.

It they simply stuck with storing the data in ONE TYPE OF DATABASE and USED BUILT-IN REPLICATION they would have been OK. But they decided to move things about between Lotus Notes and Access and went down the route of home grown replication code.

Savage
Thursday, April 15, 2004