Fog Creek Software
Discussion Board




Bothersome holdovers from back in the day

I'm wondering what are some of the more bothersome aspects of your job or of computing in general that are there primarily because they've always been there?

One of my favorites is stack based programming.  By this, I mean that most programming systems in common use assume that you call a function, it does something, and then it returns to the caller.  This means that the structure of the system is captured statically in the sequence of functions that get called.

This seems like a great way to describe certain kinds of computing, but doesn't seem to be the best way to structure most coarse grained systems at all.  To me, it seems like a system that supported asynchronous message passing as it's primitive means for communicating among objects, possibly using stack based communication by default inside the object, would be a much more robust solution.  I've seen several systems over the years that more or less built on a model like this, web services being the most recent attempt. 

It seems to me that the main reason we don't have more choices like this (or rete-programming or whatever) is that stacks started out as the method of choice and now they are what's familiar, so they shape how you decompose a problem.

Thoughts?

JSR $FFD2
Tuesday, February 25, 2003

Sequential programming is the simplest solution and is optimal when you're computing along in a linear fashion and you need the result of one step of the calculation in order to perform the next one -- which is what people want to do 99 times out of ten.

There are a few domains where this concept doesn't apply, however -- I/O is one, distributed computing another ... here are situations in which asynchronous computing can reap huge performance gains by exploiting the parallelism of certain operations. 

On the flip side of the coin, though, asynchronous programming is a mess.  Complexity doubles because every asynch method call requires a callback, and every guarantee you had about the order in which operations are completed, or even that operations are completed at all, goes out the window.  And you still have to deal with error conditions ...

Now, if I had to worry about all these caveats while working on a straightforward sequential-execution program, I think I'd go insane.

Right tool for the right job ...

SYS 828

Alyosha`
Tuesday, February 25, 2003

I'm not sure we're thinking of asynchronous programming in the same way.  What I'm talking about is sending a request off and just going on my merry way, not waiting for a call back.

Components like this would wait for one or more inputs to appear, pull it off, transform it, and then shove it onto the outputs, probably with queues at both ends (or some kind of standard synchronization signal) to buffer things.

Electrical engineers use this sort of thing to process complex signals nicely.

JSR $FFD2
Tuesday, February 25, 2003

Here is an event based programming method. I have never used it, but it is very interesting.
http://www.quantum-leaps.com/index.htm
Some people are out there trying new things. The stack comes from the hardware up. In the modern OOP world, when do you really worry about the stack?

Doug Withau
Tuesday, February 25, 2003

>>  Components like this would wait for one or more inputs to appear, pull it off, transform it, and then shove it onto the outputs, probably with queues at both ends (or some kind of standard synchronization signal) to buffer things.

I'm pretty sure that Objective-C defines method calls in this matter. So there's at least one language out there that doesn't depend on the stack for everything. Also, aren't some languages designed around the concept of coroutines - tasks with simple, fixed rendevous points and no time slicing?

But... where I have heartburn with proposing asynchronous calling as a standard programming technique is the fact that synchronization of concurrent threads and queues provide a multiplicity of additional ways for software to malfunction due to marginal design. Timeouts, deadlocks, and overrunning message queues would have to be "designed around" and anticipated all the time. And frankly, I find that the programming skill needed to accommodate asynchronous, message based programming just doesnt' grow on trees. Most programmers have enough trouble with *one* thing at a time.

I think there's a very good reason why single threaded, synchronous, stack oriented programming is the predominant flavor of programming in the world - it's general purpose enough to solve most problems, the concurrency support can be added as needed, all programmers understand it, and it's not finicky.

Bored Bystander
Tuesday, February 25, 2003

In terms of general bothersome holdovers from the elder days of IT, my favorite would be:

All software projects get behind schedule and features must be crammed in by weary developers working late into the night as the deadline looms.

I wish the IT industry would quit accepting this.  It might have been true when large-scale software development was new, but we've had decades to recognize the problems and fix them.

Brent P. Newhall
Tuesday, February 25, 2003

> I'm pretty sure that Objective-C defines method calls in this matter.

Sorry, this just isn't true. Objective-C method calls are not really all that different from C function calls; there's dynamic lookup of the function that eventually gets called, but parameters are still passed using the same old calling convention that C uses.  Sorry to burst your bubble.

xil
Wednesday, February 26, 2003

If you're into Java, here's a source forge project building concepts like the ones mentioned in Java:

http://sourceforge.net/projects/flow-based-pgmg

One of my least favorite holdovers is the function call notation used in most of the popular languages.  I'd prefer a much clearer (to me anyhow) notation like the one in Smalltalk.  Reads more like natural language and it's easier to see what each parameter is for.

Poke 53280,0
Wednesday, February 26, 2003

It does sound like you'd rather do Smalltalk than procedural systems.

You could write a message passing object system in any procedural language if you wanted, a good string processing language, like Perl would make it a bit easier, C++ has too much junk overloading it.

I think Smalltalk had its 25th birthday recently.

Simon Lucy
Wednesday, February 26, 2003

HTML forms!

I've heard the arguments for thin clients, but this is riduculous. Okay, you can use scripting to validate the fields, but scripting might be switched off. You could do the form in an applet, but the Java or .Net runtimes might not be installed.

Nope, all we have is maximum length. I'm sure that computing resources weren't so tight back in the day that browsers (I'm talking ncsa mosaic vintage here) couldn't have implemented a regular expression engine a nice simple 'mask=' attribute on the input field.

Neil E
Wednesday, February 26, 2003

I really don't know why anyone would have a problem with the stack frame.  You could always opt to write code without it (globals only).  Now that would be disgusting.

But you wanna talk about stupid holdovers:

1. PASCAL stack frames in a C language API.
2. The keywords far and near and their derivatives.
3. typedef DWORD, UNIT, ULONG, UINT32, LPARAM, WPARAM - all meaning the same thing, signifying nothing.
4. typedef's which obfuscate pointers: LPVOID, LPSTR, ...
5. typedef struct <anything>, What do people have against struct XXX <instance>?
6. IO Ports.
7. The EGA/VGA register set.
8. Using the BIOS for anything other than a bootloader.
9. Using #define IN, OUT when you really mean const/non-const.

Just a few off the top o' my head.

Nat Ersoz
Wednesday, February 26, 2003

In total agreement about HTML/CGI being the standard for thin client access on the web. 

I liked HTML back when it was simple, and rendered the same on everyone's browser.  The first perversion was when the browsers started diverging in the extensions to HTML supported, and "best viewed with IE at 800x600 resolution" became a common sight.

And then over the years a virtual tower of Babel was been built on top of HTML -- CGI made it a two-way information exchange, CSS tried to add fully-featured publishing layout, JScript turned web browsers interpreters of a second-rate language.  And all the plugins that are considered "necessary" ... Flash, Java, Acrobat Reader ...

These days I've seen all the whizzy HTML/scripting stuff that ASP.Net does to make HTML elements on the client appear as if they were on the server programmatically, and I become more and more convinced that web development is no more than a huge kludge on top of a broken metaphor. 

I have a sneaky suspicion that web development will soon collapse under its own weight, and that five years from now people will be surfing the web with Citrix clients.

Alyosha`
Wednesday, February 26, 2003

Well said, and the sooner, the better

JSR $FFD2
Wednesday, February 26, 2003

One of my favorites is embedding metadata in names, especially filenames and program identifiers.  Changing the name of something shouldn't change its identity or alter its behavior or other unrelated properties.  File extensions are archaic.

Down with file extensions
Wednesday, February 26, 2003

Files and directories.

Fixed field structures in databases.

Thats the way the money goes
Wednesday, February 26, 2003

FWIW, my bother is how everyone pads their schedules at each step along the way rather than at the end. Read Joel's column on this here (http://www.joelonsoftware.com/news/20021119.html).

When I worked at AT&T (pre-Lucent), we had a huge engineering force that continually delievered projects late to our customers. They wanted a turnaround of less than 2 months and we could never beat 6. Then one day a director implemented a new scheduling system. The process didn't change all that much, but the completion dates for each step were calculated to be half of what they used to be. Of course, everyone b*tched about these ridiculous dates (I was a little upset too), but after about a month or so of complaining, we found that we were able to do the same job in half the time without extra hours or less quality. Sure, people were pissed, but the experiment worked.

Ever since that experience, I try to make dates as tight as possible and manage the risk with padding at the end.

Tom Fairlie
Wednesday, February 26, 2003

Tom:  I'm surprised.  I'd expect the engineers to become gradually more demoralized as they were repeatedly faced with impossible schedules.

Brent P. Newhall
Friday, February 28, 2003

Interesting post Tom. I'd really be interested to know why this shortening of the deadlines achieved the result it did. I've seen plenty of cases where unrealistic deadlines had the opposite effect - things either took much longer or the quality dropped so low that it had to be redone.

The best explanation I can think of is that developers were 'gold plating' their work, in the sense of adding features that weren't strictly necessary, or using the time to investigate algorithmically nice solutions, and that the reduced timescales got everybody focussing on what was important. Is it also possible that there was work that needed to be done that was not really in the schedules, and that putting the padding at the end allowed time to do it?

David Clayworth
Friday, February 28, 2003

I know exactly why this works.  When you have had disfunctional teams and a history of late projects, the team all starts mentally padding their estimates.  They know if they are late with a piece of work, they are in trouble.  Soon, the padding becomes mentally doubling the estimate, and any 2 week piece is quoted as 4 weeks.  And sure enough, it takes 4 weeks, not because 2 weeks was a bad estimate, but because he has 4 weeks to do it in, and there is no reward for being early.  And they know up front that the project is going to be late anyway, so does accuracy really matter.

The way we have taken to solve this is to say that once the project is underway, all slack time belongs to the project manager.  Coders give their honest estimates in a range of best, likely, and outrageous, we calculate schedules based on the likely times, and quote the project times to upper management based on outrageous times. 

The coders are then held to their likely times, but the project manager can give some schedule time to the task if it runs into trouble.  We also use the likely VS outrageous times as a way to manage risk.  We dont have to keep track of which tasks have built-in slack, and, obviously, if you can't build in slack time, being accurate is more important.  We now have more faith in the numbers in the project plan.

I think the point of Tom's story is that the new deadlines given were not impossible.  They just did not have that automatic padding in it.

glaqua
Monday, March 03, 2003

What kills you when you use "outrageous times" for all of your tasks is what statisticians call the "Law of Large Numbers."  Steve McConnell talks about this extensively in his book, "Rapid Development."

To summarize, half the tasks will take longer than average, and half will use less time.  These tend to cancel each other out and force overall performance to be more or less average.  When you plan each task for the worst case, you ignore this effect so you wind up inflating your schedule.

Of course, it's easy to bring a project in early under these circumstances, but it doesn't allow you to offer a competitive schedule.  Proper buffer management allows you to pad the schedule to reduce risk to a desired level, but doesn't cause you to add too much padding.

If you want to see what I mean, do the following experiment a thousand times on your computer, and see how many times the total winds up being significantly less than your worst case estimate:

1.  Choose a lower bound and an upper bound.
2.  Use the random number generator to choose twenty values in this interval.
3.  Add these numbers up and get a total.

This experiment will show you how a chain of twenty tasks behaves, assuming a uniform distribution between the lower and upper bounds for the task duration.  If you plot the results you will see how they tend to cluster between these boundaries.  You can use a distribution like this to determine the proper amount of buffer to use to achieve a given level of confidence in your estimates without going overboard. In practice, I'd modify the assumption of a uniform distribution for task duration, though.  Use a lognormal or similar distribution instead for general cases.

A nice benefit of doing things this way is that you have data to back up what you tell management.  If they demand that you shorten the schedule, ask them which of the twenty thousand numbers that went into the calculation you should modify so that they get the result they want (assuming you are a smart ass).  Also, make sure you show them the risk level you've chosen and compare it to the risk level for the schedule they'd like.  If they want to improve their risk exposure and still keep the compressed schedule, then they had better get busy nullifying all those risks for you.

Anonymous Jonah
Tuesday, March 04, 2003

*  Recent Topics

*  Fog Creek Home