Fog Creek Software
Discussion Board

Civilization and "Leaky" Abstractions

In his article Joel discusses the affects of higher order abstractions on software development, and what happens when they fail (I really don't like the term leak in this context, though I can understand why Joel selected it).

While some of Joel's analogies stray from the realm of software development, e.g. the weird Hollywood Express example, I don't know if he considered the overall role of abstractions in human civilization in general.

The fact is, abstractions, as Joel defines them, are the building blocks for civilization. The ability to utilize existing knowledge and capabilities and build on top of them is how we advance in all tracks of science and engineering.

Joel is correct in stating the often abstractions fail and force us to fall back on the simpler "lower level" abstractions that compose or predate them. Still good abstractions are such that this doesn't happen too often. Example: when your washing machine breaks down you do your washing by hand. But if your washing machine breaks down too often you don't give up on automated clothes washing. You get a new machine.

You also do another thing when your washing machine fails - you call a repairperson. And that is another point that Joel misses. Abstractions work because civilization is also based on specialization. During the Stone Age most everyone knew exactly the same things. Now each of us has his own store of knowledge that defines and is defined by his position in society. In the context of the VB programmer: not every VB programmer in the shop needs to know C++ and Win32. You just need a sufficient number of such experts so that when the abstraction fails you have one on hand (and if the VB abstraction fails to often - replace it).

Another point on which I disagree with Joel is his expectation, and consequent disappointment that abstractions don't simplify our lives. They often don't because that is not their intention. If that were so our lives would be simpler than the lives of our forefathers. They aren't. In the Dark Ages the whole knowledge possessed by a common person (say a serf) could be contained in two pages of a book. Today, just to survive in modern society we must know volumes.

If abstractions don't make our lives simpler, what do they provide? They make our lives *better*. They let us solve problems that were unsolvable before. They enable progress. Example from software development: writing an application today for Windows (or even .NET) isn't necessarily easier than developing an application for DOS 10 years ago. Indeed often larger teams are involved and product release cycle is longer. But the resulting Windows applications do a lot more than their old DOS counterparts, and are generally more usable.

Dan Shappir
Wednesday, November 13, 2002

I disagree with this on many levels.  For one thing, people always point out the dark ages as some representative of our past.  But it was a singularly low point in any culture's history, predated by far more advanced civilizations worldwide.

As for specialization, it's often a terrible thing, despite being efficient.  Redundancy means sustainability.  Look at the US' dependence on oil; OPEC knows its cartel position and takes advantage of it.

You will always get weird emergent properties from the limitations of the substate you build on.  That's the problem -- building abstractions on things which are already limited.  Lisp is great; but shit, you want garbage collection because mem is finite.  Or let's use C; but wait, imperative styles are hard to scale to multiple processors.

I think most of the mistakes I see people make, financial or otherwise, are a result of people putting too much faith in a model.  That's the secret of the con game, leading people to believe a model that is consistent, but built on a flawed foundation.  Java is one such con game.

Wednesday, November 13, 2002

"That's the secret of the con game, leading people to believe a model that is consistent, but built on a flawed foundation.  Java is one such con game."

Can you expand on this remark please? What exactly do you mean?

John Topley
Wednesday, November 13, 2002

Your comments seem to reflect unquestioning faith in the process of abstraction and specialisation.

There are limitations, and the process is not without pain.

You say that the average serf's entire knowledge could be fitted on a couple of pages.  This is not true.  The knowledge that has survived might fit on a couple of pages, but the serf had an immense amount of knowledge that has been lost to time.

Skills in farming, fuedal law, archery, surviving on a subsidance diet.  The average serf knew plenty.

It might be said that any knowledge that has been lost has failed in the evolution of the memes.  Quite possibe, but since that knowledge is now lost, we cannot say for sure.

Abstaction is a powerful tool.  The specialisation of roles is also of great benefit.  However, there is always a trade off and that has to be remembered.

Ged Byrne
Wednesday, November 13, 2002

I don't know what Tj meant by this, but I found it funny that Java was
not IEEE754 compliant. I am not sure of the current status. This article
was written 1 March 1998 William Kahan, "father" of IEEE754.

I am currently using IEEE754 directed rounding, but I don't use Java
so I don't know if it works there. It did not back in 98.
Anyone that uses Java that knows ?
How about C# or .NET or these new Microsoft constructs ?

Wednesday, November 13, 2002

I never meant to argue that specialization is without its problems. Case in point, I always envied people who could lift the front of their car and fix a problem instead of having to wait, like me, for a tow service.

OTOH nobody can be good at everything. Indeed it's very hard to be very good at more than one thing. Increasing specialization, like it or not, is the direction in which we are heading. Just look at the various scientific fields. The most you can do is keep an open mind and appreciate the fact the TV and the Internet make it much easier to obtain information, thus allowing you to have general knowledge, if not actual specialization, in various fields of interest.

The Dark Ages were indeed a low point in Western civilization, however lets remember that human civilization is 10K years old while humanity is ten times that. So for the greatest part of human existence, man has lived with little or no civilization and very few abstractions.

Serfs during the Dark Ages did posses relatively little information: they never left their village, often not being aware of the existence of any other people beside their own neighbors and a few from the next village. They didn't know feudal law, just their master's whims, and the other things mentioned are often more instinct than knowledge. And yet, even at that time there were abstractions and specialists.

I do not have unquestioning faith in the process of abstraction and specialization. I even don't have unquestioning faith in benefits of civilization (at least not every civilization). It's just that I don't see any better alternative. Stated simply, modern man's problems notwithstanding, I'm not willing to give up on either dentists or toilet facilities.

Dan Shappir
Wednesday, November 13, 2002

In an excellent book, called 'Frontiers of Complexity,' I read about simulated annealing.

I think the application of abstraction is an annealing type problem.  You cannot do it too quickly, or else you introduce faults.  The problem is that companys desperately rush to enforce abstractions.  This is where it goes wrong.

The same happens with in civilization where countries try to enfore abstract models on their own or other nations.

The problem is always with the puritans, who see only faults with the world and not their own models.

Also,  I still suspect there was more to our ancestors than your abstracton allows for. 

Ged Byrne
Wednesday, November 13, 2002

I'm definitely not saying that Java is a bad thing.  But there are certain cons in the industry to skew public opinion.  I understand "Code Morphing" was used by Transmeta to set the VCs drooling, and they probably imagined running Mac OS on a codemorphing chip.  Transmeta knew it was just a buzzword they made up so the VCs would give them the time of day, but it presented a beautiful and coherent picture of the world.

"100% Pure Java" is another such con.  It caused VCs and analysts to to insist their companies use Java early & often.  This drove demand for Java programmers and now there is a vast developer base.  Unfortunately, the hype always came before the extensions necessary to deliver it.  (Yesterday was the anniversary of the realtime spec v1.)  So Sun was able to boil the ocean through a necessary con.  Sun engineers were always honest
but marketing did a good job too.  Not everyone was conned, just those who were a little too ready to believe, like many VCs.

I thought it everyone was used to it; companies make vaporware announcements all the time to kill weak competitors.  Sun in comparison was probably benign.

Wednesday, November 13, 2002

BTW, my real ontopic point is that abstractions cause big sustainability issues that need to be addressed. Ripple-effects will happen, if a plague or catastrophe happens to a localized supplier.  The need for packaging and long distance transport creates waste and inefficiency.  If your weak country specializes in cattle, which I hear is pretty unsustainable, then you're at the mercy of other nations' tariffs and whims.

But I have this weird feeling that
Joel: "Don't forget that abstractions are leaky,"
Dan:"Don't forget that abstractions are useful." 

Something like violent agreement.

Wednesday, November 13, 2002

*  Recent Topics

*  Fog Creek Home