|
Framework and "big code" obsessions - a data point
The blog entry "Does Good Code Matter?" is relevant to me personally.
I just ended a business relationship with a client, an ISV, that among other things has an obsession with grand designs and frameworks. Not for web sites, but for simple, stupid-assed, accounting related data entry applications.
This is mainly the owner's doing. The owner is a CPA and self educated code hack who would probably design a framework for "Hello, World" and can't imagine "just writing" a program. The guy has developed immense libraries for a new (and a "will never be done pipe dream") that seem to serve no purpose other than duplicating the built in features of the development language. Enormous, multi-hundred line procedures with many input and output parameters; unknowable variable lifetimes and initialization and termination conditions; and, layers and layers of ad hoc class abstraction. Basically the stuff is like a "science fair" exhibit of the syntax capabilties of Delphi. Really, really evil code that deserves "FORMAT C:".
One often banal truism in our industry is that the stupid and really bad programmers create the biggest and most complicated code. Having a high energy level and being dedicated to using that energy for coding is not often a good thing.
The thing I found with this guy, though, is that he was mesmerized by his own "creation". He absolutely could see no direct or simple ways to do *anything*. And he could not be persuaded that every part of his code was anything but absolutely necessary.
I believe, from observing this person and from seeing others get wrapped up in the same kind of activity, that the problem of big, bloated code is often (not always) due to professional ego and narcissism, with perhaps a martyr complex thrown in. IE: the designer simply makes an ongoing decision that his ideas are always correct and can never be simplified or abstracted - he stands alone in his demigodlike wisdom. The martyr complex comes into play in that the work of dealing with unnecessary complexity becomes so difficult, unrewarding and thankless that he and his people come to believe that coding must be pure pain and sacrifice.
Bored Bystander
Friday, March 12, 2004
I don't know about the stupidest creating the biggtest and most complicated code, but I do know that the stupid can sometimes produce massive volumes of code.
The reason is they never seem to think anything through at all when they write code. Reading their code is like reading a stream of conciousness. They just start typing straight away, and keep on typing code until it works, and never bother to go back to tidy up after themselves.
So the worst part is that they look very productive to people who don't know any better (look how much code X wrote!), and tend to get promoted instead of fired. :(
Sum Dum Gai
Friday, March 12, 2004
I once worked at a company where a programmer rode
this sort of nonsense all the way to VP Engineering.
After which, the company imploded - because nobody
could actually get anything to work...
I was long gone by then; the company _was_ a solid
company with a great product, right until this guy
decided to build his career using a hideous "framework"
that was all about "elegance" and had nothing to do
whatsoever with the company's product or its business.
x
Friday, March 12, 2004
Ah, this is a favorite topic of mine.
Personally, I've studied design patterns and OOP for the better part of six years, and I've ceased to be fascinated with the architectural mentality. Good programmers see their craft as a surgical exercise, and the purpose of architecture is to compartmentalize and componentize what works, that it may endure for posterity.
That means: short, iterative design-and-code cycles. Where "short" is defined as "as short as possible." Small projects will have much shorter cycle times than enterprise-scale applications.
Only in short cycles do the really useful, _working_, requirements-meeting bits of code that _should_ be saved exist to begin with. Otherwise the architecture is nothing but pure conjecture about what may happen. And only in such cycles is the design updated enough to capture the learned knowledge about real customer requirements in its framework.
Otherwise, Bored, I think you're just describing architectural astronautism at its finest. It's a fallacy: that if the architecture accounts for everything, then the architect is a master of everything. Nowhere in that fallacy is time or cost taken into account. And it's antithetical to real work on customer requirements, because typically that work _disproves_ the purity of the design. It's impossible for an egotistical astronaut to move forward because reality hurts.
Good programmers and architects take reality into account early and often.
indeed
Friday, March 12, 2004
Plus: I'm sure building a grand-encompassing framework to serve a Greater Good (tm) can earn more career points than someone just trying to get today's job done.
Frameworks can easily do more harm than good, cost more than save.
Joel Goodwin
Friday, March 12, 2004
>> Otherwise the architecture is nothing but pure conjecture about what may happen.
I think this is what has pretty much driven me out of software development. Almost all the work I've done for someone else in the last 15 years seems utterly senseless. It has no impact, it helps no-one, it just pays me while some misguided twerp flushes his money down the drain.
So, guys, what do you think of a company owner who creates a mess like this? Generally, in discussions like this, the emphasis is on the employee who is justifying their actions. In the case I described, it's a company owner who himself is preaching "customer benefits" on one hand while creating something that no mortal can comprehend and which never gets "done"...
Bored Bystander
Friday, March 12, 2004
"So, guys, what do you think of a company owner who creates a mess like this? Generally, in discussions like this, the emphasis is on the employee who is justifying their actions. In the case I described, it's a company owner who himself is preaching "customer benefits" on one hand while creating something that no mortal can comprehend and which never gets "done"... "
Well, I suppose the cliche answer here is "market forces will take him down." If only. :)
I don't want to hijack the thread, but I should clarify your experience with something I've learned from working for a small company:
We always romanticize small companies as being the bastions of individual, rugged enterpreneurship. And sometimes they are. But they're also often unchecked, feudalist cultures where one ego controls the direction of his own little world. That's because small company owners often have no obligations but to feed themselves and their family; they don't have to satiate outside investors, and there's noone that can give them a good ass-kicking (except perhaps the occasional customer, but that usually gets passed down to the employees).
This architectural astronautism is a symptom of a bigger problem, which is just plain, old unchecked power. I suppose it's not the worst way for him to vent his ego, but I imagine it's not a nice or productive working environment, either.
Programmers who "own" projects really experience the same thing. They're given a lot of power to create castles in the sky, and it takes a very long time for that power to be checked by any outside authority. By that time, they've surely achieved their short-term objective of being paid well for mental masturbation.
indeed
Friday, March 12, 2004
> Frameworks can easily do more harm than good,
>cost more than save.
Is there anything you can't say that about?
Is there anything you can't say the reverse as well?
Poor execution is poor execution. There's nothing
in creating frameworks that requires poor results.
son of parnas
Saturday, March 13, 2004
Yes, sure enough my statement on "cause more harm, increase costs" is so ultimately general it can be applied to teapots too.
My intended point was that the belief that having a framework will solve all the developmental ills is akin to believing that the waterfall approach to project management is sound. The waterfall approach *can* work yes, but it has a tendancy to cause a lot of harm particuarly in larger projects. That doesn't stop armies of managers dictating the waterfall method, because it's "safe".
I feel the same about grand frameworks which are often initiated at the drop of a hat, because they're a "good thing to do, will save developer time in the future", without realising this is like the Mother of All Waterfall Projects. Getting it right takes a lot of time; getting it wrong will cost even more time.
Joel Goodwin
Saturday, March 13, 2004
Frameworks are well meaning really. People have the idea that it will simplify things. An admirable goal. Another goal is that they'll allow reuse - you can just take the framework and write multiple projects on top of it. Once again, an admirable goal. The only problem is that neither of these are usually achieved by frameworks: they're more likely to be achieved by libraries.
Of course, the issue is what starts out as a library can easily become a framework.
Libraries are, in my mind, a great thing. They're probably the only thing we know are reusable. A great big object framework is questionable, but a simple library function that takes X and produces Y is much simpler and easy to use without commiting your program too much in one direction.
The other win for libraries is that they're easy to plug into later. Want to use LDAP at some point in the future? Using an LDAP library is easy. Using a framework built around LDAP after the fact is almost certainly impossible: you have to modify your whole archetecture to suit it.
The problem is, one you write a library, the framework seems to become the great temptation. In a moment of programming hubris, the programmer sees how good their library is, and thinks of how much better the world would be if everything in the project was based on it.
Our job as developers in such a situation is to "just say no"! Don't give into the temptation of over generalising. :)
Personally, I love writing libraries. It's what I'm best at, since I find it easier to write good UIs from programmers (eg APIs) than UIs for users. I get a real kick out of knowing I'm making everyone else more productive because I've abstracted the details away into a library that they can treat as a black box.
I've given up trying to write frameworks though. I've tried to make good frameworks, really I have. I'm convinced that good framework is almost an oxymoron. There have been a few I've seen (Delphi's VCL for example), but for every gem, there's at least 100 duds.
It's just too difficult to strike the right balance between being generic enough to handle enough cases to be useful, and specific enough to not make it more effort than just writing to the underlying APIs directly. Once you consider the flexibility costs you incur by commiting to a particular framework, it just doesn't seem worth it to me. So I'll stick to writing libraries. If I could only find a job where I specialised as a library and tools programmer, I might even find some joy in my job. ;)
Sum Dum Gai
Saturday, March 13, 2004
A library and a framework are no different. If you
can't make one you can't make the other. If you
make over generalized anything that is your
own failing that you should not project on others.
If framing something as library plays tricks with
your brain then ok, but you still are tricking yourself.
son of parnas
Saturday, March 13, 2004
To build a good framework, is necessary experience with the domain of the framework. Experience one'll get by developing applications on that domain.
After building several similar applications, one will have the experience, and even better, actual code that works, and can be refactored, to create a framework.
Frameworks don't pop out of nowhere. They come from your past work, evolving and improving. At least that's how I think it should work for small ISVs.
Ricardo Antunes da Costa
Saturday, March 13, 2004
"There's nothing
in creating frameworks that requires poor results. "
Well, I think there is.
Sturgeon's law states that 90% of everything is crap. Frameworks are abstract models (within a given domain) of...everything! And the more generalized they are, the truer that is.
Why create generalized abstractions, when 90% of their applications are utterly useless?
I know Sturgeon's law is a tongue-in-cheek heuristic, but in my experience, the vast majority of effort put into abstractions and so-called frameworks is wasted.
As a tangent to that discussion: how many organizations collect metrics on their OOP code to see just how often those abstract factories and delegates are extended?
indeed
Saturday, March 13, 2004
>Sturgeon's law states that 90% of everything is crap.
Then it doesn't matter. Whatever you write will be
crap. The more generalized doesn't matter because
it can't be any truer.
>Why create generalized abstractions, when
>90% of their applications are utterly useless?
Agreed. Don't write useless abstractions. Did anyone
say you should? That 10% can be extemely powerful
however.
>the vast majority of effort put into abstractions and >so-called frameworks is wasted.
The vast majority of code is crap so drawing generalizations
is worthless.
> how often those abstract factories and delegates
>are extended?
If that's all a framework means to you then it is no wonder
you see little value.
son of parnas
Saturday, March 13, 2004
^
"Agreed. Don't write useless abstractions. Did anyone
say you should? That 10% can be extemely powerful
however."
Well, I agree that the 10% can be extremely powerful. It can also be represented in more concrete terms, without the overhead of the other 90%.
Plus, I think 10% is an overestimate of an abstraction's value, because ironically, most are not encompassing enough. With OOP, for example, the level of granularity at which changes are made via polymorphism or composition, is still pretty high. There are often a lot of ugly "line by line" changes that represent business/customer reality that abstractions don't take into account.
When Joel talks about rewriting a codebase as the primary mistake a software company can make, I think he's speaking to that particular flaw of the framework concept.
"If that's all a framework means to you"
I didn't say it was all it means to me; I was suggesting that companies measure _in some way_ the degree to which their abstractions are actually used. My point isn't that there's a perfect way to measure the usefulness of abstractions, but that companies should take _some_ interest in whether they're getting a return on their purchase of an architecture.
Why the vitriol? Have I offended your evangelical sensibilities?
indeed
Saturday, March 13, 2004
> Why the vitriol? Have I offended your
> evangelical sensibilities?
You are making categorial statements about how much
frameworks suck. That the problems are unavoidable
even if they did have value, which they don't.
The evangelism is on your side.
I have used frameworks to great effect on many
projects. For you to deny the direct experience
of another in favor your vision is again evangelism.
You don't have to over generalize. Even the right
5%, for example, in the embedded world, could
mean creating a new card in 2 months rather than
15. That's a market window. That's a lot of money.
That's reality. Real example BTW.
son of parnas
Saturday, March 13, 2004
> Good programmers see their craft as a surgical exercise, and the purpose of architecture is to compartmentalize and componentize what works, that it may endure for posterity.
Well said. I wonder how often the architecture is created post-facto, perhaps as a refactoring of an existing working implementation.
Christopher Wells
Saturday, March 13, 2004
Bored, to come back to your original question, I don't there's much that can be said. The company owner is the company owner, and if he/she is over-designing their system architecture, and over-designing badly, then it's their call. If you manage to pull off convincing the owner down a better path, then kudos to you; however, seems more likely that there would be little you or I could do about said mistake.
It doesn't bode well for this particular company's future, but c'est la vie. Just let me know who they are so I don't end up working for them =)
Joel Goodwin
Saturday, March 13, 2004
"You don't have to over generalize. Even the right
5%, for example, in the embedded world, could
mean creating a new card in 2 months rather than
15. That's a market window. That's a lot of money.
That's reality. Real example BTW."
But in this scenario, is it the framework per se that creates additional value, or the real understanding of the requirements that goes into properly developing from its abstractions?
My point is that the abstractions themselves don't alleviate problems with understanding reality; they're just so much architectural and syntactical sugar.
I just find it very hard to believe that the mere presence of architectural constructs makes the project ship six times faster, versus a really good (strategic) understanding of what should be done. In my experience--which may be contrary to yours--poor understanding of reality is what delays projects.
Before responding to your post, I had suggested that the way to incorporate abstraction is after-the-fact, so that it codifies what's really understood about customer and business requirements. I still don't think that's a panacea or a major paradigm compared to understanding requirements very well, but it's useful.
So I'll concede that my "categorical' statement is not true in 100% of cases, but I definitely believe that the framework mentality/martyr complex (as Bored Bystander put forth), the Law of Leaky Abstractions, and Sturgeon's law all make abstraction a pretty risky enterprise.
indeed
Saturday, March 13, 2004
I think that 'indeed' has a good point in being cautious of premature abstraction. When you're experimenting with some new process, it's *easiest* to get something done by minimizing the number of abstractions that you deal with. When you understand the nature of the process, it's easiest to use an abstraction that lets you ignore all details except those relevant to that process.
K
Saturday, March 13, 2004
>I just find it very hard to believe that the mere presence of >architectural constructs makes the project ship six times >faster, versus a really good (strategic) understanding of >what should be done.
Let's say you have a really good strategic understanding
of how to build a plane. How long will it take you to
make a plane from scratch? How long would it take
boeing? They could do it 1000000 times faster because
even with your really good strategic understanding
there is boat load of work to do. Much of which
even with your excellent strategic understanding,
you probably missed, because the details are in
the tactics.
>But in this scenario, is it the framework per se that creates >additional value, or the real understanding of the >requirements that goes into properly developing from its >abstractions?
A good framework embodies the requirements so you can
concetrate on just what you need to do with most of the
gotchas worked out. I for one am glad to use an XML
parser framework so i don't have to develop a
strategic understanding of all the specs.
>In my experience--which may be contrary to
>yours--poor understanding of reality is
> what delays projects.
No disagreement. But once you have the understanding
you still may have a 100 man years of work ahead of
you.
>Before responding to your post, I had suggested that the >way to incorporate abstraction is after-the-fact, so that it >codifies what's really understood about customer and >business requirements.
That's how good frameworks are developed. From
experience on real products doing real things.
Why don't you think frameworks can be developed
that way?
>Sturgeon's law all make abstraction a pretty risky >enterprise.
For someone who does programming for a living
that is pretty ironic.
son of parnas
Saturday, March 13, 2004
The difference between a library and a framework is
statistical, not a law of physics.
Statistically, libraries tend to bea set of functions for
doing a particular task, and frameworks tend to be
SpaghettiHeirarchyNightmare.cpp + documentation.
Like MFC.
And don't talk about using frameworks for embedded
applications, most embedded controllers have about
256 BYTES of memory space.
You ain't gonna fit a framework in that, and suggesting
it make you look stupid.
A
Sunday, March 14, 2004
> 256 BYTES of memory space.
Not in today's embedded systems. You can have many
MB of RAM and powerful processors. Many have
multiple cards and fast ethernet networks.
>You ain't gonna fit a framework in that, and
> suggesting it make you look stupid.
We found code written to a framework is actually
much smaller than the equivalent system without
frameworks. The reason was you could derive
from and specialize classes such that the only
additional code was only what was needed to
do the new functionality. Common code was written
only once. What develops is that any new functionality
adds very little code.
son of parnas
Sunday, March 14, 2004
 Recent Topics
Fog Creek Home
|
