Fog Creek Software
Discussion Board




The API is king



Ive just read the API war article:

http://www.joelonsoftware.com/articles/APIWar.html

I found it insightfull but as an object oriented purist i would dispute the assertion that object oriented programming hasn't lived up to its expectations.  Particularly where Windows development is concerned.

The fact remains that most people just don't get object oriented programming. This is just plain and simple becuase for most of us object oriented programming is too time consuming and difficult. To write good object oriented code you must be fanatical about consistency, reduction, and symetry to the point where you will seemingly spend most of your time refactoring existing code so that it fits with the coninuously evolving analogy that is your object model.

The good thing about object oriented programming is that you only have to do this once.  That is to say, once you have obtained an elegant solution to a particular problem and you have the right abstractions to work with you will not have to do it again. The developers who use your code will have improved productivity as a result of the object orientation that you have added.

Object oriented programming is about cutting a trail through the jungle of purpose until you have a path of logic and reason which other people can follow blindly.  An object oriented API gives you a trajectory, a way of working, an idiom to follow.  This is what makes it productive, not whether it is managed or unmanaged.

The Windows API is not object oriented.  It is at most a functional API with assortments of functions collated into containers.  Using classes does not make code object oriented.  It is the nature of the design that makes code object oriented.

Java, .NET, and most of the interpreted languages have two things in common.  They are managed, but they also have good object oriented API's.  As much as Joel is correct in stating that managed code frees the developer from many constraints i believe that it is in fact the API which is equally if not more responsible for enabling a developer to work with higher level abstractions.  With the right objects and tools everything becomes easy.  With memory management, reflection, and code generation things become even easier.

Whilst i agree that managed code is easier to work with. In an unmanaged environment many common problems, such as the issue of string management described in the article, are trivial to avoid.  All this requires is a little forthought and a good idiom.  If you follow the rule of consistency and the principle of least surprise your unmanaged code will be just as useable as your managed code.

Never return pointers such as primitive strings, and where possible use pass by reference instead of returning a value. Return by value solves the memory management issue by it induces a copy (except where the compiler implements the return value optimisation). 

Assume you have a String object which adheres to the following C++ interface:

String
{
public:
    void Append(char * data, int size);
    void Resize(int amount);
    int Remaining();
    int Size();    
};

It is now trivial to pass around and allocate new memory since you never have to worry where the memory was allocated if you have an object which manages its own memory and you only pass by reference.  The f(g(x)) could be implmeneted like this:


void g(String & string)
{
    int required = ...;

    if (string.Remaining() < required )
        string.Resize(required - string.Size());

    string.Append(...);
}

void f(String & string)
{
    ...
}

void main (int argc, char ** argv)
{
    String x;
    g(x);
    f(x);    
}

The Windows API doesn't do this becuase it is primarily a non object oriented piece of code. It should be recognised that you can in fact write good object oriented code in C just as you can in C++ or Java or .NET simply by following naming conventions and passing pointers to structures.  Object orientation is more about consistency than anything else.

The fact is that there will always be applications which require a non-managed environment where the overhead of garbage collection and the lack of performance or response time guarantess is a non starter.  Windows will always need to run these applications and it will always have a kernel level interface.

Programming is about semantics and purpose not about syntax, the API is king and the language is meaningless.

Emerson Clarke
Saturday, August 28, 2004

Those who won't refactor end up with junk after a few releases.

This is just as true for procedural models as for OO.

90% of projects are in this state. (Well, I would say 100% but I don't want to overgeneralize from my own experience).

frustrated
Saturday, August 28, 2004

Speaking of which, what about MacOS X? Did Apple take this opportunity to rewrite a set of API's in an OO way like .Net, or are they still using procedural calls?

Fred
Saturday, August 28, 2004

Wow, what a great post.

I'd like to say one thing about OO:

One reason that OO is never implemented - and as a larger reason why code is not refactored - is that its hard to make the "business case" for it.

The trail of blood, which has been explained to death in this forum:
1. Fundamental research.
2. make demo,
3. create v1.0,
4. hope to redesign, cleanup, refactor, etc.

Item 3 often never happens because the customer wants new features, not refactoring. And the case never gets made to staff for redesign - at least until way late in the program - if at all.  And then its often just at the edges because outsiders want to couple into your product.

The fundamental research part, for me, has been probing hardware to make it respond, understanding new protocols, stuff that is not inherently OO in nature. So design takes on different meanings based on context - it does not mean specifically software design.

Also, there are not that many people capable of leading a project who have strong OO skills.  What I have seen happen is a strong OO tehcnical lead get quashed by managers - including PMs - who insist on "making it work"; not having a clean design. Back to biz case. Also, an OO design lead has to be able to demonstrate by example how OO is better than <the process in place> - whatever that happens to be.

There just aren't that many capable OO people to go around - in my experience.

I'm becoming an OO convert - but this conversion is only because I've seen it done right and it is compelling (the code base is amazingly maintainable, extensible, and exportable (meaning outsiders can easily write to and extend it).

Well, a little more OO cirticism.  The literature on patterns and UML is wanting.  Although a good book I've found is "Design Patterns" by Erich Gamma et al.  Very good as useful - something you can sink your teeth into.

But too often I've read the OO literature and thought what a bunch of fluff.  This will not help me solve problem X.  And BTW, kill the use case diagrams and quit wasting time on "why" OO is needed (just link your post here:)).  IMO: use case diagrams are the powerpoint of OO.  Use cases should be outlined, not diagramed.  Diagrams are harder to change, harder to create and don't convey anything.  Outlines are the exact opposite and can be text files if necessary.

Wel, that turned in to a rant.  Great post.

hoser
Saturday, August 28, 2004

Whoop, change text "item 3 never gets implementedd" to "item 4".

hoser
Saturday, August 28, 2004

"""What I have seen happen is a strong OO tehcnical lead get quashed by managers - including PMs - who insist on "making it work"; not having a clean design"""

What I don't get is why anybody ever *tells* the client about refactoring or clean designs or any of that.  Why on earth tell them about stuff they don't care about?  That's *your* job.  Just include the time to do it in the time required for stuff they care about.

Phillip J. Eby
Saturday, August 28, 2004

hoser,

> Item 3 often never happens because the customer wants new features, not refactoring.

At my last project, someone had written some proof-of-concept code which exercised the 3rd-party software that our product was to interface with ... and then I was brought in to write the product software.

The proof-of-concept code didn't have the right architecture for the product ... and it was poorly factored to begin with. Because it didn't work (it exercised the 3rd-party software, but didn't implement the functionality of the product I was supposed to write), I decided (not without some reluctance) to write the product from scratch instead of trying to refactor the proof-of-concept into a product. When I decided to do that, and said so in my Design Plan, my decision was supported by the company's CIO (who said he often likes to write-from-scratch instead of refactor).

I'll second the "Design Patterns" book. Coming from a "C" and assembly background, I learned the C++ language ... and then read "Design Patterns", which I found explained to me *why* I would want to use a language that supports OOD/OOP.

> What I have seen happen is a strong OO tehcnical lead get quashed by managers - including PMs - who insist on "making it work"; not having a clean design.

I don't see how that can happen.

When you're writing from scratch, you need to do design-before-coding ... surely anyone knows that. There's no question of "making it work" before coding, and no question of coding before designing.

When you're adding a new feature to existing code, the politics can be a little trickier: still, having a clean design is part of my definition of its "working": because if I don't understand the software that I've written then I can't assert that it's not only working with the current set of test cases, but is also likely to work correctly with any conceivable test case; and if the design isn't "clean" then I'm not likely to understand it.

Besides, a manager or PM who doesn't understand design-and-code (and refactor) doesn't get a vote in choosing the development process: I'll just tell them that I'm "working on" it or "implementing" it, and keep the details (like whether I'm currently refactoring, or designing, or testing, or whatever) as a private implementation detail.

> I'm becoming an OO convert - but this conversion is only because I've seen it done right and it is compelling (the code base is amazingly maintainable, extensible, and exportable (meaning outsiders can easily write to and extend it).

Isn't "how well the code base is documented" also a key factor in its being accessible to "outsiders"?

I'm willing to hack undocumented code bases, but a lot of my colleagues are pretty reluctant to do so.

I'm currently (as an experiment) documenting an existing code base: its internal structure as well as its public API; and I've found that it has taken me 20 or 30 hours to doocument some classes that took me 40 hours to code.

Christopher Wells
Saturday, August 28, 2004

"Speaking of which, what about MacOS X? Did Apple take this opportunity to rewrite a set of API's in an OO way like .Net, or are they still using procedural calls?"

MacOS X is really the latest version of NeXTStep which was written in the nearly 90's.  At that time, yes, they did take the opportunity to write an entirely OO operating system API.  They even created the Objective C language to do it. 

Mac OS is just the latest version of NeXTStep with additional seperate procedural APIs for backwards (source-level) compatibilty with previous Mac OSes.

Almost Anonymous
Saturday, August 28, 2004

--- The Windows API is not object oriented.  It is at most a functional API with assortments of functions collated into containers.  Using classes does not make code object oriented.  It is the nature of the design that makes code object oriented. ----

Incorrect.
Windows API is object based.

For each object there is a constructor  - CreateHandleXXX
and a destructor DestroyHandleXXX
Encapsulation is achieved by having lots of functions that receive HANDLE to object as argument.

No inheritance, little or no polyphormism, but that's good enough for definition of 'object based'

Michael Moser
Saturday, August 28, 2004

Speaking about your example, you can always have a wrapper class for any API, so that it gets a String object.

Thats exactly what MFC if good for.

Michael Moser
Saturday, August 28, 2004

> little or no polyphormism

Well, for example many different kinds of object can be passed to the ReadFile API; VirtualAlloc allocates many kinds of memory, and CreateFile opens many kinds of file and kinds of device; and most importantly, you can do SendMessage et al. to a lot of different types of window.

Christopher Wells
Saturday, August 28, 2004

>Why on earth tell them about stuff they don't care
>about?  That's *your* job.

God bless you.

I've met a lot of people in this business who have a hard time separating responsibilities in this way.  It's true for interprogrammer communication too.  I think that programmers ought to expect that outsiders will want a declarative description of their software (for use) and an imperative description only for people who'll be working on it too.

When I go to a mechanic I only have to say, "the windshield wipers don't work" (there's an implied contract here that we all understand about what qualifies as "working" windshield wipers) -- then he gets in and fiddles with the frobnobbers in such a way that my criteria are met.  Keeping the wires straight and the oil off of labels is his job, and it'd be silly for him to lecture me on the esoterica of best practices in his line of business.

I think that all this stuff about documentation and code organization really tells us something about what kinds of business models work well for us.  We *know* that these things are important, but it's also true that in many cases short term evaluation shows no difference between a hacked together mess and a well-designed system (if the car works even though it's drive-train is made of pretzels, it still works -- but for how long?).  So we have to make sure that we'll be evaluated over periods of time (and that the important properties of our work will be measured with more weight than the less important ones) in such a way that the quality of our work shows itself.

But I think that's the limit of what the client has to be bothered with (a declaration of the problem or solution which the program addresses, and an agreeable system for evaluating programmer performance).

Kalani
Saturday, August 28, 2004

it's -> its

I can't believe I did that.  All this Internet reading has ruined my grammar. :(

Kalani
Saturday, August 28, 2004

I completely agree with Phillip.
Refactoring is what I use to do my work better, and faster : I don't have to ask for time to do it.
I always tell my colleagues that there's no point in asking for a redesign or cleaning of a piece of software : they won't be given any time for it. They should know, this has never ever happened.

Pakter
Saturday, August 28, 2004

Object orient programming is not at a
"high level of abstraction". It is low-level
programming and suits the problem
domain of writing simulators and not much
else.

An example of a high-level programming
language would be Mathematica, which
suits the problem domain of symbolic
processing. Others include Lisp (see
PaulGraham.com), Prolog (logical inference)
MatLab (matrix processing) and even
frickin' SQL (predicate calculus, if written
properly).

Except for writing simulators, I really
don't see where the claims for such high
productivity for object languages come from.
Take for example a signal processing
application for telecommunications. You could
do this in a page of MatLab; If you tried to do
it the object way starting with the nouns/verbs
in the problem description, you would be busy
for eternity and once you had finished, the
code would be so massive it would be a
nightmare to debug.

And Lisp is functional.
Windows is not functional, it is procedural.

AAA
Saturday, August 28, 2004

>>Incorrect.
>>Windows API is object based.

as I have pointed out, many people equals OO with using a language which has Explicit syntax to support dynamical polymorphism.

You can use even C to implement OO design.

A few month ago,  I was interviewed by the CTO of an outsourcing company.

In his lecture, he tried to make two points (because as a C ++ programmer , I want the software development manager position).
1. All the Chinese programmers are foolish coders who can not design (The CTO is an America-born young man ).
2. C++ is not an OO language (Java is OO) because it does not support interface

I showed him my design and proved any qualified C programmer can implement Interface in C.

Of course, he did not give me the job.

--------------------------------------
D2KSoft, Different and Elegant
http://www.d2ksoft.com
We provide advanced tools only for YOU,
the smart, tasteful and lazy hacker!
--------------------------------------

redguardtoo
Saturday, August 28, 2004

OOP suits any case when you need to serve varying behaviors under a uniform interface. It does not really depend on high or low level; you can implement it at any level. Just how it is implemented - with Java-like classes or Javascript-like prototypes or Perl-like pseudo-objects -  is utterly unimportant.

Simulation is just one case where OOP is beneficial; others includes GUIs, user-customizable applications, parsers, and many more.

What most people are missing is that OOP is primarily about encapsulating complex behaviors in simple interfaces. (Unfortunately, most people think encapsulation is about "private" data and "public" methods.)

.
Sunday, August 29, 2004

Parsers are pattern matching and therefore
you take a huge hit in productivity and
correctness as soon as you step away from
declarative programming and towards objects.
In fact they match Prolog best.

As for GUIs, they are mostly IDE generated so
programming style is irrelevant. In cases where
programming is relevant, objects become just
as painful as they usually are - see:

http://www.tcl.tk/doc/scripting.html

I believe Ousterhout covers the case of users
extending an application in this paper as well.

What were the "may other domains" where
objects are allegedly a good fit?

AAA
Sunday, August 29, 2004

--- Parsers are pattern matching and therefore
you take a huge hit in productivity and
correctness as soon as you step away from
declarative programming and towards objects.
In fact they match Prolog best. ---

Prolog is very slow.

Prolog is very hard, in fact you need a ! cut for almost everything;
try to read and understand prolog code, it is ver very hard.

Michael Moser
Sunday, August 29, 2004

If you have a high quality implementation you can
get fast Prolog, fast Lisp, fast anything. OK lets
say Prolog is way too hard, what about ML?
Miranda? Haskell? There are many declarative
languages that are asy to program in, and they
fit pattern matching waaaaaaaay better than
objects. Or are you claiming Lisp is harder and
less productive than C++ for processing operator
trees?

AAA
Sunday, August 29, 2004

" to the point where you will seemingly spend most of your time refactoring existing code"

And how is that a good thing?

I have to agree with hoser, most software is written to be sold. If it sells but it's not a work of art, move on to the next product. If it doesn't sell, move on to the next product.

TomH
Sunday, August 29, 2004

AAA, I'd say OOP is good for tactical reasons. I can't imagine structuring most apps under an "OOP architecture," unless the language forces me to pay lip service, but I'd certainly use a little OOP spice here and there.

Experimenting on myself, with a multiparadigm language which doesn't force me anywhere, I find I use it in places to give data a little "oomph," but I don't go nuts with it. I don't believe in grand themes where everything is just an aspect of that theme. Unless I'm modelling modernist literature, maybe.

When you find a quote like, "With the right objects and tools everything becomes easy," then either you interpret it very loosely so it can mean whatever people need it to mean, or you just read it as PR. I don't think the original poster intended it to sound that way, but when taken seriously, people hear big promises.

Incidentally, the language I was referring to is Common Lisp. I don't want to think OOP and then talk to a client in their domain language. I want to think in their domain language, occasionally using things like OOP to describe that domain.

Tayssir John Gabbour
Sunday, August 29, 2004

I don't think it's honest to say that languages like Lisp or ML are "declarative" -- it's common to write "imperative" code in Lisp or ML.

I don't necessarily agree with the statement that Prolog is best for specifying parsers either.  With the Prolog syntax you can easily embed the declaration of a context-free grammar, but the nature of Prolog evaluation is such that your program is typically evaluated as a depth-first search.  If you want to do a breadth-first search for your parse results instead, that's too bad.

I do agree that the problem of parsing typically is solved best with a simple declarative language (context-free grammars, regular grammars, etc), but that doesn't really have any effect on whether or not "OOP" is valuable.  In my parsing library, for example, I have a function that takes a CFG as input and produces a pointer to a "parser" object as output.  The parser object is a front for whatever parsing implementation is best for the input grammar (something that the caller doesn't care about).  This is a common type of synthesis of "OOP" and declarative-language techniques -- another obvious example is SQL with ADO.

Kalani
Sunday, August 29, 2004

> Just include the time to do it in the time required for stuff they care about.

If you are bidding for a contract and your bid is (say) twice the other guy's it doesn't matter if your end result will be better (more maintainable etc.) because they will only see the price.


Monday, August 30, 2004

*  Recent Topics

*  Fog Creek Home