Fog Creek Software
Discussion Board




YAOSD (Yet another open source discussion).

Okay so here's my problem with open source.

Having the source-code isn't necessarily a good thing.
I don't want to look at the source code for your operating system, database server, etc.  Why?

Because I want you to give me clean, well designed and documented interfaces into your system.  I.e. Interface design and implementation are two seperate things.  I want to program to an interface, not an implementation.  The feeling I get is that most open-source projects don't give much thought to APIs and the like, because of course you always have the source-code to look at.

I've noticed on the Groove web-site that some people don't like that fact that Groove isn't open-source.  Who gives a shit, from what I've seen Groove has a nice, well designed COM-based API (documentation could use a little work though).  Isn't that better than having the source code and having to score through 99-billion lines of code to figure out what's going on.

Another point -- I don't have extensive experience with open-source, but as far as I know LINUX uses POSIX, which is probably an adequate API but hardly bleeding edge.  How about using components to build your operating system API, or a functional language based interface, etc.  The open-source guys complain that microsoft isn't innovative, but why the hell are they using an API from the 1980's.

There I feel much better now that I got that off my chest...

nojuan
Monday, September 02, 2002

Source tells you:
a) bugs
b) performance characteristics
c) ability to not just be a consumer, but producer; "peer to peer"

anon
Monday, September 02, 2002

Also, having the source give you a fall-back option if the code's owner decided to end-of-life the product (or goes out of business) or just is unresponsive to bug reports. The code doesn't need to be GPL'd to be open source.

That's what happened to my current company. We used a piece-of-crap middleware product called ISIS. The company that produced it went out of business. Luckily, we had the source so we were able to maintain it ourselves until we converted over to a different middleware package. The code was crap, too, but if we didn't have it at all we would have been in real trouble.

jeff
Monday, September 02, 2002

>> but as far as I know  LINUX uses POSIX, which is probably an adequate API but hardly bleeding edge.

>> The open-source guys complain that microsoft isn't innovative, but why the hell are they using an API from the 1980's.

Yes, Linux uses POSIX - no excuses.  Bleeding edge?  OK, look.  For all of you programmers out there that never ventured outside of Visual Basic, it is time you broadened your horizons.  Go buy the book "Advanced Windows Programming", the author is _Richter_.  You will learn that buried deep down, under the .NET, under the COM, under the BS du jour, under the Window Manager, lies a very well designed POSIX-like Win32 operating system.  It has events, semaphores, message queues, file io, shared memory, device drivers, virtual memory, shared memory, and on and on... Win32 and POSIX are functionally equivalent (for the most part - I can think of one feature Win32 offers that POSIX does not: the "heap" object - that would be a nice feature to have).

>> How about using components to build your operating system API, or a functional language based interface, etc.

What does that mean?  You mean you want all OS system calls to be COM?  Simply put: just because something is different doesn't mean it is better.  POSIX is perfectly functional as it is, (as is Win32), why would anyone change it without a demonstrated performance or feature advantage?  Yeah, I know, MSFT does this all the time.  Thankfully, they are rather unique in this manner.

Quit being an MSFT co-dependent.  Just because MSFT labels something a technology, doesn't mean that it is:
. better
. faster
. innovative

Do they just lead you by the nose, and you willing follow?  That alone would be a good argument for Open Source.  Put down the Visual and the Basic and expand your horizons.

Nat Ersoz
Monday, September 02, 2002

"Do they just lead you by the nose, and you willing follow? That alone would be a good argument for Open Source. Put down the Visual and the Basic and expand your horizons. "

Given the cult surrounding open source, when it's really not that interesting or important by itself, the same arguement applies equally well against Open Source.

Robert Moir
Tuesday, September 03, 2002

Something intersting about Open Source is you can learn from it (if it's good code, or see the mistakes if it's not so great). From Delphi 1.0 until at least Delphi 4 (I don't know if it's true now), you got 95% of the library source code.

And you could really see the simplicity and the elegant design created by the Delphi developers (Anders Heilsberg in particular).

It was for me a very valuable learning experience.

(Same thing apply also to the Java JDK sources and Jakarta projects for instance)

Robert Chevallier
Tuesday, September 03, 2002

I guess in theory there is nothing that prevents open source projects from creating nice, well documented and maintained API's. In practice the "use the source, Luke" attitude may well hinder this development. Open source is the extreme in white box reuse. Since all the bits of the code are clearly visible, there is no clean dependency border maintained.

This situation invariably leads to: "I build code that depends on certain implementation details of your code since I can see how your code works and it saves me so much effort just grabbing the handle to that struct directly instead of going through your tedious methods".

A good API is like a contract between the code underneath and its clients. You use the API and I certify that future versions of my code will still satisfy this API. There has been ongoing research into the domain of reuse contracts to give us better and better ways of specifying this contract (beyond the .h file). Information hiding is a crucial part of the reuse strategy. "I am not going to give you this information since it is not static and you should not depend on it. I might even change it in the next build". It spares me the dilemma of having to choose later on:
"I never said that I would keep on using the fooTrees to store the fudBars. It is very bad both for predictable runtime characteristics, and I just coded it that way in version 1 to have a temporary solution while I concentrated on completeness of the API. Now your successful Gigacrusher directly grabs the fudBar children from the node, instead of calling getRelated(). Should I do a decent version 2 and break Gigacrusher (and 13 less cool but decent applications), or stick with those horrible fooTrees?"

Yes, I do agree that in last resort situations like "it does not work and they don't fix it" or "they went bust, so they can't fix it anymore", having the source comes in mighty handy. Maybe you should consider not working with no. 1, and asset acquisition of the rights to the source in a bankruptcy situation could help in problem nr. 2.
Using sourcecode with a badly specified reuse contract is an implicit future fork of that code that you will have to maintain yourself.

Just me (Sir to you)
Tuesday, September 03, 2002

"I don't want to look at the source code for your operating system, database server, etc"

*whoosh* (sound of the point of Open Source going over nojuan's head).

With OS you don't have to. You can, but you don'r have to. Just because you do not want to does not mean that nobody else should be able to.


Tuesday, September 03, 2002

ach.

s/be able to/not be able to/


Tuesday, September 03, 2002

nojuan, interesting criticism, but be nice to have examples.

Rob Pike on POSIX compliance in Plan9:
"To be a viable computer system, one must honor a huge list of large, and often changing, standards: TCP/IP, HTTP, HTML, XML, CORBA, Unicode, POSIX, NFS, SMB, MIME, POP, IMAP, X, ...  A huge amount of work, but if you don't honor the standards you're marginalized.  Estimate that 90-­95% of the work in Plan 9 was directly or indirectly to honor externally imposed standards."
http://www.cs.bell-labs.com/who/rob/utah2000.pdf

Sammy
Tuesday, September 03, 2002

BTW, I feel bad when Kernighan (I believe) went nuts because he lobbied hard against the lawyers to get Plan9 opensourced, but people continued to criticize him harshly for it not being open.  There's just so much bad blood because all the Slashbot crusaders went militant during the dot-com boom, when every stupid idea sounded realistic.

Now it's out of fashion to do this, but I think the opensource community deserves a lot of the backlash it's getting.  Even if overall the Slashdot gestapo did more good than harm...

Sammy (posts too much on opensource topics)
Tuesday, September 03, 2002

nojuan,
You are making a broad swipe implying that opening your source code equals a kludge api or even none at all. You argue that POSIX is a good example but immediately disqualify your argument by saying you dont have any experience with it? Is your argument against Linux? Because Linux does not represent open source as a whole. Open Source simply allows developers to view the source code, so I can't see how that means you can't use an API or that it automatically makes the API a kludge. I bet I could find just as many exaples of good open source APIs as you could point out one's that you deem bad design. Nobody is sticking a gun to your head telling you to use the source code, it's just available if you want to use it.

Ian Stallings
Tuesday, September 03, 2002

Just for the record, I don't program in VB (not that there's anything wrong with that).  I do systems-level stuff on Windows/sometimes AIX, working with; threads, critical sections, etc... so I am intimately familiar with the POSIX-like underpinning of Windows (BTW - I think the big thing missing from POSIX is I/O completion ports).  It's all nice and wonderful, works like it's supposed to but it's not terrible interesting or new.  Didn't LINUX start out as Linus's something interesting to do project.  Well why not something more interesting than POSIX.  I'm sure it didn't/doesn't matter whether Linux succeeds in the market, so why the heck the same old shite.

My point was not that you HAVE to look at the source code, but the fact that the source code is there discourages good API/contract/whatever you want to call it design.  Here's an example, Some years ago I wanted to add some functionality to SendMail:

So I ask, where's the API to extend the system? 
There is no API. 
Why Not? 
Because you have the source-code, you don't need an API. 
But I don't want to go through one-million lines of code, figure out what's going on, determine where I should add my code, etc.
You Microsoft lovers are all the same...

Maybe the situation has changed since then, but that was experience anyways.  BTW - I agree having the source code is very useful as a learning tool.

nojuan
Tuesday, September 03, 2002

Sir (riding in on a seemingly white horse of contract laden object oriented API's) mentioned:

>> A good API is like a contract between the code underneath and its clients. You use the API and I certify that future versions of my code will still satisfy this API.  Yadda...

OK, "Sir", do tell.  What open source component do you have a problem with?  What peice of the puzzle does not live up to what Martin Fowler considers "Public vs. Published" interfaces.  (see http://martinfowler.com/articles/published.pdf)?

Nat Ersoz
Tuesday, September 03, 2002

We had a problem like this in a news processing system. The software is not "open source" in the sense that it is hosted on SourceForge, but since this is an effort of several dev. teams spread over more than one company, the setting will do. A component relied upon the internal ("unpublished" in your articles sense) structure of another component. When the structure changed later on, a crucial functionality in the app failed completely.
As it turns out the reason for this was that the developer that made the error had used the structure since it was crunch time before a release, and fully intended to clean it up after the release, but then never got round to it.
If he had not had access to the source, would this have happened? Who knows? People do stupid things like relying on unpublished, undocumented interfaces in Windows and then complaining their app breaks in the next release. And there they do not have access to the source.
Maybe it is just a personality thing. Me, even if I did put up warning signs in glowing neon saying "UNPUBLISHED: DO NOT RELY ON THIS, THIS WILL CHANGE!" and some moron goes ahead and builds on it anyway, I still do not enjoy breaking that code.

Yada ... http://citeseer.nj.nec.com/cs?q=reuse+contract&cs=1

Just me (Sir to you)
Tuesday, September 03, 2002

Well then, it appears that poor documentation and the breaking of contracts is not a symptom of open source projects at all.  In fact, some of the nicest documentation around is applicable to open source:

http://www.xml.com/ldd/chapter/book/bookindexpdf.html

Compare this kernel reference against the Windows "Baker" book for device driver development - the Linux version is far more comprehensive and readable.

I am sure that there are some open source projects that suffer from constantly changing interfaces (notably, Mozilla, from what I'm told).  But that is certainly the exception, not the rule.  Mostly, a working Linux distibution is built upon existing API standards and the Linux, GNU, X, Qt, ... are usually very well done.  Never mind that the entire O'Reilly selection usually applies for application development.

Just because you have the source, doesn't mean you necessarily need it.

Nat Ersoz
Tuesday, September 03, 2002

nojuan: well, sendmail sucks ass. I would never have expected a system that unleashed sendmail.cf on an unsuspecting world to do anybody any favours.

But since we're sharing anecdotal evidence, Apache provides an API [1] which has enabled useful extensions such as the ubiquitous mod_perl and mod_php, as well as inane (but easy to write) hacks like mod_msff [2].

1: http://httpd.apache.org/docs/misc/API.html
2: http://unlikely.org/mike/hacks/

Daryl Oidy
Thursday, September 05, 2002

This has to be the weakest argument against open source I have seen in a long time. You seem to think open source APIs have poor interfaces because they let you see the source of the API and you could code around them.I suppose you [nojuan]  think that closed source APIs tend to be good because if they are not they will not be used. I will tell you from experience that the open source community will not subject themselves to poor interfaces.

People will use whats good and either ignore or improve whats bad, that is the power of opensource.

Mark
Saturday, September 07, 2002

*  Recent Topics

*  Fog Creek Home