Fog Creek Software
g
Discussion Board




Sincere question about open source

What is the philosophy regarding change control in the open source community?

What I mean - there is the low-level mantra regarding open source of "if you don't like it, you have the source code and can change it yourself."

But doesn't that fork the tree? If you put the effort into customizing the Linux 2.4 kernel, how did you deal with the 2.6 release? Is there a merging philosophy? Do you get the changelog and walk through making the changes yourself?

Or are you not supposed to actually hack the kernel yourself, but rather make changes via libraries and API's?

Just wondering,
Philo

Philo
Saturday, February 14, 2004

The policy, as far as I've observed is this:

The kernel is a special case, because it warrants changes that can't really be modular (modules can't do everything, can they?). For instance, every hard-realtime patch I'm aware of is a patch; a fork. That's the nature of the kernel; it's kept pretty 'pure'.

In general, the philosophy seems to be a good bit of the 'you can make your customizations yourself'. This hardly ever forks projects, because often patches are accepted. For instance, I wrote a frontend to the mplayer program a while back, and I wanted a few more slave-mode commands. So I wrote them, sent in a patch, and now they're in the CVS mplayer. Patches are always accepted with goodwill, even if the maintainer decides not to apply it. Forks are extremely rare, and even rarer is when a fork doesn't take over the project (elinks feels very much to have replaced links for instance).

I also have been getting a sense of greater modularity than commercial programs (I may be very wrong here, though). A huge number of big projects have plugin APIs.

Mike Swieton
Saturday, February 14, 2004

You know I was wondering the same thing, glad you brought it up.

Alex.ro
Saturday, February 14, 2004

Yes, I'd estimate that 95% of changes are accommodated by users finding a bug or feature opportunity, either the user or the developer writing a patch, and the patch getting applied. The remaining 5% become forks (whether for technical or personal reasons).

I suspect there are quite a few developers who customize source code here and there without sending in patches. There's nothing legally wrong with this (unless the code is GPL or LGPL and goes into a product), but it does lead to duplication of effort.

The increased modularity of the open source world is both a blessing and a curse. On one hand it makes it easier to coordinate a herd of volunteer developers, on the other hand it leads to fragmentation and fifty ways of doing the same thing. (how many audio players are there for Gnome and KDE again??)

Dan Maas
Saturday, February 14, 2004

I just want to clarify that for someone coming from the commercial software world, getting a patch applied may be an utterly alien experience. As far as I know Microsoft and Adobe don't accept random code fragments submitted by unknown individuals. But it's very common in open source for developers to take almost any improvement even from a total outsider.

Dan Maas
Saturday, February 14, 2004

There's no 'one way'.  But generally, you submit a patch to the maintainers. 


Saturday, February 14, 2004

Philo, with regard to the phrase, "if you don't like something, you have the source and can change it" - the comment notes on several possilibities. 

A user may find a bug or missing feature, code it and release it as a fork.  This is generally considered to be a "bad thing", as forking creates confusion and often splits programmer effort with little benefit.

Alternatively the patch may be coded and submitted to the project and accepted as part of the core. 

Or it may be identified as a patch for those who want the specific abilities provided by that patch or set of patches.  The Linux kernal has several patch sets providing alternative VM systems, real-time extensions, etc that do not necessarily need to be in the core.

Submitting a patch is typically seen as the largest benefit of open source software.  If the user sees a missing feature or bug it can be coded and applied - either locally (at first) or potentially globally (if the community and/or maintainer(s) deem it worthy/necessary).

Lou
Saturday, February 14, 2004

The difference between the proprietary closed-source development model and the open-source model is really a cultural difference.

Adobe (say), won't accept a patch from an unknown outsider, because they do not publish their code -- so how would one write a patch? A company may publish a plug-in API, but this doesn't let you fix or modify the core of their app.

In the open-source world, everything is set up so that it is easy for anyone (who can code, write documentation, etc.) to contribute. There is probably a public CVS server, there will be web-based code documentation (usually automatically-generated using something like Doxygen), there may be a dashboard showing the status of unit tests on recent builds (these may be run automatically on a regular basis), and there will also be a mailing list. Documentation should exist to help newcomers.

If there is functionality that one would like to see added, the above mechanisms serve to help the developer get into the project, do the coding and submit their contribution. Often this will involve dialogue on the mailing list, so the project leader(s) will be aware of the work being done and can advise on how best to do it. Sometimes the proposal will run orthogonally to the project leader(s) overall aims for the project, and this is when a project may get forked. Otherwise, the patches will be accepted, evaluated, tested, and perhaps rolled into the main branch of the project, to be released at a future date.

C Rose
Saturday, February 14, 2004

"But doesn't that fork the tree?"

well, yesish, obviously there are different degrees of changes.
for instance, if I want I can simply grab the linux source and make changes in my version, possibly distributing that around my company as the 'blessed' version.
If I do that then obviously Im responsible for updating the changes to the latest Linux source as it comes out, and clearly on the whole its going to be a PITA for me to maintain.
beats me why anyone would want to do it, but the possibility is there if we need it.

OTOH I can create a 'patch' that alters the source code, and I can release that patch into the Linux community..the idea being that its merged with the actual linux source by anyone who is interested in doing so.
If people want to use it they can, if they want to ignore it they can, on the whole if a bunch of people like the patch and begin using it, its prolly going to be passed up the chain until either it stops being liked and used, or the maintainer decides to enter it into the 'main' branch and it officially becomes a part of the source for good.

At any given time there are a heap of patches for Linux floating around the ether, some are often used, some are not, some will go on to become part of the main source, some wont...it all depends on how motivated the creator is to keep it up to date, and how much other people like it and use it.

"If you put the effort into customizing the Linux 2.4 kernel, how did you deal with the 2.6 release?"

<shrug> thats up to you...presumably some people just move the changes over manually, some automate the changeover, some persuade the maintainer of the tree they are using to adopt it into the source.
Redhat for instance maintains a separate tree from the Linus tree, and they accept patches into that more quickly if they drive the source to where they want to go....overtime Redhat will push the patches over to Linus via their maintainer in the hope that Linus will agree on its usefulness/importance/popularity etc and move it into his tree.


"Is there a merging philosophy? Do you get the changelog and walk through making the changes yourself?"
yes, and sometimes :)

I can decide to use nothing but the tree that Linus maintains, or I can maintain my own tree, moving across patches that I like myself, or I can decide to stick with a tree maintained by someone else, presumably because I like the decisions they make regarding patches.

overtime patches that are 'good' and much praised tend to find their way up the hierarchy into the main tree, while patches that either dont solve a useful problem or that are badly implemented and cause problems tend to wither and die :)

evolution at its best IMO.

FullNameRequired
Saturday, February 14, 2004

What a bunch of wankers you open source followers are.

If I want some feature on Windows, I can write an application or library that does it. That's the whole point of Windows.

a
Saturday, February 14, 2004

" I can write an application or library that does it. That's the whole point of Windows."

so?  how does that make us wankers?  you wanto use windows, go use windows. 
I develop a _lot_ of closed source stuff for windows and mac, thats my business.
I also have a real interest in open source software because I rather like most of its tenets.

No one is stopping you from continuing to use windows, no one wants to stop you from using windows, no one cares if you do use windows.
Personally, I like osx, philo prefers windows and Ive no doubt someone on this forum swears by Linux...there is no reason all those preferences cannot coexist, and these days theres no reason I cannot prefer to use one while developing for another.
(love wxWindows and REALbasic <g>)

FullNameRequired
Saturday, February 14, 2004

Dont feed the troll..

With regards to the above question though>

Libs are rarley forked. Most applications developed for linux (afaict) have a very clear cut separation between user interface and core functionality. So, if some one is unhappy with the functionality of AppXYZ, they would probably make use of LibXYZ and collaborate on that, and then add their own LibXYZ_super++ for their own application and tuck their own gui ontop of it.
So, what appears to be an abundance of apps with very simmilar functinality may in reality be just variations on the same app.
I noticed this personally when I switched from gnome to KDE recently. I swapped out almost every app, but alot of the libs are still the same.
The downside of this is that dependancies to become quite complex.

Eric Debois
Saturday, February 14, 2004

Some projects are happy with forks some aren't.

From what I've read Linus Thorvalds is quite happy for people to fork the kernel.  I read somewhere him quoted as saying that was (in his view) the great benefit of the GPL - it meant that forks could be always be legally remerged if they proved useful. 

I believe that at one stage there were several high profile trees with Alan Cox's tending to merge patches that were being considered for the main (i.e. Linus's) tree.  I also believe most commercial distributions use their own tree - but track the "main" tree reasonably closely.  When Linus considers development work complete on a version he hands over to someone else as maintainer. 

As to merging philosophy on the main tree - I think it boils down to if Linus thinks it's stable, from a reasonably trustworthy developer, useful and will improve the kernal he merges it.  If he doesn't, you  can start your own tree and you get a chance to prove him wrong.

If you're interested most of the development work takes place on the kernel mailing list (see http://www.linux.org ) and there are various sites which track development (eg http://kerneltrap.org ). 

a cynic writes...
Saturday, February 14, 2004

I've always thought Open Source was kinda' forked up. :)

sgf
Saturday, February 14, 2004

Serious open source projects are very tightly managed, as much as closed source commercial development teams are,  they have a core group of responsible engineers that validate submissions.  Sometimes that validation is brain dead and they want to stifle development in a particular direction.  If the project is useful enough then you'll get a fork that matters.

There's two broad philosophies in relation to source control and managing the corpus.  One says the tip shall always build and function, though versions may branch off for experimentation, the tip is always sane and woe betide anyone that breaks the tip.

The other philosophy is that the tip is where we are today and the branches are those frozen releases in the past, or they're branched in order to freeze.  The tip is meant to build, but it might actually be rubbish.

I prefer the former philosophy it makes more sense to me.

When I was peripherally involved in Mozilla and offering fixes they seemed to use the latter method.  I'm still convinced that half the stop-go progress of the development in the middle years was because of that, that and the relative immaturity of some of those managing the project.

Simon Lucy
Sunday, February 15, 2004

It's often far easier to fix your problem on your specific configuration than it is to solve the general case. You're free to solve the problem in the quickest possible manner, even in ways which would be inappropriate to distribute.

Robert Sayre
Sunday, February 15, 2004

At a previous employer we were deployng an alpha build of samba to replace the aging NT4 PDC.  However there was a problem with the release at that point,  when configured as we required it, the server was incorrectly comparing DOMAIN\user to user in one module.

Now, we whipped up a quick patch for that module which did the obvious and removed the domain and the \ from the value before searching.  This wasn't the best solution, however it worked for us.  We submitted a bugreport and attached our 'patch' to it.

One of the samba maintainers picked up the patch, saw it was ugly, dug around and fixed the actual problem.  We continued to use our 'patched' version until the next release of the samba code in a months' time.

So, our patch was a short-term solution to a real bug, and bore no relation to the final fix.  However thanks to our having the source we could fix it and move on.

Koz
Sunday, February 15, 2004

Philo, I'm surprised no one has brought this up, so I guess I'll have to be the one. If you're interested in what *CAN* happen in the OS community regarding "fix it yourself" vesus "but it's our stuff," look into the history of Emacs versus XEmacs.  Here's a link to one person's account of the story:

http://www.jwz.org/doc/lemacs.html

Andrew Burton
Sunday, February 15, 2004

"Here's a link to one person's account of the story"

yeah, its a _good_ story that.

It really highlights the strengths of OSS IMO, in cases where no agreement can be reached on agendas, _everyone_ still has reasonable alternatives.

FullNameRequired
Sunday, February 15, 2004

*  Recent Topics

*  Fog Creek Home