Fog Creek Software
Discussion Board




Upgradable API standards?

I need to start a new project that will have an plugin API.  The API needs to be upgradable without breaking older plugins (eg, as we think of new things to add we need to be able to add features to the API)

I've seen the COM way (I guess this is the COM way), where you ask for a pointer to a list of functions by name and version.  That way older plugins will ask for the 1.0 list of functions where as a newer plugin can ask for the 2.0 list of functions.

Are there better ways?  Any other styles or systems I should check out before I jump into this?

Gregg Tavares
Sunday, June 27, 2004

Overview of plugin systems is on:
http://xplc.sourceforge.net/doc/others.php

Pavel
Sunday, June 27, 2004

Why not just make the plugin programmers recompile their plugins when there's a new version released?

(Depends on your target audience, I suppose, and frequency of releases. It might sound like a dumb idea, but Alias Wavefront do this with Maya, and there are probably other examples too.)

Tom
Sunday, June 27, 2004


Read about interfaces versioning in COM

Max Belugin (http://belugin.newmail.ru)
Sunday, June 27, 2004

"Why not just make the plugin programmers recompile their plugins when there's a new version released?"

Please... no, don't.

Is there anyone other than Tom that thinks that is a good idea? I certainly hope not.

Dennis Atkins
Sunday, June 27, 2004

Bad idea.

Brad Wilson (dotnetguy.techieswithcats.com)
Sunday, June 27, 2004

It might sound really bad, but I reckon it's not. As always, it's a boring trade-off. How often will you release new versions? How much effort will you expend getting all this versioning stuff working? How many new bugs will you introduce?

You'll need some shim code here and there too, as even your structs' layouts will have to stay the same across versions. Will you enjoy that?! Or maybe you are intending to force your plugin programmers to query for god knows how many different types of interface to handle the objects you're exposing? Do you think they will like that?

But, like I said, it depends on the project. My thinking is just that, if you're asking about different schemes for this kind of thing, you must not be sure about the one you're intending to use at the moment. And if there's some question about that, it must be something slightly complicated. So, if you can get around the complexity by not doing it at all, you may wish to consider that instead. And forcing people to recompile their plugins will remove the need for binary-level compatibility, leaving you only having to maintain the rather easier source-level variety, without pissing the plugin programmers off too much.

(Or maybe it would piss them off. I think I mentioned, it depends on the project :)

Tom
Sunday, June 27, 2004

Let's give an example of why this is bad:

Up until Oracle 8 there would be an ODBC driver called 'Oracle ODBC driver'. So for a few years our software and setup program (which are DSN-less) have been using this 'fact' to be able to connect to an Oracle database.

Now here comes Oracle 9 and suddenly the Oracle ODBC driver is called 'Oracle on ORAHOME92' or something like that. This makes all third-party software (including ours) that depends on this 'fact' go haywire.

I know, people are going to respond that we should have use a DSN, but we happen to have a huge dislike towards the Windows registry. So sue us...

The point I'm trying to get across is: How do you know up front how many programs that interact with your software are going to need to adjust to your changes ?

You don't !

And this brings us back to Joel's discussion about how MS lost the API wars. (MS isn't the only one...)

Eric Hop
Sunday, June 27, 2004

What possible advantage could there be in forcing people who use your api recompile every time you release a new version?

Aaron F Stanton
Monday, June 28, 2004

Well, it's awful convenient for *you*, to avoid the versioning scheme problem and the backward compatibility problem and so on.  Problem, of course, is that it's not at all convenient for *them*, and you probably have neither the influence to convince them nor the ability to recompile their code for them.

Mikayla
Monday, June 28, 2004

So it's pretty much ok if you're the only one using the api, or if you have massive leverage on the end user (which I suppose includes you if you're the only one using it).

Sounds like it's a really bad idea if you want to get anyone new to try to use it.

Aaron F Stanton
Monday, June 28, 2004

It's not too hard. First, use API functions as opposed to API data structures as much as possible. e.g.

int action1();
int action2();
int action3();

as opposed to

struct actionarg {
    int do_action1;
    int do_action2;
    int do_action3;
};

int do_actions(struct actionarg*);

This is because it's a lot easier to ensure binary-level compatibility of function signatures than complicated data structures. If you use data structures, you'll need a field that indicates the version, plus code to translate older versions to newer versions (Windows uses this with all the .cbSize members).

Second, you have to maintain compatibility of the function API. What approach you take depends on how much the API is going to change. If you are only going to add a few new functions here and there, you probably don't have to do anything. If you want to change function behavior a little, you'll need to have a different name for the new function (like all of the FooEx() Windows API functions). If you want to change things a LOT, then I suggest entirely separate APIs for each version.

C++ is dangerous here, because it's not always clear when a given change is going to break binary compatibility. e.g. making a member function virtual may or may not break compatibility depending on the compiler. (btw, in C++ land you have to worry about the compiler's own ABI versioning; on some platforms every new compiler version breaks binary compatibility for you).

One approach in C++ is to make plugins derive from a main plugin class, with virtual functions to implement the guts of the function. This works pretty well as long as you are very careful of the ABI issues.

btw, I don't consider Maya a model for good practices in plugin programming. Look at Lightwave 3D - plugins written for the 1997 version of their software still work fine today. They do it by breaking the entire API into smaller modules which are versioned separately, and there are no direct function calls into the API; you first call a single "setup" function that returns you some function pointers, and the setup function is informed of the API version so it can give you the right ones.

For a good example of C++ API continuity, see the TrollTech QT libraries.

Dan Maas
Monday, June 28, 2004

[What possible advantage could there be in forcing people who use your api recompile every time you release a new version?]

* When a plugin doesn't work any more, you'll be able to persuade the end users to blame the plugin author, instead of you.
* Plugins won't fail in mysterious ways due to incorrect assumptions about the API which are no longer true, requiring you to maintain plugin(-version)-specific workarounds in your application *forever* (lest users blame you instead of the plugin author's for the mistake), and/or to smash a network card with a hammer. [1]

Me? Cynical?

[1] http://weblogs.asp.net/oldnewthing/archive/2004/02/11/71307.aspx

A cynic
Tuesday, June 29, 2004

*  Recent Topics

*  Fog Creek Home