Fog Creek Software
Discussion Board

Runtime software evolution

I am a PhD-student, interested in runtime software evolution, i.e. performing updates to programs while they are running. This is of relevant for software systems with very high availability requirements, for example in phone switches.

Since JoS seems to be visited by a lot of experienced software engineers, it would be fun to hear about your experiences from updating software systems with high availability requirements. How do you handle updating in practice? Is there any kind of technology that is of help?

Jens Gustavsson
Friday, March 7, 2003

With the QNX operating system, you can stop/unload any device driver or process at any time, update the file, and start it up again.  The system picks up where it left off and all is fine.  I've heard of QNX systems running for years without ever having to shutdown for an update.

The windows world on the other hand has gotten used to rebooting after almost any update.

Friday, March 7, 2003


Despite other failings Java provides good language-level support for runtime code replacement. Dynamic class loading 'class.forName()' and the ability to tinker with the class loading mechanism itself make for a clean way to update applications without stopping the VM.

Of course pretty much any language with support for eval() will let you do the same thing,  but the mechanism in Java seems well thought out exposed to the developer.

Seems that this type of code replacement in compiled languages would be a lot harder.


Friday, March 7, 2003

>Seems that this type of code replacement in compiled
>languages would be a lot harder.

Harder, yes, but not impossible.  In C++, using LoadLibrary() in combination with class inheritance will yield much the same result.  That is, any behavior that you anticpate changing at runtime gets handled through a base interface that can be implemented in multiple ways, and you can choose the particular implementation to use dynamically.

This requires careful planning, but so does the Java way, it seems to me.

Michael Eisenberg
Friday, March 7, 2003

Heh - those $155 million dollar architecture astronauts at Groove Networks have done this in a manner that allows a user to dowlnoad apps from seveal third parties across the web and integrate them into a collaborating whole new application customised for the job the user had in mind!

Oh, and check for updates as well!

Friday, March 7, 2003

Reliability in telecom systems is often achieved by distributing the software among two or more identical hardware components, any of which can be booted without affecting the operation of the application.  So, you simply install the updates on one component at a time until the entire system is done.  My experience is with telecom databases, where this is a good strategy.  With switches, you might have more hot-plugging of modules going on.  I'm not going out on a limb to say that, in practice, even the simplest update strategy is hard to get right!  Good back-out procedures are important, and often overlooked.

Rob Daugherty
Friday, March 7, 2003

.Net has great support for this with app domains. You can drag a plugin into a directory and it automatically becomes available, or drag it out and its gone. Very nice...

Webservices are another thing that are somewhat related, and perhaps the next step architecturally...

Robin Debreuil
Saturday, March 8, 2003

I went for a fairly extreme approach when I was working on a research project. I developed a binary compatibility checker for Java (based on the language specification) that would guarantee that two versions of a library would link and execute correctly with existing classes. I added in a plugin architecture to allow the switching of libraries at runtime.

With Java 1.4 you can take advantage of the debugging architecture that allows you to change the JVM class definitions on the fly, the app wouldn't need to make use of any plugin architecture to be upgraded on the fly.

AFAIK this sort of stuff in the research stage and there isn't any commercial product that does this so it probably isn't that useful to you.

Miles Barr
Tuesday, March 11, 2003

Thanks for all the answers.

I am aware of several technologies similar to the ones you mention. Actually, we are currently developing a research prototype for applying dynamic patches to Java programs using the mechanisms in the Java debugger architecture. We will use it for investigating tools and methods to support runtime software changes.

The intention of my original question was to get information on how the problems of updating high availability systems are handled in practise. I guess that many problems are solved by administrative routines rather than technology solutions, but I am not sure of this. Those of you who have used some technology solution, were there any problems with them? How did you assure that the updates did not introduce errors? Examples and anecdotes about real world update scenarios are of great interest to me.

Jens Gustavsson
Tuesday, March 11, 2003

I used to be one of the architects in the Nortel Meridian 1 PBX project between 1991-1995.  Indeed live updates of high-reliability systems was one of the issues that we dealt with.  We had a dual-processor system with one in "hot" and one in "standby" status.  During upgrade, the standby would get Flashed with new firmware and "switchover" would occur during completion.

Additionally, there were memory protection issues which may or may not apply in traditional desktop systems.  I think Microsoft may have introduced code memory protection in Windows XP recently.  There is a whole range of interesting real-life computing problems that realtime embedded programming have to deal with that traditional desktop programmers don't have to face.

Hoang Do
Wednesday, March 12, 2003

*  Recent Topics

*  Fog Creek Home