Fog Creek Software
Discussion Board

Server protocols

I'm interested in people's opinions of different technologies for implementing the client-server protocol for a database type server.

So if you were implementing your favourite RDBMS (and client) today what would you use to get them to talk?  To restrict the scope a bit, lets say Windows only OK and no Java technologies.

As I see it the main contenders would be (in no particular order):
- Microsoft RPC
- .NET remoting
- SOAP over HTTP
- 'roll-your-own' directly over sockets
- a 'BEEP' like toolkit (

Does any one have any recommendations or horror stories of using these in a high volume, highly concurrent environment?


Rob Walker
Friday, September 12, 2003

I have no plans for developing my own RDBMS, and I don't know if anyone else on this forum does. I do know that if you really think about it you'd keep it abstract so that you can later optimize it for different development platforms.

Dave Van den Eynde
Friday, September 12, 2003

If I would do an RDBMS type application, depending on the complexity needed I would check out an open source database server protocol. Take a peek at how PostgreSQL implements its server protocol for example.

Without having looked at it, it is probably some lean mean socket homebrew protocol, optimized for high volume data transfers and throughput. Minimize the overhead.

The "minimize overhead"-goal should disqualify DCOM and the like. I wouldnt even touch CORBA.

Friday, September 12, 2003

I agree with Patrik, except that I'd also look at the available references for the standard commercial databases, too.

MS & Sybase use the "Tabular Data Stream Protocol" and references for it can be found through google. There is even an open-source version of TSD available at

Friday, September 12, 2003

Or just straight HTTP.  Depends on your performance requirements really.  Binary protocols such as TDS have significant performance advantages over most of the protocols you mentioned.

christopher baus
Sunday, September 14, 2003

Creating a binary protocol socket based protocol is not much harder than to create one that uses HTTP, since all the socket-stuff is needed for HTTP requests as well.

In my experience, designing something that you know is limited is far from optimal, given the extra cost of doing it properly is small.

Monday, September 15, 2003

Thanks for your comments.  I wasn't aware of the TDS protocol, and that makes for interesting reading.

Whilst rolling a custom implementation might give the best performance, it is also not surprisingly the most work, and offer plenty of scope for subtle network related bugs that only show up under heavy load.

I'd like to leverage some existing infrastructure where possible, for example:
* asynchronous requests
* events
* ability to cleanly interrupt running queries, or result sets being returned
* security support

Whilst existing products do seem to use custom protocols is that still the best choice today if you don't have that legacy (and if network performance is not likely to be the bottleneck).

Rob Walker
Monday, September 15, 2003

*  Recent Topics

*  Fog Creek Home