Fog Creek Software
Discussion Board

Shrink-wrapped architecture

Hi All,

We're a small company, currently selling a (relatively) small shrink-wrapped stand-alone product. It has a database backend, a GUI frontend, and some fancy-dancy stuff in between the two. It comes on a CD, and has a very simple installation procedure.

Recently we've been looking at several improvements, such as limited integration with third-party applications on the users machine, distributing read-only data views throughout the client's office, and possibly monitoring things from our office (only with the client's permission, of course). We've examined various options, such as creating ActiveX controls that can be embedded in these third-party apps.

I have a few requirements;

1. Installation must be a breeze, and can be done by a non-IT guru. If at any time the client is required to reboot some SQL server somewhere, it's a failure (in my mind).

2. Must be seamless. The CEO of the client company should be able to wander into the CFO's office, quickly pull up one of our modules, and say "That's the data I was talking about."

3. Uses existing standards. Whether it be COM, .NET, Corba (gulp), or whatever, I want to be able to give a simple interface to a third-party company, and tell them that's all they have to meet to work with us.

4. Straightforward to implement. I'd rather not send guys on courses for a couple weeks, and I don't want to become a networking wizard to be able to use it. On site, I want the framework to be able to handle network failures,  and I don't want to be writing scripts to handle complex client-server architectures.

In short, I want to hand the client two cd's (after the cheque clears ;-). The first is for installing the main unit with the database, and the second is for installing satellite units. Installation takes a couple clicks, no fuss, no muss. Our clients all use Windows, but we've been keeping an eye towards cross-platformness.

Am I dreaming, or is this possible? Can anyone point me towards a framework that might meet my needs. Are there any good references (dead tree or on-line) that would help me out?

In short, I want to create a strongly networked product, with a simple installation procedure, without becoming a networking company.

ps. I'm not quite as ignorant as I may be protraying myself to be, but I want to be careful not to lead you kind folks in the wrong direction.

Monday, March 3, 2003

Could you state, concisely, what it is you are trying to accomplish?  I am having a hard time following you.  Then again maybe it's just me.

Bic Pen
Monday, March 3, 2003

Yes, I am having a little trouble making this clear without using a lot of buzzwords. I want a framework that we can use across a client's network, 'middleware' perhaps, that easily lets us communicate between our modules.

I've seen dozens of middleware applications that seem to be flexible, and do all the XML/Web connectivity etc. that would be nice, but from what I've seen they usually require sending someone to install it on the client side. I don't want to get into that.

Does that help, or am I digging myself deeper here?

Monday, March 3, 2003

Sorry, I'm still a tad lost.  I'm thinking you want a communications library that the user can install and that your application can use to communicate with itself across the clients network?  or that let's you communicate with your application remotely?  Sorry, read your post several times and still can't grasp it.  I must be really tired.

Bic Pen
Monday, March 3, 2003

How about both? I want it to be part of our installation, so we can be sure of versioning issues, etc. 

As far as us communicating with our clients installation, it's really not anything insidious. Our application is a scientific application, and at times extremely computationally intensive. I can foresee situations where the user might want to dump out some of their calculations to our faster workstations. Perhaps kind of a reverse SETI operation.

I feel like this post is slowly dragging down into wordy nonsense, but I want a framework where I can be sure in 6 months that I can add a new plug-in, or interact with a new accounting software package that a customer just installed, without worrying about complex server issues.

Does anyone know of any good books that might help me with these questions? Some kind of architecture study based primarily on shrink-wrapped software?

Monday, March 3, 2003

How about the Rodney Dangerfield of distributed protocols... RPC? (remote procedure calls)

RPC is implemented similar to CORBA, using IDL files to specify the caller/callee interfaces. COM is built on top of RPC.
RPC, as an overview, literally means "remote procedure calls". It allows you to issue C type function calls to server applications running at other workstations. And a program can be both a server and a client, given the proper stubs and declarations.

RPC meets some of your criteria handily - it's somewhat platform independent, having implementations for Unix as well as Win32 (caution - I can't vouch for the ability to use interplatform calls in real life); it avoids complexity in terms of registering components and assigning the correct COM programming models; all each end of the connection needs is a machine name or IP address.

The main shortcoming is that it is C specific so it doesn't support remote object method invocation and marshaling. If your functionality makes sense to cram into non object oriented procedural calls, you could do a lot worse.

Bored Bystander
Monday, March 3, 2003

No, you're not making things easy. It's a bit like a guessing game. What exactly does the product do?
From the little you give away I would assume you're using Access, Delphi or a proprietary ISAM backend and have some kind of data-oriented GUI app. Communication between the layers would be API in nature, and you want to know if you can give it long arms.
I suggest you forget about the questions of ease of installation until you tackle questions like:
a) can I do it at all?
b) is my app workable given the available bandwidth and latency constraints?
c) will the market still want it after I do the necessary?
It's a topic of interest to me, and we're in the shrink-wrap business. Feel free to email if you don't want to discuss in a public forum.

David Bennett
Monday, March 3, 2003

The question about RPC is: which one? There are two very different protocols which both use the name RPC.

ONC-RPC, also known as Sun RPC (since it came from Sun) is older, well supported on Unix systems (it's the foundation for NFS (Network File System)), and a real pain to get running on windows systems without shelling out cash. There was one free port of ONC-RPC to Win32, but the guy pulled the source and went commercial with it. The original Unix code is under a BSD-ish license if I remember correctly.

The other option is DCE-RPC. This *is* implemented in Win32 out of the box (in fact, DCE-RPC is the foundation protocol for DCOM), but you can't get the unix side without paying for the DCE suite.

DCE RPC is newer, and technically superior (it pays attention to things like authentication, which ONC-RPC does not). However, it's also more complex.

If you go the RPC route, I'd strongly suggest going the SOAP or XML-RPC route instead. It's much, much simpler to work with, the packets are human readable (a godsend for debugging) and there are tools for every environment under the sun. Heck, I even know a company that did a SOAP endpoint for PalmOS.

Chris Tavares
Monday, March 3, 2003

If I understand what you're looking for correctly, .NET may be the best way to go from both sides:  development and from the client perspective. 

Using DCOM for networking is a nightmare.  Writing your own sockets library would just be reinventing the wheel, and CORBA is just nasty IMHO. 

.NET offers a pretty simplistic and very flexible network communication infrastructure called .NET Remoting.  It can handle things like switching from a Binary formatter to HTTP/SOAP communication channels easily through an XML config file.  Different network object lifetimes, etc.  Conversion between XML and data objects, etc.  Much, much nicer and pleasant to work with after having dealt with DCOM and CORBA.

The only issue that may be problematic is that the client has to install the .NET runtime (~20 MB), although you can also distribute the runitme on the CD also.  But having the client install the runtime shouldn't be too painful.  I think you can also automate the runtime check and installation if it doesn't exist also.

I suggest looking up .NET Remoting, try out some of the sample code and applications to see if that may be what you're looking for.  The .NET team seems to have invested a lot of time to make communications on a network easier than DCOM was.

Tuesday, March 4, 2003

Sounds to me like the networking part contains two separate issues:
- the reporting side (CEO shows CFO a result of a calculation)
- the calculation outsourcing part (something like a renderfarm job, right?)

The reporting issue can be effectively handeled with a little webserver embedded in your app. Whaterver your platform these things exist and you do not have to write them yourself.
For the "calculation outsourcing part", use an HTTP based RPC mechanism such as web services if you can get away with it. Messing with any other form of RPC will not fly unless you have very complete control over every client network (especially firewall etc.).

Just me (Sir to you)
Tuesday, March 4, 2003

Excellent question here.

I understand your original question perfectly. Anyone who has ever designed a business application that needs to share data will instantly understand this requirement.

Fact is, that 10 years ago, you made a piece of software. Put it on a disk, and sent it out. The user installed the application. Done!!

Now, virtually every business with more than one pc, and that is just about every business I know, you have the requirements that Shrinky laid out.

In fact, I don’t know of ANY application I have written in the last 10 years that DID NOT have Shrinky requirements.

Thus, a data part of the application gets installed. A client part that allows each station to use that data is then installed on each pc. And yes, this should be as easy as installing Excel.

So, what do we often use?:

Well, for good deal of software the solution is of course using JET and a file share. Thus, your “data” disk installs and sets up a file share on the server. (in fact, the admin sets up a folder with full rights, and then installs the data disk into that dir).

Then, you simply install the client cd that really is just your standard software install (say VB, C++, or even the P&d for ms-access). Then all of the clients simply connect to that data set. (I wrote my own code that called the file open dialog (api) to allow the user(s) to browse, and then connect to that file share. This is easy, and only has to be done once. I do this now, and thus can simply send off two cd’s to a customer. The installing of the software is thus a snap.

The problem is that a file share is NOT good for any type of wan, or remote connections. Thus, the real debate here is do you want to use a file share, or are you willing to use a data engine on the server? (but, that training, and setup comes into play now. The customer might not even have a IT support guy).

So, while my solutions right now use a MDB and JET, I am also keen on what other people use to share their data on a network. I am guessing that Shriksy requests covers about 98% of any software package being developed that must share data.

So, yes, I also am looking for that obvious requirement for any piece of commercial software that needs to share data.

Anyone? Anything else besides a JET/mdb file share?

(in other words, what are you people using for your clients that solves this problem of a trouble free install that shares data?

Albert D. Kallal
Edmonton, Alberta Canada

Albert D. Kallal
Tuesday, March 4, 2003

Thanks Macarana. I've worked with CORBA before as well, and I don't want to repeat that experience.  I'm seriously considering COM for now, as .NET is not an option for various reasons. Could you give me some idea of what is involved in moving from COM to .NET Remoting? Is it a rewrite, or does it handle it well?

One idea I've looked at is putting an XML layer between the COM/Corba/whatever transport layer in an effort to simplify the transport interfaces. Moving from COM to SOAP (or whatever) in a couple years would then be easier, no?

Just Me, in my situation I don't want control over any network, so I would agree with you about the web services.

Albert, trust the Canadian to say the most ;-)

Good to know others have the same issues. In our case, there is usually an IT person on the client site, although sometimes it is a part-time consultant. One other requirement that our clients sometimes have is the desire to demo the product for a month or two. Few places would be willing to do a large install for a temporary project.

A file share of some kind would likely serve our purposes for now, but as you mention, I could see requiring remote access in a year or two. Any solution we use would have to at least have provisions for this.

Sign me,
A Frozen Ottawan

Tuesday, March 4, 2003

Like many others, we had a similar issue at my current company.  At first, we had an overly complicated IPC of both CORBA and DCOM (don't ask why).  When that continually caused problems, we punted them both.

We ended up going with SOAP over straight TCP/IP sockets.  There's nothing about SOAP that requires HTTP.  That way, we have a simple networking scheme, and we can use all the standard XML/SOAP tools out there to build and parse our packets.

The only catch is that if firewalls are involved, you have to have them handle your port (you'll have to pick some "random" port to use for your sockets).

BTW, this was took much less time to write from scratch than we spent integrating our expensive CORBA package.

Tuesday, March 4, 2003

One more thing about our IP/SOAP mix is that it's platform independent.  We run on both Windows and Solaris, but really we could port it to anything without any hassle.

And like others pointed out, it avoids requiring the 20MB .NET Framework on every client.

Tuesday, March 4, 2003

>>We ended up going with SOAP over straight TCP/IP sockets.  There's nothing about SOAP that requires HTTP

Hum, that is interesting. Does not this still require a web service of some type running on the server?

What is the min required on the sever side to do this? Can this be setup on a simple install disk for the client?

Albert D. Kallal
Edmonton, Alberta Canada

Albert D. Kallal
Tuesday, March 4, 2003

You only need a web server if you're using HTTP. What the previous poster was talking about was a simple socket. All you need on the server side is to open a listening socket, and on the client connect to it. This is fairly straightforward stuff.

Chris Tavares
Tuesday, March 4, 2003

What Chris said.  You don't need a Web Server at all.  SOAP doesn't require that it's sent over HTTP.

It's actually a pretty simple architecture to set up and it doesn't require you to integrate with any thirdparty software, other than maybe and XML and/or SOAP package.  We use Xerces in ours.

One other drawback with our solution is that we had to create and parse our own SOAP headers and stuff.  A very minor drawback, as it's only a few simple XML tags, but it's worth mentioning.

The big advantage for us is that it's platform and language independent.  We can (and do) connect from Java client on Windows to our C++ server on Solaris.

Tuesday, March 4, 2003

*  Recent Topics

*  Fog Creek Home