Fog Creek Software
Discussion Board




the new fogbugz setup program

this isnt meant as a flame, even if it sound like one. i am just a little disappointed about todays article.
until today i had the feeling that joel knows what he is talking about and usually had the same opinion as joel on the topics he discussed.
and now this:

1.: "i needed longer than expexted ...". this paragraph sounds like someone had a wrong specification to begin with. and as it looks like it wasnt a specification from "outside", like from a customer......

2.: actually performing system-changing tasks within a setup-program and then having to reverse them ?
maybe i overlook something here but i cant see any reason for doing so. you could easily store the selections and input from the user in memory and then perfom the needed tasks after the user confirmed the last option of the last selection (i.e. pressing the "finish" button).

just puzzles me coming from joel....

Matthias Meyer
Thursday, October 03, 2002

Actually, I wasn't going to comment, but I must admit that when I read the article I too was wondering why you'd want the setup program to make the changes in the middle of the "setup options" instead of at the end.

The only explanation I can think of is that later choices are dependent on the _results_ of the setup choices at earlier stages. (e.g. The user selects a particular upload location, and the setup program checks that it's available and alters later choices dependent on thef facilities that the upload location provides. Bit of a contrived example though).

Adrian Gilby
Thursday, October 03, 2002

Hehe, no one should respond here so Joel gets pissed and has to write that article.

But to spoil the surprise, I think James Murphy explains one reason in another thread...  Remember, Fog Creek's main differentiator is happy-user usability, so there's a reason to give a user instant feedback on failure.

Of course, that's just my guess...  I don't know anything about ASP or Fogbugz.

Sammy
Thursday, October 03, 2002

Why should a specification come from outside? FogBugz is software produced for a wide market, not any given monkey. And why do people think misjudging how long something will take is a sign of doing things badly? From what Joel wrote I'm thinking he's never done any installation software before and thus lacked the knowledge needed for an accurate estimate.

Mr Jack
Thursday, October 03, 2002

I thought about that issue as well (the need to actually perform the actions instead of carrying out a script when the user clicks finish), but I too came to the same conclusion that step '> n' depends on the actual result of step 'n'.

I'm just wondering why Joel chose 'VC++ and MFC all the way', when it seems (concerning CityDesk) he was only interested in 'going to the metal' when it made sense to do so.

Rick
Thursday, October 03, 2002

Shouldn't the setup program be part of a daily build? Not something left until the end?

R
Thursday, October 03, 2002

I thought you only wrote something from scratch if it was part of your core function?  Also, I didn't read in the description of the problem anything that the other install programs couldn't already do, so what is that other 75%?

load
Thursday, October 03, 2002

Since FB is server software there are many system level tasks that need to be done - account creations, account testing, service installations, stopping and starting services, permissions granting, manipulating the metabase, etc.  The win32 api for this stuff is not what you would call "user-friendly".

The reason the setup needs to unwind is because even if you succeed in doing step 1, step 4 could fail.  If step 4 fails (which you can't know before you actually do step 4) then you don't want the changes that steps 1-3 did to actually stick if the user cancels.

Writing it in MFC was the quickest for us, since 90% of the code HAD to be in C anyway (it was mostly all API code).  In addition, previous experience with writing the Next-Back wizards made it easier to write than putting all the DECLARE's into VB.

Buy FB 3 when it comes out and you can see how neat the setup program is :)

Michael H. Pryor
Thursday, October 03, 2002

It seems that you are trying to combine two separate issues, installation and configuration.
Many installation programs make that mistake, making things harder than they should be.

Was there no way to simpy copy what had to be copied, and then leave configuration up to a specialised tool?

Erik
Thursday, October 03, 2002

Maybe I'm missing something, but wouldn't it have been much easier to simply collect the users decisions up to the last stage and then commit them all in one go?

Mr Jack
Thursday, October 03, 2002

This is why I hate programmers.  Everybody thinks their way is the best way.  Leave Joel alone.

big gulp
Thursday, October 03, 2002

The only way to learn is through discussion.

Mr Jack
Thursday, October 03, 2002

@michael h. pryor:
i saw that need for undoing all changes made if step 4 fails, BUT if you do the changes while the user is still in the setup-process, and maybe he changes his mind about some path 3 times (or mistypes 2 times) you would have to undo and redo the changes 3 times..if you wait until the user has made all settings like he wishes, you will only ever have to do it once regardless of how often the user steps back and forward in the process.

Matthias Meyer
Thursday, October 03, 2002


Let's suppose that your last step is to copy some files to a folder, after creating a user, set up some services, etc., and something goes wrong. You need to undo some things; a common approach is just undo everything and bail out. I'd be very happy if the setup program I'm using let me change a parameter and continue, without starting again.


I think I understand why they chose this model. Kudos to Fog Creek, I know most companies would have chosen something less complicated :-)

Leonardo Herrera
Thursday, October 03, 2002

A new appreciation of installers...  basically fog creek is saying
1) interactive programming is better than batch programming when you have a non-trivial process
2) fogbugz installs are non-trivial b/c of things outside your control
3) the user doing the install is a programmer, and you want to give her the best tool

Sammy
Thursday, October 03, 2002

Random thoughts:

Doing the declares for windows in VB requires that you include one reference to a .tlb file (that doesn't have to be distributed with your app) that may, for the time being at least, be found here:

http://www.themandelbrotset.com/Technical/Typelib.asp

James Murphy
Thursday, October 03, 2002

Random thoughts 2:

There's an awful lot of second guessing going on here - in my experience getting even a simple install, written with a decent tool, to behave both as you want and as you'd like can be challenging - especially if you want to do something that's not quite as the developers of the installation tool envisaged.

Because Fogbugz is a server based tool the install is almost inherently not simple especially if you want it to be tidy if it runs into a problem. Its a fairly safe bet that the developers at fogbugz baled on the existing tools because they couldn't make them do the right thing (by their definition) despite their best efforts which inevitably leaves you with only one place to go...

James Murphy
Thursday, October 03, 2002

I Wonder how much of the complexity was due to the non standard way things are done from one version of windows to another.  My very limited install expereince comes from writing an RPM to install JBoss on developmer machines, and there I knew I only had to target Red Hat 7.2 and 7.3 machines.  I could use the RH toos for creating user accounts, and everything else could use Bash.  I also did not want the install to be interactive, I wanted it to be identical on eveyr machine I had to support.

One thing that C++ gives you is exception handling.  Could make your cleanup code easier...or harder.  I haven't though about this in a while, but I think I would define the activity behind each page as a series of objects which each implment the same interface.  If a step fails, handle the rollback in your current try block and retrhrow the exception,  keep passing back the excption until you get to a point that can handle it.  Incomplete rollback...hmmmm.

However, I think setup programs are best if designed to get all your information up front, and then attempt to install as a single transaction.  Since Something like FogBugz is more likely to be installed on a single machine within an organization, it probably would be that big a deal, but take something like Postgresql, which we needed on each developer machine, and you want that install to go as non-interactively as possible.

I also have to agree with the comment above abou8t splitting configuration from installation, as that means you focus on Configuration tools, and not on install tools.  You get more bang for your buck with config tools, as they can be reusued when things change.  Joel, can your installer double as a configuration tool?  That would be cool.

Adam
Thursday, October 03, 2002

Just out of interest,

how would you handle the "single big transaction" if it involves discrete sequential interactions with multiple subsystems (User management, file system, IIS configuration, ...). Seems to me that you would have to roll back the discrete parts. Now, if you do have this capability anyway, why not offer the user direct feedback whenever he is deciding on one of those discrete parts the moment he completes the step?

Just me (Sir to you)
Thursday, October 03, 2002

Jesus Christ!  Obviously most of the respondents have never written a setup wizard that had to do what Joel did.  I have, and it looks simple when you first start out, but the end design looks nothing like what you started out with.  My custom program had to do this:

* Install SQL Database schema/data
* Create NT User accounts with certain permissions
* Installing IIS web servers (included starting/stopping IIS and writing things to the IIS metabase)
* Installing and configuring COM+ applications
* Creating users in Active Directory.
* Performing some other custom steps.

Each of the API's for configuring SQL Server, IIS, COM+, ADSI are not the easiest to use and you do not use the API for IIS in the same way that you use the API for COM+.  Once you have the API logic coded, then you have to hook it up to the UI (In my case a VB form).  As Joel said, you have to be able to rollback all installation actions so you do not leave the installation in an inconsistent state (NT accounts that didn't get deleted, COM+ applications running as Interactive User, etc.).  So you need to have rollback logic to undo whatever it was that the installation did (I ended up using a State design pattern from the GoF book for this).  Also, you cannot just gather all data, wait till the end and then report to the user that they do not have access to SQL Server because they are not logged in as Admin.  That is a shitty way to treat your users.  How would you like it if you have to fill out a multipage web form, only to click the submit button and have the server tell you that you enetered an invalid email address on page 2?  Not very user friendly.  So cut Joel some slack, he did the right thing here....

Tim
Thursday, October 03, 2002

I'll pass judgement on this method when I know more about the decisions.  But it still raises some questions.  What happens if the user's computer crashes in the middle of the setup process?  I think there are two issues here; making sure an action can be taken, and actually taking it. What if each step (installing the db, creating accounts, etc) tested to see if they have the correct permissions, but the actual action was taken at the end?  Having undo capability really raises the complexity; is the complixity necessary?  I anxiously await Joel's article.

Monsur Hossain
Thursday, October 03, 2002

Regardless of whether you perform the installation/configuration steps as you go, or at the end, you still need to have rollback logic.

Tim
Thursday, October 03, 2002

what is  fogbugz setup program?

vividly vivid
Thursday, October 03, 2002

i've spent 2 years writing setup scripts in installshield and wise.

if what joel did took less than a month to do (which i'm sure it did) and if it works, then he did the right thing.

it would have taken him just as long to do it in installshield or wise, and it would have sucked ass.

expert
Thursday, October 03, 2002

Does anyone else find it funny that some companies pawn
off the "installer project" on the least experienced
developer?  At least Joel realizes the importance of this
project and tackled it himself. 

Thomas R. Dial
Thursday, October 03, 2002


  Yes, that´s true.  When I was an intern, I was responsable for the "installer project" at our company.

  First I crafted one by hand in Delphi.  Later I started using other tools.  When somebody needed some instalation program, they´d come to me.

  Now that I´m not an intern anymore, I´ve never done it again. 

Ricardo Antunes da Costa
Thursday, October 03, 2002

I'd say Joel hasn't written that program at all yet, but is just trolling for some thoughts on it.

Alberto
Thursday, October 03, 2002

Joel has trolled for thoughts on this before but here's another angle:

1. Fog spends a lot of time investigating installers and are never satisfied.
2. They wonder why the heck they can find an installer to do the job they want.
3. Being high ego software engineers (that's a good thing) they say, "Let's try to write one ourselves. Maybe we produce one that's ideal for FogBUGZ that we can use for some of our other products.
4. They learn why it's such a pain.
5. They learn stuff (tricks and gotchas) about automated installations they didn't know before.  They become experts relative to most of the population.
6. They learn some respect for the genre.
7. They exercise their engineering skills and test their user interface theories.
8. Maybe they can sell it:  "FogINSTALLZ"

Rock and roll!

tk
Thursday, October 03, 2002

For the sake of argument, talk about a fictional product call FictionalBUGZ. I'll take Tim's list above (repeated here) as representative - assume FictionalBUGZ setup requires to:
* Install SQL Database schema/data
* Create NT User accounts with certain permissions
* Installing IIS web servers (included starting/stopping IIS and writing things to the IIS metabase)
* Installing and configuring COM+ applications
* Creating users in Active Directory.
* Performing some other custom steps.

If this is indeed the case, then IMHO Fog Creek did the right thing; These requirements are specialized enough that InstallShield and friends require, in my experience, about the same amount of work as starting from scratch[1] to get things right.

What bothers me is that these things all have to be done at all. For a relatively simple task (mostly database frontend), one has to install a database server and web server that can deal with 1000x the load they're going to get. And properly configuring them - which is usually easy if they have just been installed, but is very tricky if they already were installed on the machine and previously configured.

Furthermore, these components have to be upgraded more often than not if you value security (and just _occasionally_ if you value robustness). Having been a part-time admin at just about any place I worked in, I've always preferred monolithic apps to "database from vendor1 + webserver from vendor 2 + appserver from vendor 3 + kitchensink from the kitchen" style apps if all other things were roughly equal[2] precisely because of these reasons.

If, for example, FictionalBUGZ uses an embedded database library (e.g. SQLite or libmysql), and an embedded Web Server (too many options to even list), it would be simple to install and run as a service as there are no dependencies. One step setup / one step install would be sufficient and much less prone to failure, InnoSetup or NSIS would probably be an overkill for this kind of setup.

But then it won't be able to support a seatcount of 100,000, I hear you say. Well, guess what - it's not intended for that market; And database performance is usually NOT the problem in increasing the seatcount by an order of magnitude or three.

Too cut a long story short (too late for that, but ... whatever), I believe that IIS and MS-Dev/MS-SQL are the wrong tools for the jobs - and that wrong tools bring their friends along (complex installer).

[1] Assuming, of course, a competent designer / programmer, which I take for granted that describes all of Fog Creek's people. I have many war stories about  e.g. InstallShield that are good arguments in this "in defence of the NIH syndrome" case. If you can't assume competence of designer / programmer, you shouldn't be trusting their software for anything of value. Really.

[2] All things being equal is usually hard. For example, even though I prefer monolithic to many-vendor style apps, I prefer no to have vendor lock-in. Unfortunately, monolithic and vendor lock-in go hand in hand more often than not. Also, note that multiple vendor style is NOT necessarily modular - more often than not, they are tightly coupled. Most apps[3] require a specific Database (e.g., Oracle 8) and a specific web server (e.g., IIS). They are, in essence, just as monolithic but are not controlled by a single vendor which makes them even worse than monolithic. You might claim e.g. ODBC gives you choice in which database server to use, but that is rarely if ever the case in practice; And when it is, you can just as well include an embedded ODBC database.

[3] Java apps are not immune either; They have greater agnosticity (sp?) with respect to the app server / web server, but the incompatibilities between  database servers plagues them just the same.

Ori Berger
Friday, October 04, 2002

So will "FogInstallz" be released as a product ?

With what it seems to do, and what most software companies need to do i think their would be a huge market for a product that can do this. I myself will be attempting this exact thing (installing SQL Schemas, AD Users, COM+ etc) soon and, well Wise and Installshield just don't cut it!

Chris
Saturday, October 05, 2002

If anyone is attempting to do what Joel did with his Fog install, I suggest grabbing a design patterns book and implementing the install as a State design pattern.  Works really well and simplifies the when/what to do to perform each install/uninstall step.  I have an installation program written in C#/Windows Forms that uses the State design pattern if anyone wants to take a look at it...

Tim
Saturday, October 05, 2002

Yeah... I really think install programs should be "willie" complicated.  I really reflects well on what is being installed... and I always like reading the advertisements during the install on what I'm installing... kinda makes me want to go out and buy a copy.

Yep.

Get a life.

Joe AA
Saturday, October 05, 2002

I guess Joel knows why Visual Studio.NET has over 50 people working on the setup program.  Concidering it does all the above (Rollbacks, rollbacks after crashing, automated installing, remote installing, etc) it's not all that surprising.  Hell the app has it's own massive Database Engine the task is so complex.

Lucas Goodwin
Saturday, October 05, 2002

I find it curious that on one hand Joel talks about users of his bug tracking software wanting custom fields even though there are plenty of alternatives. 

Then on the other hand complains that established software is not good enough for writing the install program.

My main concern with the approach taken is that testing the install program is now orders of magnitude harder.  I agree with a previous post that installation is separate from configuration.

I don't think that the installer that Joel wrote should be released as a commercial product for many of the same reasons.  It just adds complexity to the install program.  The install program should be made as simple as possible and do 1 thing - install.  The reason being is that even pure installation is not a trivial process.

JohnG
Sunday, October 06, 2002

Most of you guys just don't get it....

Tim
Monday, October 07, 2002

What is it we don't get?

JohnG
Tuesday, October 08, 2002

john,

I agree that installation and configuration are 2 distinct functionnal steps.

It seems logical to me that you want automatic launch of the configuration after installation. So why the install tools cannot also be the config tools?

Robert Chevallier
Tuesday, October 08, 2002

What if you want to do config outside of the install?  Maybe you want to change the config later, maybe it got corrupted?  Would you need to reinstall?

Its just a case of atomic operations.  I see install as one atomic operation and configuration as another.  If config fails, you should not need to reinstall.

I am sure that people could argue that reinstalling isn't so bad.  A premise I have found in computing is that you can do anything any way you want and still achieve the same goal.  I try to aim for simplicity though.  It makes creating software and testing it easier.

I don't mind the installer running the config after it is done.  That makes a lot of sense.

So I ask the question back:  Why would the install tools need to be the config tools?

JohnG
Tuesday, October 08, 2002

I think that in Joel's case, install and config HAVE to be done at the same time.  If you need to specify a super user id, or install a web directory in IIS, these are both install AND configuration steps.  Now, it is a good idea to have a separate configuration program so that a user can reconfigure things once they are installed, but in MOST cases, the user will assume that the install is all that is needed to get up and running, so why not combine the install/config so that all they have to do is run the installation program, enter in some configuration stuff and let the install do the rest.  If they are happy with the install, they do not need to alter the configuration, if they are not happy with the config, they can use the config tool to modify the configuration without having to perform an uninstall and then a reinstall.   

Tim
Tuesday, October 08, 2002

Agreed.

JohnG
Wednesday, October 09, 2002

Thanks for the response, Joel.

Mr Jack
Wednesday, October 09, 2002

[also posted on http://beust.com/weblog]

Joel is a funny guy.  He is a very insightful and pragmatic developer, whose writings are often sound and to the point.  However, once in a while, he tends to do exactly the opposite of what he recommends.  He has been arguing strongly in the past against subtle points such as "resist the temptation to rewrite code just because it looks bad" and "don't reinvent the wheel", but his latest undertaking leaves me greatly amused.

So Joel decided to write his own installer because "none of the existing ones was sophisticated enough to fill his needs".  I find this perplexing, because WISE, InstallShield et al. seem to be good enough  for thousands of Windows programs, and I can't really imagine that FogBugz (a bug database, that Joel also wrote from scratch) is more complex to install than, say Office.

Come on Joel, admit it:  you just like to write code.  It's okay, you're among friends, we understand.

Cedric
Wednesday, October 09, 2002

Office doesn't install via WISE or InstallShield, but rather installs with the Windows Installer.  (WSI? WMI? Can't recall the acronym soup for it at the moment).  It's a very advanced piece of install software actually as it's also what VS.NET, IE, and pretty much every other piece of MS software was built around.  Even then though the VS.NET team had to add a slew of functionality to their installer that wasn't in the Windows Installer.

Anyways, point being that WISE, InstallShield, et all, are designed to meet customer needs and until just recently with the drive for sysadmin-friendly internet service packages all these install programs had to worry about was installing files, copying files, extracting files, registering some DLLs, entering some registery settings.  They didn't have to create new accounts, stop and start processes, and the myriad of other tasks a sysadmin normalling ends up doing.

Obviously if WISE contacted Joel then this is something they've noticed and want to remedy (A very good thing).

Lucas Goodwin
Wednesday, October 09, 2002

*  Recent Topics

*  Fog Creek Home