The Old Joel on Software Forum: Part 3 (of 5)

A linker for .NET?

OK, i'm not sure i'm well enough qualified in compiler design to really comment on this (I skipped that module at uni for HCI instead ;) ), but I can think of questions...

So - if this would be so useful and the technology exists (which I'm sure it does, for c# at least) why do you think MS is preventing it? To speed take up? Bloody mindedness? Strikes me as less likely...

Secondly - are manifests really that hard? Wouldn't a GUI tool for manifest generation sort this? I haven't tried to fiddle with them that much.

Thirdly - if this was so obvious, why didn't Sun do it with Java? Technological or political barriers?

Ok - i'm not saying that any of Joel's points were wrong, it's not really my field, but I am curious as to what others think, and any other theories people have on this.

Andrew Cherry
Wednesday, January 28, 2004

It's not a linker they need. They had one in VC++/VB6 and they still had DLL hell with mismatched versions of shared libraries.

What you may be looking for is a way to compile all your assemblies into a single Intel binary.

With all the interdependencies of libraries, components and objects, this may not be possible.

Anonymous
Wednesday, January 28, 2004

I'm thinking that what Joel is looking for would be a C#/CLR equivalent of GCJ, the GNU Compiler for Java (part of the GCC project). GCJ can compile Java source to native code object files, and can also compile bytecode (Java .class files) to native code object files. These files then get linked with libgcj, which contains the core classes ported to native code as well as a garbage collector and bytecode interpreter (so GCJ-compiled apps can still load and use Java bytecoded classes), to make a native executable. It's not perfect at the moment, but it's a decent solution for many applications.

Unfortunately, I don't know of any such effort for C#/CLR code. Perhaps eventually something like this might be an outgrowth of the Mono or DotGNU projects.

Eric Bowersox
Wednesday, January 28, 2004

I think Microsoft and Sun are operating under the assumption that dynamic linking "works." i.e. that you can "upgrade" the implementation of an arbitrary library and expect no problems. This is the common wisdom on dynamic linking, right? That you can "upgrade" a library to fix implementation bugs, and you save memory because all client applications share the same library.

I think Joel is one of a growing minority that realizes dynamic linking isn't the panacea it appears to be. In the real world, the connection between client application and library is often so intimate, with both relying on implementation bugs or quirks of the other, that upgrading a DLL to a new version is an accident waiting to happen.

Personally I also think the fewer dependencies, the better. Link everything statically except for the fundamental core libraries that must be dynamic (USER32, GDI32, etc).

Dan Maas
Wednesday, January 28, 2004

No! Dynamic linking is a good thing. It's perfectly fine to statically link everything when you have a little product like FogBugz, but where I work we have about a hundred products that all share a common framework. Currently they are statically linked. This means that (a) the exe is HUGE, and (b) EVERY SINGLE PRODUCT must release simultaneously. We're replacing this framework with one that is COM-based, with each product (and many components of products) as separate modules. This way, we can update, bug fix, and upgrade individual pieces without making the user get a whole new giant exe, and without forcing the whole world to release in lockstep.

I see your point about .NET, but most of your complaints are about Windows (e.g. the need to reboot all the time) rather than about dynamic linking itself.

Joe Ganley
Wednesday, January 28, 2004

Sure it'd be nice to have a linker, but it'd be nice to have world peace too. You can have users download the runtime and install it with your app, its not a movie length install because you don't have to install the latest IE (assuming they don't have less than IE5.5) and you don't have to install the critical updates that require reboots. Its not quite as bad as Joel makes it out to be.
I think the benefits of the runtime far outweigh the occasional pain in the ass install - garbage collection, no dll hell, etc.

K
Wednesday, January 28, 2004

"I think Microsoft and Sun are operating under the assumption that dynamic linking "works." i.e. that you can "upgrade" the implementation of an arbitrary library and expect no problems."

Actually, no.

.NET has the concept of "assembly evidence". What .NET encourages is not backward compatibility, but side-by-side installations. Although they can do some overrides in this are (such as running a 1.0 app on the 1.1 framework), I don't think that they encourage that in the general case. It's very hard to do right.

Brad Wilson (dotnetguy.techieswithcats.com)
Wednesday, January 28, 2004

Uhmm, how about including the .net runtime in your installer?

How about unstalling only .net in windows update?

Oh, and those messages you had to press "OK" for were the certificate questions about installing software from a web site. It's useless for you anyway, so just click the "trust all software from ms" check box...

Mike Fedyk
Wednesday, January 28, 2004

I think AC is right to make the Java comparison, and it's apt for several reasons.

SUN didn't release a static linker for Java - or even a native compiler - for the same reason that Microsoft doesn't want to release a linker for C#/.NET: they want independent software developers (ISVs) to be the channel that gets their runtime installed on users' computers.

How did this work out for SUN? Well, Java has gotten pretty popular. For server-side software. But not for desktop applications or the applets it was originally touted for. Much of the reason is that desktop application developers were faced with the quandary that Joel outlined and decided to avoid it by avoiding Java. (Inability to get native-level control of the look-and-feel was another, smaller reason. Performance was a red herring.)

With that said, things may be different once .NET becomes ubiquitous in the way that Windows is now ubiquitous. Joel and Fog Creek seem comfortable with testing and releasing their software on several versions of Win32 (Windows 2000, NT, XP, etc.)

Until then .NET may only hold traction in the server-side space, where developers can assure the availability and version of the .NET runtime.

/LT>

Lenny Turetsky
Wednesday, January 28, 2004

And Joel, you didn't have to install the .NET runtime from Windows Update. You could have installed it from the Redistributable download Microsoft makes available. You can ship your product (or the demo) with the redisributable and avoid putting your customers through 2 hours of Windows Update reboots. (but your systems should have those patches anyway!).

Go to http://msdn.microsoft.com/netframework/technologyinfo/howtoget/default.aspx#section3 and read about Microsoft .NET Framework 1.1 Redistributable.

Michael
Wednesday, January 28, 2004

Your column reminded me that Microsoft has always considered a linker to be an afterthought. Back in the early '80's I worked for a company that developed Intel-based workstations, Convergent Technologies. Microsoft traded complete rights to their COBOL compiler (a big deal then) for our linker, which they used basically unchanged. That's why all version 4 .exe files and earlier all started with "GW", which are Greg Walsh's initials, one of Convergent's lead engineers.

Tom Ball
Wednesday, January 28, 2004

Uh, I mean isn't the obvious probem the same problem that .DLLs are meant to solve. There is no good reason to have multiple copies of the same code running all the time. If everyone statically links in all the functions they need, that's a big waste of memory.

It's a tradeoff. One the one hand you save a lot of memory by using .DLLs or the .NET runtime, and you can supposedly get upgrades for "free" -- but on the other hand you now have something that can change beneath your feet that you have to test.

Andy
Wednesday, January 28, 2004

The whole argument here is this won't work without resorting to understanding 'manifests' - which is deemed impossible. Please!

Also, overlooked is the reason a runtime is used in the first place, which are numerous. Take even code security for example - how can you insure that if your final file is a traditional win32 exe? Note that even they require a 'player' (much larger that 20mb too), it is just distributed differently. If it took 2 hours to install a program that took 60 seconds to make, I'd suggest spending an extra 5 minutes putting in an installer that links to the .net redist next time - if that is what you need...

This really strikes me as an "I don't understand this so it must be wrong' argument, sorry.

Robin Debreuil
Wednesday, January 28, 2004

Longhorn, the next version of the Windows desktop platform WILL have an integrated .NET runtime shipping with it. I know that's a long time off, but let's face it, it's kind of hard to include the runtime when it ships AFTER the OS (i.e. Windows XP shipped before .NET framework 1.0). Windows Server 2003 included the framework 1.1, so it doesn't have this problem. So I think that it shows the new pattern, that a version of the framework will be included in future releases of the Windows platform, but it's hard to go back in time and add the bits to the box after they ship...

None of this changes the issues today, but having a linker that would have to put all needed parts of the framework into each executable wouldn't be something people would like either. Then the complain would be, "Why are all my .NET executables at least 15MB in size" (guessing that putting in at least the runtime and base class libraries in there would add at least that much, and some programs might be even larger). And now people would be upset having to download the redundant runtime support code (even embedded in each program) over and over again. So you can't win with a linker either, you just change the complain to something else.

If you want distribute .NET based programs from the web, add a link to the location of the .NET runtime redistributable. It's no different than people who add links to the Flash, Quicktime, Adobe Acrobat Reader, or other needed product to their site and tell user to go there first if they don't already have it. Or host it on your own site, if you don't want to link to Microsoft. You can find the redistributable runtime at: http://www.microsoft.com/downloads/details.aspx?FamilyId=262D25E3-F589-4842-8157-034D1E7CF3A3&displaylang=en which offers all the different languages the runtime supports as well.

Lee
Wednesday, January 28, 2004

>It's not a linker they need. They had one in VC++/VB6 and they still had DLL hell with mismatched versions of shared libraries.

No, but they never had a linker that let you take out of the VB runtime library what you need, and use that. Big difference here. Joel is obviously asking for a linker that grabs all that code crap and puts into one dll, and you .exe application (or, just one .exe).

Further, the whole point was that the linker SHOULD give you a option to static link. In other words, go through all those library things on your computer, pull out exactly what is needed, and link it. This would be FABULOUS for all ActiveX controls you use also. That way you have no dependencies. That way, even when some other application comes along and updates the ActiveX calendar in office, it will not start breaking other applications (I deal with this type of dll hell many times).

In addition this would mean that you don’t need the VB runtime installed on the PC either!.

Further, if the linker ONLY grabs what code libraries and dependencies it needs, the size of the resulting code is not TOO large. Further, for general stuff like text boxes etc, they are windows a api’s anyway. A lot of just forms and user interaction is windows api. So, in fact, a lot of that runtime stuff IS NOT ever used!

We never did have a static linking option in VB, and it would have really nice.

With VB, it would be nice to link in certain parts, especially some ActiveX controls. Those ActiveX controls are not that large, and usually a product only has a few special ones your use. It would be nice to link in those, and not have to use the existing ones, since they CAN change. So, even if I can’t link, or grab parts of the VB runtime, it sure would have been nice to link in other dll libraries, and especially the ActiveX controls used (heck, should the linker should assemble this extra junk into ONE nice dll lib for you). If one could do this, then installing other new software would not break my application.

Now, back to the issue of the runtime library. Few, if any VB developer complain about the runtime library? Why is this so:

Answer:
The runtime library is stable, and not changing. If a runtime library is stable, then the benefits QUICKLY move to using a runtime library in place of static linking. You can compile and create applications that are EXTREMELY TINY in VB6. If that runtime library is stable, and not changing, then without a doubt, then that runtime library becomes your friend.

In the case of Joel’s problem, the bad evil guy is not a runtime (in this case the CLR). The bad evil here is dealing with a runtime THAT CHANGES! That is the real problem here!

The solution I suppose would be either to not change the runtime, or have given us a static lining option. Perhaps after a few years, and the CLR becomes stable (or us developer get really familiar with it and learn what not to use and what not to do!), then we would STOP using static linking. It would also be nice way to treat the developers. And now, the onus would be on MS to get its runtime real good, we are happy enough to stop static linking! Right now, the onus is on the poor developers to hope that this moving target does not break anything!

I mean, after all, we can very much consider the windows API a runtime system. We don't complain about the windows API being a requirement for our software. I mean, really...how about we link the whole windows system into each application! Really, I mean, if runtime is bad, then so is the whole windows API!

The main reason why we don’t complain about the windows API is because MS makes real sure that it don’t change, or makes a very huge and honest effort not to break anything.

So, the real evil here is not a runtime, but a changing runtime. If the .net runtime don’t change, or if when we update it there is no breakage or problems with our software...then we don’t have a problem.

Albert D. Kallal
Edmonton, Alberta Canada
kallal@msn.com
http://www.attcanada.net/~kallal.msn

Albert D. Kallal
Wednesday, January 28, 2004

The amusing thing is that .NET *does* have a linker - al.exe. It's command line only, and VS.NET doesn't use it because it doesn't do multimodule assemblies. But there is one there.

Of course, the lack of the linker isn't what Joel was complaining about; he was complaining about having to distribute the runtime. And yet he ships VB apps, which have the *exact* *same* *problem*. It's just that, VB being older, the runtime's already installed.

To be honest, MS isn't really targetting the desktop application market with .NET; they're targetting the server space where these kinds of problems just aren't an issue. They already own the desktop app space, after all.

Chris Tavares
Wednesday, January 28, 2004

Sounds like MS have been plagarising Sun again. Java also has this problem (despite GCJ, you legally cant pull apart the swing libraries). Java even has manfests in its jars (same terminology).

There are commercial tools to strip down the libs, and combine them together (the closest you get to linking) - but they have limited success. They run into trouble with things like reflection and lazy class loading.

I have no idea why they are so adamant on stopping you create monolothic executables. No idea...

Oh and CityDesk is NOT the best example of a simple distribution. I remember I got a polite message telling me I needed the windows scripting lib/host (I think) and it directed me to microsoft. From there on I was on my own, and had to work out what to do, then redo the install. I think it was because I only had IE5.0.

For a good example of monolithic executable friendly GUI apps, check out http://www.memecode.com/ (he is a bit obsessive about not needing an installer, or dependencies etc).

Mike Neale
Wednesday, January 28, 2004

I think that, between nobody wanting to use .NET for something shrinkwrapped until most users have the redistributable, plus the new Longhorn APIs, is going to bode poorly for MS.

The longhorn one is great. MS needs something new in Longhorn to sell, hence all of the new APIs. However, nobody's going to use them until there's installed base. .NET gets some installed base simply because an existing windows system gets a free upgrade. But MS can't do that especially well with Longhorn. Sure win32s worked, but times have changed somewhat.

The problem with the .NET linker is that the environment is pretty heavy-handed; I'm not sure if a linker is enough to make it acceptable for shrink-wrap software.

Flamebait Sr.
Wednesday, January 28, 2004

But MS are forcing .Net down your throat, no matter what you think.

Sure, there are still lots of VB and MFC "legacy" apps, and will be for some time, but that won't stop them dropping and real support for the non .Net way (which includes MFC an so on).

Mike Neale
Wednesday, January 28, 2004

"But MS are forcing .Net down your throat, no matter what you think."

There's nobody at the door with a gun, making me write .NET code or else they'll shoot my dog. Sheesh.

Brad Wilson (dotnetguy.techieswithcats.com)
Wednesday, January 28, 2004

> This really strikes me as an "I don't understand this so it must be wrong' argument, sorry. <

I have to reluctantly agree. One of the touted benefits of .Net is that you can use XCopy deployment with it. However, that only works if the target machine already has the Framework installed on it. From a practical standpoint, it's really user-hostile to just dump a .Net exe on an end user and expect him or her to download the framework independently. No wonder his "midnight countdown" distribution didn't go well.

If Joel really wanted to distribute a professional .Net application, he'd just need to create an installer package. (The installer setup that comes with Visual Studio is probably fine for most needs -- no need to resort to InstallShield.) The package could either include the Framework Redistributable, or prompt the user to download it the Framework separately from WindowsUpdate if needed.

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnnetdep/html/redistdeploy.asp

Also, he could have two packages -- e.g., a 28 MB package that includes the Redistributable, and a 5 MB package without it. His site could use a script to determine which version the user needs, and recommend an appropriate version. Yes, the 22MB size of the redistributable is a PITA, but Joel's parade of horribles seems overblown.

Robert Jacobson
Wednesday, January 28, 2004

Who cares if all your applications are 10 megabytes and include 8 megabytes of shared code that could be in a DLL or share assembly?

I've got a 160 gigabyte harddrive that I'm unlikely to fill for the next 4 years at my current usage rate. I've got room to spare, Microsoft, Sun. While I admit 160 is a bit much for Joe Compaq, how many applications do you think he's installing? 5 a day?

Let us create monolithic static-linked executables! It isn't the fucking 80s and the miniscule space wastage is worth the compatibility guarantees.

Mister Fancypants
Wednesday, January 28, 2004

Has anybody tried the Salamander .NET Linker? http://www.remotesoft.com/linker/index.html

Ryan Duffield
Wednesday, January 28, 2004

Statically linked executables aren't necessarily huge. There is a wonderful C library replacement for Linux called dietlibc; many applications statically linked against dietlibc are smaller than if they are dynamically linked against glibc!

Part of the reason statics can get huge is that library developers have stopped (or never adopted) the old practice of putting each library API call in a separate .c/.cpp file. Some linkers unfortunately decide to include or exclude code on a file-by-file basis, not a function-by-function basis.

Also template-heavy C++ like STL and CORBA implementations can inflate executables a LOT.

I've seen ranges between 600KB and 12MB for executables that do basically the same thing.

Dan Maas
Wednesday, January 28, 2004

"And yet he ships VB apps, which have the *exact* *same* *problem*. It's just that, VB being older, the runtime's already installed."

NOT TRUE.

The VB 6 runtime is about 1.1 MB. 1/40th the size of the .net Runtime. (The .net Runtime is actually more like 40 MB. The COMPRESSED size is 22 MB.

ALSO, .net requries IE 5 (or is it 6 ?) or greater.

That's even HARDER to install. And potentially problematic. Remember the horror stories from the old I.E. 5 or 5.5 upgrades.

P.S.
Do folks with customers really think that this is a non-issue for your customers?

The real Entrepreneur
Wednesday, January 28, 2004

"There's nobody at the door with a gun, making me write .NET code or else they'll shoot my dog. Sheesh."

Don't be so sure... <hears a heavy fisted knock at the door> ;)

Seriously though, MS sees it as the future. Otherwise, where is VB7? (non .net) and so on. They can't physically force you, but if they stop support, eventually it will die.

Of course, I imagine MFC will be kept alive for some time. I can't beleive that everyone doing VC++ style ISV things will all switch to C#.

Mike Neale
Thursday, January 29, 2004

Ok, here is my guess:

Wouldn't linking to a good old win32 .exe remove all af the advantages of the old platform?
Let's see: here is this code I download form a website. If it is a .NET program, then based on all the evidence I have I can selectively allow operations to take place or not at a very finegrained level. This allows me to trust things like href .exe's etc. I >know< they are going to be contained in a nice sandboxed environment. The basic idea we already got with Java, and .NET has taken this to the next level.
Now if the code ws a regular .exe, produced by the fabuled linker, the "runtime" is just good old Windows. Yes, the nice thing about this is that the user does not have to download it, but this also means you have non of the benefits of the .NET runtime as a user or administrator. Yes, you could try to muck around with "Run As" and a less priviledged account etc. but .NET just makes this a few orders of maginutde simpeler and more powerfull.
Yes, having to provide the runtime for those that do not have it already is inconvenient for now.
But the "linker" scenario, while keeping the advantages to the developer of having this nice productive .NET environment would deny all benefits of the modern runtime to the end user.

Just me (Sir to you)
Thursday, January 29, 2004

"Wouldn't linking to a good old win32 .exe remove all af the advantages of the old platform?"

Of course this should read >new< platform.

Just me (Sir to you)
Thursday, January 29, 2004

For my customers, it's a non issue. Our setup.exe is currently over 120Mb, but then we don't do demo download versions over the web.
Manifests do solve the "wrong version" problem so if you need back track to an earlier version because the new runtime screwed you life is OK. (I would recommend strongly that you try it!)
It's better than problems you used to get with MFC, various subtly different broken versions around on users hard discs, and which one you got depended on which app loaded first.
I haven't seen the MFC problems for a VERY long time.
Joel has forgotten how the office 95 (and I think the 97) team got around this problem they force overwrote your DLLs regardless of version number (NB I may have the wrong MS product here, but I'm sure I remember one of them doing it).

Peter Ibbotson
Thursday, January 29, 2004

Although I think there is an interesting argument to be made in favour of static linking, I don't think static linking in .NET would be at all practical except in extremely limited situations.

As others have pointed out, I think Joel's main argument is fueled by his frustration with the Windows update process which is completely a seperate problem.

It would be pretty cool to have a utility that transforms simple .NET apps that don't make use of runtime-supported features (Code-access security, Remoting, reflection, etc.) into self-contained native applications. But it would eliminate much of the value of the .NET platform, and so would only be practical in very limited situations (like, perhaps, Joel's countdown clock).

The whole model of .NET revolves around self-contained self-describing assemblies (packaged as .dlls and .exes). This is one of the big advantages of .NET over Java.

I think of the .NET runtime as the evolution of the Windows operating system. It would be pretty silly to statically link with all the windows system libraries. Sure there will be some pain when upgrading from one version of the runtime to the next, but just like upgrading OSes 90% of the problems arrise due to developer relying on some undocumented behaviour. I think Microsoft did a pretty good job of making .NET 1.1 backwards compatible with 1.0 (much better than WinXP being compatabile with Win2K), but they also did a good job of ensuring that apps designed for .NET 1.0 can continue to run in the 1.0 runtime even after 1.1 has been installed (in most situations this is the default behaviour).

Anyway I normally highly respect Joel's oppinion, but in this case I think its mostly a case of wanting .NET to be something it isn't and wasn't designed to be.

Rick Byers
Thursday, January 29, 2004

>There's nobody at the door with a gun, making me write .NET code or else they'll shoot my dog. Sheesh.
Did you recently paid a visit to the ActiveX/COM section of MS site? Even the specs for COM were withdrawn and links are mostly 404. And this not vaporware, lots of programs are still based on ActiveX objects.
What do you know about Visual Studio 6 Service Pack 6? Is in beta for like a half of a year. It seems it's like Win NT4 SP6a, as far as I know Windows NT is still in extended support, but no service packs are scheduled because it works flawless or because it is a marketing scheme.
The ugliest thing I saw was the moment when MS removed the OpenGL section of the MSDN Online. Suddenly opengl.org became a pile of 404's from around the world, from universities and corporations, though the two most important graphic chip sellers were writing newer OpenGL drivers and were planning the specs on the next version.
It is a thing that starts from small issues: I never understood why they didn't provided a Mini size to WMP 9 in other Windows versions than XP (I'm talking about the Windows Media Player minimized in the taskbar, but wit all the controls available). I know it could be done, even Google draws windows over the taskbar, but it was the X factor - the thing that should tell me XP is better than 2000.
Well, I'm not biting and I'm sure Win32 plain C is the best way for an desktop app. But wait, there's no funny wizard, what should we do? Torture the poor devs, force them to write code?
Win32 was the most reliable thing in the last 9 years. If it worked on Win95+IE4, it works now; the rest (MFC, VB runtime, Java and even the young .NET) were morphing into something more powerful, yet different on things that should be supported.
http://www.gotdotnet.com/team/changeinfo/Backwards1.0to1.1/default.aspx
(Microsoft seemed ashamed of these changes, so they pointed me to a "independent" site.)

Luci Sandor
Saturday, January 31, 2004

My MSDN Library, January 2004 edtion, has the usual sections on COM and ActiveX. Microsoft likes to move pages around, did you try searching for a new location?

Chris Nahr
Sunday, February 1, 2004

See

http://www.remotesoft.com/linker/

DialogDroid
Thursday, April 15, 2004