Fog Creek Software
Discussion Board




rub a dub - Use templating system

I read the rub a dub article
http://www.joelonsoftware.com/articles/fog0000000348.html and I'm suprised there is no link for it's own forum.

I was reading about html code in the programming code.  And I shudder. Why not just use templates?

I have developed applications using the HTML::Template module freely available from CPAN to allow any number of views for the purpose of each call. Want XML, create XML templates and an XML filter, want text,  or wml, or need a version for 2.0 browsers ... just add a filter, if necessary, and create the templates.

I never have to change the core programming in order to present another view.  I am suprised that such a method was not chosen.  I never put html with programming code unless it's a quick throw away or proof of concept, or unless I can't find a better way (and I'm encountering that with PHP). 

Peter

Peter J. Schoenster
Thursday, January 24, 2002

You rock!

Anon
Thursday, January 24, 2002

Uhm, just a quick question here...

You changed thml to xhtml, and started using stylesheets in stead of FONT tags... and everything still works?
Even on Macs? With Netscape 4.51? No problems whatsoever?

Or maybe this just isn't a concern for some reason?

*bewildered look*

Anyway, nice article ;)

Chris
Thursday, January 24, 2002

The whole point of CSS is that a correctly written browser will handle things Just Fine without understanding CSS (a la Lynx/w3m).

The only reason to support buggy browsers (like Nav 4.7 or IE 3) is because they have enough market share that you're forced to do stupid, complex things to deal with it.

Deciding to drop support for bugs in old versions of Navigator is like deciding to stop working around old bugs in Win 3.1, or all the periphial manufacturers who won't support USB on Win95.  It's Perfectly OK.

(Especially if people can still get access to an old version of your software if they insist on running old systems).

Rodger Donaldson
Thursday, January 24, 2002

The sooner we stop supporting Netscape 4x, the sooner people will adopt Mozilla.

It is ofter difficult to get rid of embedded code in you html (jsp, asp, or other)  if that is the way things were originally designed. 

Refactoring Rocks.  What Joel describes here is one of the Big Refactoring Martin talks about at the end of the REfactoring book.  3 weeks is optomistic...I bet the code wasn't wuite as bad as he made it out to sound.

Joel, what did you use for the middle tier?  Haveing not done ASP in years, I don't know if it supports objects, or if you have to write COM components or what to support that.

Adam Younga
Thursday, January 24, 2002

I used some kind of @import trick to hide the style sheet from Netscape 4. Netscape 4 users get a perfectly usable page with default fonts and grey background :) I still uses tables for layout (especially since a lot of what FogBUGZ outputs *is* a table).

The so called "middle tier" was implemented in classes written in VBScript. I could have made COM objects in any language, but VBScript was there and is good enough if you're disciplined.

Joel Spolsky
Thursday, January 24, 2002

Adam,

I discovered, to my delight, that VBScript 5 supports classes. Alternatively you can write your ASPs using JScript (aka ECMAScript aka JavaScript) which is object-based.

Walter Rumsby
Thursday, January 24, 2002

I use primarily JSP, and I have to say that my biggest complaint is how hard it is to do classes in jsp.  Basically, you have to do them in Java code and then import them.  Yes it forces the code to be more organized, but it does slow down development.  As much as possible, I think UI should be interpreted, as it is usually do something, view the result, change it a bit, view again, etc. 

Our site uses the Tabs metaphor for naviagation.

Adam Younga
Thursday, January 24, 2002

Sorry, posted too quickly.  Our site uses the tabs methaphor for navigation.  I had to pull the code to do this (submit the current form and navigate to the page specified by the tab (image) into Java code.  The round trip development is quite slow.  Anyone found a way around this with JSP?


IDEA, the java IDE, has excellent support for refactroing.  I wouldn't be surprised if you see some of that stuff find its way into Dev Studio next releas.

Adam Younga
Thursday, January 24, 2002

You can do classes in JSP, just use the <%! %> instead of <% %> and they basically become inner classes. I'm not very keen on this personally since I find most UI in JSP can be abstracted better using custom tags.

Gerald Nunn
Thursday, January 24, 2002

I like the idea of refactoring a lot. But the problem is, you cannot use the technique described by Martin Fowler if you cannot implement unit testing in your platform / language.

That applies to web gui / work flow code, embedding realtime programming. It also requires extra effect to do *automated* unit testing in language like C. Both dejagnu and check are difficult to use on win32 platform IMHO.

But JUnit / Java and Python rocked ! Suddenly you become very confident in making the changes, because you can always verify the behaviour with just a click of a button and examine what you have overlooked.

I wonder how Joel tested his web apps when he refactored. Let's say you use extract/move method to transform a block of messy code into a class, how do you do the regression test to make sure that it behaves exactly the same ?

1. User fills out a form in the new bug page, click the submit button, data goes into database, operation succeed message is printed...

2. User fills out a form, new code shouldn't let any checked exceptions slip through, OTOH, it shouldn't block any well-formed data...

Without an easy way to automate unit testing in our C code, I dare not touch the code written by the team, despite how ugly it looked and how hard to add features on top of that.

Anyone can share experience on that ?

Mac
Friday, January 25, 2002

Heres what I think...

Rub-A-Dub-Dub
Three men in a tub,
And how do you think they be?

The butcher, the baker, the candlestick maker. 
Turn 'em out, knaves all three.
And how do you think they got there?

They all jumped out of a rotten potato,
'Twas enough to make a man stare.

Tony
Friday, January 25, 2002

Well, it is possible to do a refactoring without automated usint tests, but in this case it is much harder to fix mistakes you doing on your way.
Suddently you find yourself in the debugger spending your time not very efficiently.

I found that there is no such a big problem to refactor messy code that have no unit tests.

Here is my experience. We decided to start cleaning one hairy subsystem. Of course we did not had any unit tests. We choose one single class and decided to refactor it. In several minutes we found that it is almost impossible to test this class separately from others - there were so many dependencies.

So we ended with quick test for the whole system - something that called smoke test rather than unit test. Then we did a little refactorings until we were able to instantiate our class separately from others. At this point we wrote a unit test for this class (a simple one) and moved further.

It is not that hard and after some time you will have unit tests all over the place.

Roman Eremin
Friday, January 25, 2002

Hm, my first reaction to the article was "when this guy writes whole articles about re-using code why didn't he take one of the 1000 open-source bug tracking systems instead of writing his own from the scratch"? It would have made sense if there was the intention of selling it from the beginning (because with GPLd code that wouldn't make sense), but otherwise I cannot see any reason to start something like this.

Tim Jansen
Friday, January 25, 2002

In my experience too many unit tests slow you down.
In many cases the unit test doco procedure is more time consuming that the code changes.
Its sometimes very hard to make this point in a politically correct world of unit tests, review, system test etc.

Good developers are worth their weight.

Tony McConnell
Friday, January 25, 2002

If you want to test on a variety of platforms and browsers, head over to http://browsers.evolt.org
You can then download every browser ever released. Everything from Netscape 0.6 is there - and for every platform.

Full disclosure: I am one of the admins for www.evolt.org , the site that hosts the above browser archive. Evolt.org is a community of web developers, and our mailing list has over 2600 members. However, it is run purely as a non-profit volunteer effort, and none of us receive any monetary compensation for our efforts. So I'm not advertising here :)

MadMan
Friday, January 25, 2002

For templates in PHP, there a lot of systems.
One is FastTemplate (old) and there are some new ones.
A guy made a cool content management system based on templates and PHP. It's called sitellite (www.simian.ca).
I happen to like the way it is done (classes, clean code, XML doc comments).
He releases the source code if you want to read it.
And it's interesting in my view.

Philippe Back
Friday, January 25, 2002

Tim,

Take a look at the almost contradictory article 'In Defense of Not-Invented-Here Syndrome'

http://www.joelonsoftware.com/articles/fog0000000007.html

I think that the issue is about keeping in control of your software, not reuse.

Ged Byrne
Friday, January 25, 2002

The people who originally wrote Netscape 4.x never had the same goals as the one who work for the mozilla.org project.

You see, it's not as easy as you often suggest in your articles.

The first task for writing Mozilla is to implement full support for DOM0, DOM1, DOM2, DOM3, CSS1, CSS2, CSS3, XML, XML-RPC, SOAP, HTTP, HTTPS, HTML (all versions), XHTML, XSLT, XSL-FO, Gopher, FTP, POP3, IMAP... (and of course, lots and lots more - you get the point.)

Now add that the code for these standards needs to be quite simple to maintain and to implement on all platforms that you intend to support (the solution is cross-platform code), it has to actually /work/ and compile on all those platforms as well.

Also, you need to implement support for the hundreds and hundreds of quirks that people take for granted daily on the web -- the beast needs to be compatible too.

Another thing that needs to be taken into consideration is that it has to be fast, and perform well. Given the constraints above, this is not easy - you need to write *much* better code than regular developers who write native applications.

Unlike some software specifically targeted at programmers, your application needs to be easy to use.

Obviously any given non-Microsoft company can't do all of this by itself, so you decide to make it Open Source, and so that other people can benefit, reuse and help fix bugs in the code. Think about all the management, and the organization, the tools and everything you need to do to make the development go on smoothly among programmers all around the world.

Yes - you must have an extremely well thought-out architecture and organization to build such a beast, it is not easy.

Please think twice before you lament such hard work done by thousands of people.

Thanks,

-hbw

hbw
Friday, January 25, 2002

but Microsoft seems to have produced a product just as you describe, but without rewriting IE from scratch or open-sourcing the code.

cop
Friday, January 25, 2002

I hear the problem is that Netscape could not hire top-quality programmers.  Instead, they had a few of them, and hordes of less-qualified ones to make up for it.

Whereas, Microsoft had over a decade of growing slowly and selectively.

Therefore I think that Netscape is more a convenient punching bag than anything else.  I agree that writing from scratch is usually an enormous and arrogant mistake.  However, it is stretching to say that Netscape died because of any particular bad decision.

Mike Reynolds
Friday, January 25, 2002

Nice article, and I see Joel's point.

However (you saw it coming :), I doubt most software projects could do a line-by-line examination of their code in three weeks. 

Especially if a team of 15 people took 3 years to develop it.  We're talking hundreds of thousands SLOC here.

What if the original developers are gone?  what if the nastiest pieces of code are PERL -- multi-megabyte .pl files.

I am honestly interested if anyone has done a refactoring effort on a larger scale.  At some point, re-architecting the code must make sense.

I agree the "scrap it and redo it" approach is taken too often, but Joel's example doesn't seem appropriate for large projects.

Stephen Johnson
Friday, January 25, 2002

One thing I founf in refactoring...Start with your ugliest class.

You needto be in a refactoring mindset:  How can I do this right?  You can't be rushed to get a feature implemented, you'll cut corners.

I would love to be able to spend 3 weeks and do nothing but refactro my project...but I can't yet justify the time and expense to my Boss.

For Http Smoke tests (in Java) use Http Unit.  If you are uncomfortable with Java, there are frameworks for varisousother languages.  On a Unix system, you can do amazing things with wget.  I'm sure there are MS options as well, but I haven't had top work in that world for web based development in a long time.

Back to the ugliest calss thing...Go to your longest method, extract logical chunks into methods.  After a couple of these, you should be able to extract a mehtod, and reuse one that you extraced elsewhere.    What will happen in you mind is you will develop a better pattern of how to solve these kinds of problems else where, and you will want to extract new classes to help out.  That's when you'll want to get into some of the more obscure refactronigs like replace conditional with polymorphism.

The companion book to refactrouing is the Design Patterns book.  As you think about restructuring code, you now have a set of targets where to go to.  I find Design Patterns work much better as refactroing targets than for Big Design Up Front, mainly because they are so enticing.  You may over design up front, but you are less likely to do so during refactoring.

Adam Younga
Friday, January 25, 2002

We wasted 8 months building a v2 from scratch and finally made the right decision and stuck with the original system (which has scaled from 0 accounts to 13 million). Going 8 months without any major service enhancements nearly killed us. It's really difficult to exagerate how dumb rewriting a system is in the vast majority of situations. The only good examples seem to be systems that were horribly implemented in the beginning but most of us don't have to deal with that.

pb
Saturday, January 26, 2002

If you translate XML with XSL server-side, then the output is pure HTML, therefore you don't have to worry about browser compatibility. Am I right?

anon
Monday, January 28, 2002

Good article Joel, but I wonder if you've learned enough from your refactoring to take your programming to the next level where you don't have to "scrub" your code at all, and you don't code yourself into such a corner that you have to work on it for three weeks until "every line of code is different now?"

In your article you said:

----
While David worked on version 2.0 we honestly didn't think it was worth that much effort, so he tended to do things in what you might call an "expedient" fashion rather than, say, an "elegant" fashion. Certain, ahem, design issues in the original code were allowed to fester.
----

<RANT>
Arghhhhhhhhh!  No, no, no.  David (and others at Fog Creek, it seems) have made a terrible mistake, especially given that this is 2002, and people have been hashing this issue OVER AND OVER for years.

The myth that doing something in an "expedient" fashion (can you say "just hack it?") is faster than doing something "correctly" is just that: a myth and totally false.

<SUB-RANT>
And to be perfectly frank, I would never hire someone who believed that myth (just as I wouldn't hire someone whose C++ code had GOTOs or pointer arithmetic in it, or rolled their own link list class, paying no heed to STL; e.g. someone who just DIDN'T GET IT), for it simply isn't faster, and I would expect any professional, supposedly experienced developer, to know so.
</SUB-RANT>

If you follow good practices you can write your software correctly and go just as fast as the guy who is hacking it.  The difference--you both get it done, but you have unit tests and a fluid OO design capable of going where it needs to go, whereas he has a "Big Ball of Mud" that he can't change without a terrible gnashing of teeth.

Imagine writing ALL of your code in the same fashion that you did during your three week refactoring: at the end of the day it is always shippable, you can test it, you're confident that you're not adding new bugs, etc.

It is totally possible.  All it takes is discipline.

My motto, BTW, is "shut up and ship!"  Get the software done and get it out the door.  But you can do that AND do it right.  I rant because most developers don't seem to think so, and I end up looking at (and refactoring) their shoddy code, knowing that they are probably smarter than me, and certainly capable of producing better work.  Sigh, but it was ever thus.  :(
</RANT>

Sorry about the rant, but I run into this mindset (let's just do it quickly now and inspect all of the bugs out of it later, or, we're only going to use this code once and then it will change so we'll just hack it and rewrite it later...<cry> it is sooooooo stupid) time and time again, from guys who think they're really good coders.  Laugh.  :)

Sigh, but I guess it worked for Microsoft.  :)

M$ aside, and to repeat myself again for more effect--you CAN write solid, well tested, OO code, FROM THE BEGINNING, and go JUST AS FAST as the hacker.  All it takes is discipline and some skill, which can easily be learned--and after a while, not even that...just taste, aesthetics, and experience (all of which I would expect from someone who had been a professional developer for several years).

From the XP camp, I think Kent Beck summed it all up best when he said: "I'm not a great programmer; I'm a good programmer with great practices."

I would hire a good programmer with great practices over anyone else any day of the week.

Wouldn't you?

John Lindsey
Tuesday, January 29, 2002

John Lindsey,

I like that line of thought.  But on one hand, good software is an iterative process.  Even TeX, which is probably the least buggy, non-trivial, widely distributed program, had to undergo a total rewrite.

Joel has the added constraint of his business realm:
http://slashdot.org/comments.pl?sid=24483&cid=2658454

If you look at the consumer software industry in one light, a lot of it is structured as having people pay for continued development of a product.  So people don't pay for a finished product, they pay for an iteration. 

Of course, I am a fan of measuring twice and cutting once, and I definitely think that commercial software often leads to bad, stagnant technology.  But to give Joel credit, FogBugz was not intended for external release, was his way of learning ASP, and a bunch of other oddities conspired to force refactoring.

People make mistakes, and aren't omniscient.  A good system needs to be self-healing.  And organic.

Ringo
Tuesday, January 29, 2002

By the way, that last comment was contingent on Joel not having been responsible for VBA's security model specification.

If he was, then I'll rethink my position on having more foresight.

Ringo
Tuesday, January 29, 2002

*  Recent Topics

*  Fog Creek Home