Fog Creek Software
Discussion Board




What do you think of Lisp?

I've read of few of Paul Graham's essays on Lisp and he seems to think that start-ups today should be using Lisp because it will help them right cleaner code faster.  I believe this may have been true back in '94ish when Paul built what is today Yahoo! store.  However, it seems that today other languages/platforms (.Net, Java, etc.) have tons of libraries that would allow most coders to bring an app up pretty quickly.

Are these libraries (database connectivity, GUI stuff, Internet connectivity) available for Lisp?  If not, I would think that any gain in programmer productivity would be eaten up by having to roll too much of your own stuff.

How do you see Lisp being used in the future?  Is there enough library support, or will Lisp mainly be used for applications where programmers would want to roll most of their own stuff?

Chris
Wednesday, February 25, 2004

The people have voted with their feets. I can count on one cleft-hoof the number of serious web systems built with lisp (the Orbitz engine and Yahoo! Stores, which has been rewritten in a modern language -- see http://discuss.fogcreek.com/joelonsoftware/default.asp?cmd=show&ixPost=31402). Software developers have never used lisp outside of some tiny niches.

I wrote a little rant about one misfeature in Yahoo! Stores once.

http://www.joelonsoftware.com/news/fog0000000322.html

I'm pretty sure the reason Yahoo! Stores couldn't survive the "back" button *because* it was written in Lisp and used closures for state. This is my interpretation of Paul Graham's article at http://www.paulgraham.com/avg.html , but I don't have enough information to say that for sure.

Paul Graham is brilliant and I'm sure lisp was great for him and his team, but I think most of the productivity of lisp came from the fact that it's garbage collected. I don't see any reason why a lisp programmer today would have a productivity advantage over a C# or Java programmer (and yes, I know lisp.)

Joel Spolsky
Fog Creek Software
Wednesday, February 25, 2004

Of course that's what a typical Blub programmer would say.

rob mayoff
Wednesday, February 25, 2004

The one thing that makes me feel more productive in Lisp vs other "modern" garbage-collected languages is the ability to pass around untyped functions as data and build calls to functions on the fly. (you can certainly do this with Java/C# "reflection" type features, and sort of do it with C++ templates at compile time, but it's more straightforward in Lisp). That and the syntax of Lisp can be learned in about five minutes.

However, you can code yourself into a corner in any language :)

Dan Maas
Thursday, February 26, 2004

I never said there was anything wrong with having closures in a language. I said that there was something wrong with using closures to maintain state in a web app.

And I have the ultimate respect for Paul Graham -- I think there's a good probability that in a year or two we will credit him with being the man who solved spam. But I think that if you try to ignore the fact that millions of programmers around the world have learned lisp and don't prefer to use it, you're in the land of morbid cognitive dissonance. And this attitude that "lisp is only for leet programmers so it's good because only l33t programmers will work on our code so our code will be extra good" is just bullshit, I'm sorry. Plenty of brilliant programmers know lisp just fine and still choose other languages. Most of them, in fact.

Joel Spolsky
Fog Creek Software
Thursday, February 26, 2004

I still think of APL as the most underrated language. For complex matrix operations, it was great!

SG
Thursday, February 26, 2004

Phillip Glass has an article in "Software Disasters" about a CAD system that was written in Lisp in the '80's.  It could never get sufficient speed to succeed.

Given Moore's law, it still seems the speed of increase of our desires for 'intelligence' and features in our code outstrips the speed of increase of the hardware.  Thus 'efficiency' of the languages used remains a critical factor.  C/C++ hold the performance lead here. 

I don't want to restart the language wars, so I'll just say all languages have a trade-off between the language constructs and the speed of execution.  For the product produced in a language to be commercial, the language has to be able to produce a product that runs fast enough for your customer.

AllanL5
Thursday, February 26, 2004

Maybe a little bit off-topic but you all seem to know about lisp.

I've been tempted to learn/try out lisp for a long time now as it seems to have a lot of cool features and to be very flexible. However (and I know this is somewhat superficial but still) I really can't stand the syntax with hundreds of nested parentheses, the fact many of the keywords seem rather cryptic and dated, and the fact (I think?) it doesn't seem to like infix notation for common arithmetical operators. It just seems like too much of a chore to parse this kind of syntax in my mind, after getting used to prettier-looking languages.

I guess I'm just wondering if there are any versions of lisp which use more conventional syntax (ideally something more C-style or python-like) - maybe just a pre-preprocessor would do here - or similar more readable languages with better syntax, that anyone can recommend?

I always thought using closures for web applications sounded very cool/elegant as you could structure the code in a linear page-by-page fashion requesting data from the user when you need it, rather than having separate code run for each page load sort of thing... but i've not tried it, and I can see how it could screw up the back button. Surely someone can figure a way round that though? in the cases where the back button should work (clearly it shouldn't reload a lot of web application pages anyway, pages that 'do' things that shouldn't be repeated).

Ultimately it's this ugly dissonance between the web browser model which was really designed for static content, and web applications which are really trying to implement a GUI for an application in a medium which isn't quite suited to it...

-Matt

Matthew
Thursday, February 26, 2004

Agreed. And then the lisp defenders would argue that it's theoretically possible to write efficient code in lisp, as long you avoid "cons," which is at the heart of everything that makes lisp a great programming environment. So you get the worst of both worlds: lisp syntax without functional/recursive/list-based programming.

And what I REALLY can't stand is the idea of cons-less programming in lisp. It's like carbohydrate-free cupcakes: probably possible, almost certainly disgusting :)

Joel Spolsky
Fog Creek Software
Thursday, February 26, 2004

I had the pleasure of eating lunch with Paul Graham last year and got a bit of insight into one of his newer projects, ARC. I was more or less impressed with what I heard - it sounds like he's addressing a lot of the "features" that make dialects like clisp a pain to write practical code for. For completeness, and because he does a better job of describing his language than I do, you can read his thoughts at his web site:

http://www.paulgraham.com/arc.html

Interestingly enough, on a somewhat unrelated note, my sense is that a lot of functionallanguages resort to assorted tricks and tomfoolery to attain low-level efficiency. I think that the accepted wisdom is that the best way to compile HasCl is to cleverly convert to c and run gcc.

Matt Gline
Thursday, February 26, 2004

I thought the problem was that he used continuations (his own continuation hack), not just that he used closures per se.

K
Thursday, February 26, 2004

I learned Lisp in college and it has some nice aspects.

However, even if I thought it was the best real-world solution to any programming problem I had, I'd probably still avoid it simply because in my experience the people who do use it are a bunch of boorish jackasses with huge persecution complexes.  A sure sign that the technology they are supporting is marginal, at best.

Mr. Fancypants
Thursday, February 26, 2004

(Tayssir's comments were deleted at his request. I thought they were a valuable contribution but will honor that request.)

Joel Spolsky
Fog Creek Software
Thursday, February 26, 2004

+++For the product produced in a language to be commercial, the language has to be able to produce a product that runs fast enough for your customer. +++

This is really the only performance that ultimately matters. 

I've been asked in interviews about the performance characteristics of a Mutex vs. Critical Section and which I use for code that "needs to be fast."  This is utter crap. 

While having a good understanding of low-level constructs *IS* important (link to Joel's article here), worrying about them when they're not a problem is a waste of the most expensive ingredient of software--the programmer's time.

If I had a dime for every time I patched someone else's prematurely optimized code, I'd be a ... well, I'd be able to buy a nice lunch, at least.  Maybe Knuth can afford dinner--I only get lunch.

For example, I went to edit some C code that stored string lengths (along with the strings they pertained to) in an array to "save on strlen() calls."  This is fine, if speed is an issue--but it wasn't.  This was an "anonymous" data structure (think argc and argv interlaced into one).  You couldn't edit these strings unless you also remembered to (manually) compute the strlen() and put it in the code, also.  Oh! the headaches are coming back...

Gabriel Krupa
Thursday, February 26, 2004

I don't see Lisp's syntax or speed (a good compiler will generate very good code) as being the reason it's not used much.  I have a much simpler theory: it's too damned expensive!

Let's say you want to write in Java.  Sun made this fairly easy: download the Java development kit (and even their open-source IDE) for free, write a program, compile it, run it, and distribute it to all your friends.  Life is good.

Let's say you want to write in Lisp.  Typically commercial Lisp implementations cost a several hundred dollars.  If you want to use CLIM (Interface Manager -- write graphical applications), that'll often cost you several hundred dollars more.  And often these are just "development" licenses: if you want to actually distribute your program, you need to buy a "distribution" license which can cost into the thousands.

I'm less familiar with Smalltalk, but from what I've seen, Smalltalk environments tend to be priced like Lisp environments.

(Sure, there are free Lisp implementations.  Most are really bad, and only serve to turn people off of Lisp programming.  The free GCs, in particular, are notoriously bad.)

These pricing policies pretty much ensure that no freeware/shareware/open-source/student programmer will ever pick up Lisp or Smalltalk for their project.  If you don't get the students today, you won't get the CTOs tomorrow.

Other software packages I can think of with similar pricing also tend to have niche markets.  It's not specific to Lisp.

I'm a Lisp programmer.  At least, I wish I could be.  I write mostly in C/C++ these days.  I don't think C++ is a better language, but for projects with small budgets, Lisp isn't worth it.

KH
Thursday, February 26, 2004

"(Tayssir's comments were deleted at his request. I thought they were a valuable contribution but will honor that request.)"

What is it with these open-source adherents?  They're so preachy and sensitive that they hurt their own feelings with their own posts.

chris
Thursday, February 26, 2004

KH: Well said.

There are several language systems out there that I would like to use to write some or all of the applications I am involved in creating.  Smalltalk, Lisp, and Eiffel would all be better than the VB crap and even the occasional Java that I work with now.  I've used each of these tools in the past (long ago jobs) and would continue to do so eagerly, now, if only they were affordable.  The company I am currently working for has a site license for MS products.  There is no way they'd even consider plunking down additional money for competing products.  I've asked.

ES
Thursday, February 26, 2004

Joel,

    If you think that closures don't work for web apps and the "back" button, I'd say that you should have a look at Seaside - http://beta4.com/seaside2/ - and Avi Bryant's blog:

http://www.cincomsmalltalk.com/userblogs/avi/blogView

with respect, I'd suggest that you have no idea what you are talking about with regards to this topic.

James Robertson
Thursday, February 26, 2004

"I never said there was anything wrong with having closures in a language. I said that there was something wrong with using closures to maintain state in a web app."

Using closures to maintain state is without a doubt the biggest leap forward for web-app developer productivity that anyone has ever made.  It does introduce some issues of its own, and it sounds like Paul Graham may not have solved them all.  This isn't surprising - getting the interaction right between a highly stateful web application and the browser back button is tricky.  It's taken a lot of iterations, but I would argue that I've gotten it right in Seaside.  In fact, Seaside apps tend to handle the back button considerably better than most apps based on more traditional session management.

I will admit that it would have been harder to build Seaside in Common Lisp than it was in Smalltalk - the ready access to the underlying raw bytes of Smalltalk objects makes it much easier to play games with state than in Lisp where closures are completely opaque.

I wrote a little bit about these issues here:

http://www.cincomsmalltalk.com/userblogs/avi/blogView?showComments=true&entry=3240140310

Avi Bryant
Thursday, February 26, 2004

ES + KP: Squeak Smalltalk is a free, open source Smalltalk with an extremely active developer community.  I and several of my colleagues use it for commercial development every day.  Overpricing definitely hurt Lisp and Smalltalk in the past, but it simply isn't a valid excuse these days.

If you need commercial support or native-GUI Windows development, there's also Dolphin Smalltalk, which is great and under $500.  I doubt many developers can't afford that.

Avi Bryant
Thursday, February 26, 2004

It's not just cost, though.  Java had the full might of Sun's marketing department behind it, for instance.  That is the first time I'm aware of that a language (platform, whatever) has been marketed in that manner.

Free implementations of C++ (like GCC) were available, but generally were behind the feature curve, so I don't think they did much to promote C++'s acceptance.  I remember many commercial x86 C++ compilers, most in the $50-500 range.  Now there are only 3, that I can think of (for the PC) that anyone seems to use: Intel, Borland, Microsoft.  Microsoft dominates, Borland is still hanging in there, and Intel seem to have the "we know these chips best" niche.

There are lower cost Smalltalk solutions around that are commercially viable (check out Dolphin Smalltalk -- www.object-arts.com for example), for example, so I don't think language acceptance has too much to do with cost alone.  The cost of Lisp tools could be a function of demand, rather than being causal.

It seems to me that C reached a critical mass, took over, and now any language that doesn't look or act like it has an unhill battle for mindshare.

Smalltalk, Lisp and the other more esoteric functional programming languages are, in my opinion, just too "hard" for the masses to learn.  By comparrison, declarative, imperative, statically typed languages work in a way humans seem more comfortable with:  "Given A, B, C:  Do this, then this, then this.  When this happens, do this.  What type are you?  Well, then you do this."

For example, try explaining to someone how Smalltalk's ifTrue:ifFalse: works sometime.  It is NOT just Smalltalk's way of doing an if/then/else.

Sports Illustrated (probably) outsells The Economist and Foreign Affairs combined.  Which makes you think more?  What sorts of movies are blockbusters?  Do most people spend more time watching TV or reading+learning+growing each day?

I may be wrong, but I think a lot of people don't "get" what Lisp or Smalltalk or whatever are all about.  C is easy, because it's basically a portable assembly with nice control structures and a good standard library.

Easy = good = popular, it would seem.

Gabriel Krupa
Thursday, February 26, 2004

Someone asked about a language with many of the features of Lisp but a normal syntax - there is one.  It's called Dylan.  I believe it was created by Apple in the 90s (?) in order to make the power of lisp available without the brain-twisting curve of (car (cdr '(1 2 3)).

Here's a link to an open source implementation
http://www.gwydiondylan.org/

Of course, Python has an awful lot of the feature of lisp anyway, and (I think) a brilliant syntax.  I prefer common lisp, though, for two fundamental reasons:

1) lisp macros (think C preprocessor with the full language at its disposal, but that barely gets at the surface) are the ultimate in language extensibility
2) CLOS - common lisp object system.  If you can stand a OO system with minimal enforcement of encapsulation built-in (like Python or Javascript), CLOS is just plain wicked.  Nothing else I've seen (python, java, smalltalk, objective c, etc) comes close to its power.  (Dylan, btw, copies some of its most fundamental ideas.)

A few random things to note here that some people reading this thread may or may not know (though many probably do):

Speed - Common LISP code, written with the same level of care that it takesto write fast JAVA code, can be close to C speed.  It isn't entirely true that you have to avoid CONSing...generational GCs supposedly have now made consing as fast as cons-less programming.  That said, it's not like C in that the obvious way to write a problem is probably slow.  Graham's argument (I buy it) is that you write a working program in LISP in far less time than C, and only then try to tune it, and the total time is less, with less risk.

Building standalone executables: LISP can do it, even open source lisp.  ECL does a fine job at this.  Interestingly, gcj does the same for JAVA.

Memory - lisp has a reputation for bloat, and some implementations (such as cmucl) still take up wicked memory...but the smaller ones (clisp and ecl, for one) take up less memory than most JVMs, and feel at least as responsive IMHO.

If anyone wants to learn more about LISP, I highly recommend Paul Graham's books (ANSI Common LISP for beginners, On Lisp for intermediate LISPers who want to learn what macros are really for...)

price: if you're building commercial apps (startup, etc) to sell, you may need to use a commercial implementation, but there are many of varying price (Franz is the most popular and expensive, there is also Xanalys's LispWorks, MCL for macs, and some others I'm not thinking of).  If you are running server-side apps, you can almost certainly use an open source LISP engine.  Cmucl sucks RAM but is very, very fast (cpu) and supposedly has a better compiler than the commercial lisps.  clisp has good unicode support, and takes very little ram, but uses bytecode (one of the only ones).  ECL is also a good compromise, though I found it a bit quirky (if you use it to deliver standalone executables they may need to be GPL...IANAL.)

While none of the open source solutions above have environments (IDEs), emacs and SLIME (lisp interaction mode), or ilisp mode provide fairly good IDEs.  (Good enough for me. May anger visual studio users.  Very different strengths and weaknesses.)

Francis Browne
Thursday, February 26, 2004

The argument that C-like languages are easier than smalltalk never made sense to me.  Smalltalk reads like English.  Which is easier to read:

self drawFromX: int  AndY: int  ToX: int  AndY: int

or

this.drawLine(int, int, int, int)

ES
Thursday, February 26, 2004

Oh.  Regarding database libraries, Guis, etc, all of this exists, though it  may not be as well-baked as some languages (I haven't used any of it, so I can't speak for it).  A good place to look for this stuff is the wiki at

www.cliki.net for UNIX open source common lisp software, or search archives of the newsgroup comp.lang.lisp (warning, serious flamage <g>)

Databases:
http://alpha.onshored.com/lisp-software/#uncommonsql

GUIs: cmucl (and probably sbcl, which is derived from it) has an environment called CLX, which I believe is an OO wrapper onto X primitives.  As someone mentioned, the classic LISP GUI environment is called CLIM.  There is an open-source implementation of that which seems to be making good progress at
http://clim.mikemac.com/
though it doesn't sound like it's ready to develop finished apps.

Someone's working on Qt bindings for cmucl (which would make it UNIX only)
http://www.cliki.net/Qt

Here's a good page of commercial and non-commercial common lisp implementations (hosted on Association of Lisp Users' wiki)
http://alu.cliki.net/Implementation

LISP seems to be at its strongest in programs that are calculation-heavy (CADs, research, modeling, AI, etc) and interaction-light, though plenty of GUI programs have been written on it (emacs and AutoCAD were largely written in LISP dialects, though not Common LISP).  Some vendor had a quote along the lines of "nobody's ever asked to use LISP to build another accountign system or something so well-understood, not that you couldn't..."  I can't find the quote at the moment, but it does seem that common lisp tends to flourish solving problems very new, aggresive, or poorly understood.  While new languages like JAVA and .NET's languages have accrued many of the benefits of LISP (esp GC and runtime typing, sort of), I still think there are fundamental things LISP kicks their butts at.  (However these do usually seem to help at things like complex modeling, simulations, etc...not interaction-heavy, computation-light programs like most off-the-shelf programs.) 

Francis Browne
Thursday, February 26, 2004

I agree that Smalltalk _looks_ like it's easier to read, but the underlying principles aren't as easy to grasp.  Again, I bring up #ifTrue:ifFalse: as an example.  To fully grasp that the compiler doesn't do _anything_ special to give the language control structures (like if/else, while, etc.) is non-intuitive.

This, coupled with the concept of a running image (no source code files?  no compile/link/run?) threw me for a loop when my friend introduced me to Smalltalk.

I think most people also find:
  3 + 5
easier to understand than:
  (3 5 +)

I think it's a lot of little things like these that make the "hard" languages hard.

Gabriel Krupa
Thursday, February 26, 2004

> "What is it with these open-source adherents?  They're so preachy and sensitive that they hurt their own feelings with their own posts."

I have no idea what is up with them.  I'm not one. ;)  While I enjoy discussing it in the abstract, I do not think opensource is capable of fulfilling the world's software needs to the exclusion of commercial software.  However, it does help software companies, since it lowers the cost of the stack of software consumers must buy.  Otherwise software won't scale.  And new companies will be increasingly squeezed out.

I mainly asked them to be removed because a personal emergency made it difficult for me to focus properly and be a good representative of a community.  Even if I perceive myself to be just some guy with some info, people expect this role to be played.  Really, I would have liked to show why _I_ liked it, and even if anyone didn't agree they might see my POV.  But I failed.

Anyhoo, I put up this link that shows a couple lisp techniques that people may translate to their own languages.  It explains how lispers mitigate parentheses:
http://lambda.pentaside.org/article/symbols.html

Recursion and functional programming are actually not forced on the programmer.  There are actually lots of do/for/loop constructs in CL.  And you can make your own.  Iteration/state is good, fast, and every programmer knows it.  There is no one true style.

There is actually something that Paul Graham doesn't mention.  There are different lisps, and Common Lisp tries to let you do what the language implementor can do.  You have a disassemble() function, which prints the asm a function compiles to.  Also there's the ability to declare types for performance, such as when you're operating on small numbers which you guarantee will fit in a machine word.  Many features like this make Common Lisp a complete language, and if you need to write all this dirty optimized code, you can likely make a macro to serve as a facade.  For maintainability.

Oh, and I can dump the contents of memory to an image, and come back to it later like a videogame.  I think this was stolen from Smalltalk, I could be wrong.

Anyway thanks Chris, little insults like that always snap me to and give me a sense of humor.  And if I've misrepresented lisp, I hope someone calls me on it.  There is a LOT of history, which sometimes leads to arcane names, and learning stuff is never an all-or-nothing thing.  I think it's important not to typecast yourself as a lisp, java, whatever programmer.  That makes people incomplete.

Tayssir John Gabbour
Thursday, February 26, 2004

Hmmm. Joel, you seem to have a somewhat academic view of what "Lisp" is.  You say you know Lisp but you don't say which one. Lots of folks learn Scheme in college and think that's all there is to Lisp. Have you ever written production software in, say, Common Lisp? Modern-day Common Lisp compilers pretty routinely generate code that is efficiency competitive with, say, Java, and blows the doors off interpreted languages like Python. For many tasks, with proper declarations to help the compiler generate optimal code you can get within striking distance of C and sometimes even surpass it. And not by writing "cons-free" code. (It is true that you shouldn't use conses when they aren't appropriate--if you wouldn't use a linked list in some other language because of the big-O implications you shouldn't use conses in Lisp. Luckily Common Lisp has arrays and hash tables and all the other good datastructures  you might need built into the language. But getting efficiency in Common Lisp is mostly a question of using good algorithms and then providing appropriate (optional) type declarations.)

FWIW, I don't think it's just theGC that makes Lisp incredibly productive. I've done serious Java programming for much of the past decade--I was the Java language lawyer at Weblogic in the early days before we got bought by BEA--and I can tell you I find Common Lisp to be a much more productive language than Java for reasons that I won't go into here as I'm writing a whole book about it:

  http://www.gigamonkeys.com/book/

Cheers.

Peter Seibel
Friday, February 27, 2004

Here are some comments from a Common Lisp user (and former Scheme
weenie):

    
"But I think that if you try to ignore the fact that millions of
programmers around the world have learned lisp and don't prefer to use
it, you're in the land of morbid cognitive dissonance."

Where did you get this figure from? There are (and always have been)
very few institutions that attempted to teach Lisp, and those that
have almost always taught it wrong. I personally know of not a
_single_ person who knows Lisp, and I've looked (I'm currently
studying at the University of Calgary, which has a fairly large CS
department).

"I know lisp"

With a statement like the following, I am having doubts:

"... "cons," which is at the heart of everything that makes lisp a
great programming environment. So you get the worst of both worlds:
lisp syntax without functional/recursive/list-based programming."

This is an argument of ignorance. You seem to think (and presumably so
do your millions of brilliant programmers) that linked lists is the
only (good) thing about Lisp. I hear this from a great many people who
claim to know Lisp, and unfortunately this is almost always the result
of the above stated miseducation.

You should also realize that heap allocation has nothing to do with
pseudo-functional programming or recursion. By your own argument, you
should be discouraging the use of C# and .Net, as "theoretically,"
memory allocation and garbage collection have to be avoided to write
efficient code.


I'm going to address some of Joel's examples. Orbitz the website is
_not_ running Lisp, and it's not their search engine. ITA Software
runs their airfare search engines on Lisp, and provides them to
Orbitz, among others. Although I haven't used Y store, I highly
suspect that the back button example is bogus - from the screencap,
the tour sounds like a Javascripted slide show (it would be nice if
someone could confirm this - currently, the Yahoo Store tour is some
multi-megabyte multi-media monstrosity).

Also, it should be pointed out that Joel is confusing continuations
and closures. What Paul Graham raves about in his articles (and
explains in _On Lisp_) is that closures let him fake continuations in
Common Lisp. I really would have to agree with Avi - using
continuations to manage web sessions is an incredible leap forward.

PS - It's possible to (de)allocate safely without a garbage collector
(but not without using macros or writing your own language). Check out
Thinlisp. http://sourceforge.net/projects/thinlisp/

PPS - I hope some people will decide to (re)educate themselves about
Lisp as a result of this discussion. There are quite a few resources
to get one started. http://www.lisp.org/alu/res-lisp-education.clp

Vladimir Sedach
Friday, February 27, 2004

The Lisp crowd may seem like a caustic and bitter bunch of people, but quite frankly, they have good reason to be.  What they take issue with doesn't even have to do so much with Lisp itself as it does with a common tendency in computing circles for people to ignore history.  What am I talking about?  The average programmer today has almost no knowledge of the past in computing.  A good half century of development and all people have today from the past is a vague idea of punch cards, vacuum tubes and vast machine rooms.  Between that distant past and the present, almost everything else is ignored.  Even though it happened only a rough 20 years ago, for example, it is the rare person who knows what caused the rise and fall of the AI companies.

So what, you think?  Well, the old adage says that those who ignore history are bound to repeat it.  And this is apparent even if you look at ideas that came within years of one another.  Think NextStep versus the modern IDE.  Nowadays, fans of Cocoa know almost without a doubt it is a powerful development environment.  However, only a few years ago Objective-C and the NS frameworks were really no more popular than some of the "fringe" languages of today.  Now if you accept this, then so too will you have to accept the fact that the same phenomenon happened with some of the fringe languages of the past versus the mainstream languages of today.  Recently, for example, at an EclipseCon, Grady Booch was quoted as saying, "For those of you looking at the future of development environments, I encourage you to go back and review some of the Xerox documentation for InterLisp-D"  ( http://home.comcast.net/~bc19191/blog/040205.html ).  Why is this?  Because Lisp environments made significant advances in ways to help the programmer do their job, which is also true of Smalltalk.  Meanwhile, the rest of the world--perhaps because of the rapid rise and relative youth of the IT industry--was all too happy to simply shut out these developments and recast the world in terms of their own favourite languages, as if it were a matter of religion.

This is deliberate ignorance.

To every one in their own local context, what they do always makes sense; to a markup languages person, inventing XML and XML-databases make sense.  But to the person versed in the relational model, it is a naive throwback to the past that shows how much more worth hype has in this industry than actual progress.  And in programming languages, there is this silly sense of concept "foo" to the exclusion of all others.  Whether that foo is C, C++, Java, C# or even the concept of OOP itself, the only difference is the time period you observe.  In many ways it's as if in our fight against complexity, all we have come up with is ever so slightly bigger sticks and stones--that is, when we can even be bothered to fight because we're all too busy chasing shiny objects on the horizon.

So what do you think of Lisp?  Learn it, use it for something more than just a toy program and see for yourself.  You owe yourself as much if you actually care about programming as a craft.  It is a crucial part of a long heritage that we should stop refusing to inherit.  Any less, and we will forever be reinventing the wheel, thrashing about trying to improve the world of programming.

MW
Friday, February 27, 2004

When I read Paul Graham's essays, I wonder if I'm missing something by not programming in LISP.

However, I've known lots of smart people, including my wife, who attended MIT and learned Scheme. Most of them thought that the language concepts were interesting, but they are perfectly content with other languages.

For whatever reason, it seems that only a fraction of the people exposed to LISP are drawn to it.

Julian
Friday, February 27, 2004

Hmm. Now that Joel has posted a number of rather uninformed opinions about Lisp, I can safely ignore Joel's essays, books and software.

Sorry, Joel. It's not your opinion that I mind (everyone is entitled to one and I respect that), it's
the lack of knowledge that shows from what you
wrote.

Jan
Friday, February 27, 2004

I have used LISP during 10 years to build an expert system. I really liked the language and my productivity with this language. It was really powerfull to build a complex system. However, after 10 years, I switched to C++ and then to Java. You know why? Try to search a Job in my country (Belgium) as a Lisp programmer: nearly no chance. With C++ or Java, I have more choice and I prefer to choose the company and the people I work with than my programming language.

Moreover, if you decide to build your system with Lisp, you will have less choice when finding people to work with you.

Laurent
Friday, February 27, 2004

The Squeak website is offline.

Alex.ro
Friday, February 27, 2004

If you think Lisp is too expensive, write your own interpreter. That's how I learned the language :)

Lisp is probably the one non-trivial programming language you can write an interpreter for in one sitting.

Dan Maas
Friday, February 27, 2004

"It seems to me that C reached a critical mass, took over, and now any language that doesn't look or act like it has an unhill battle for mindshare."

Seems like a good argument. But than again: Why is nobody using "The D Programming Language"?

http://www.digitalmars.com/d/overview.html

It's not that it doesn't exactly address the shortcomings of c, c++ and java AND has a c-like syntax, a very robust compiler and a very, very good feature set. The compiler source is even GPL'd. But it doesn't have sun's marketing dep. behind it. So it is doomed to a niche. I have just only discovered it myself and plan to start using it for programming modules for my Ruby programs. Using Ruby with the most beautiful syntax and D for performance is my preference. Although I still want to learn Lisp because of the learning experience. It is all about the continous learning anyway, isn't it? Only after you learned everything you can effectively decide what is best for you. Only problem is that the learning nevery stops if you are serious about it. That applies to everything not only to programming.

Sascha Ebach
Friday, February 27, 2004

I'm rather undecided on Lisp but I >love< CLOS. Does that make sense to anyone else?

I also feel that a lot of advanced language development sometimes feels like a reinvention of Lisp.

Lisp offers unsurpassed power in writing obfuscated code, and its parctitioners sometimes seems seem to keen to challange the C programmers desire prove.

Lisp development environments seem to suffer the curse of the retaining head start: they used to be very far advanced of the competition, got leapfrogged, and now can't seem to break loose of their past.

Just me (Sir to you)
Friday, February 27, 2004

Language advocacy: Check out OCaml www.ocaml.org
It's a modern language that: supports procedural, functional and object-oriented programming; has an efficient garbage collector; offers closures and first-class functions; has parameterised modules (like C++'s templates 'on steroids'); provides automatic type inference (so no casts are required); does not use pointers (for safety); has bounds-checked array access (which can be disabled if needed); offers an interactive interpreter, so you can figure stuff out interactively; compiles to bytecode that is portable over differenct machines and OSes; compiles to native code (with performance close to, and sometimes better than, C); is available for a range of platforms, including Windows, UNIX, Linux and Mac OS X; a powerful pattern-matching mechanism (like 'switch-case' on speed), that points out conditions you haven't considered at comile-time; provides an exception mechanism; comes with a range of libraries (such as arbitrary precision arithmetic, multi-threading, networking etc.); comes with a UNIX (Posix) programming environment (even on Windows); dynamic linking, so you can recieve compiled code from a remote location and then dynamically link and run it; can be used as a web scripting language for Apache via mod_caml. It's had most exposure within the academic community so far. A few "high-profile" applications written using OCaml exist (the Unison file synchroniser, system management tools for the Lindows Linux distribution, NBCi shopping and the code generation for the FFTW high performance discrete Fourier transform library). There is a forthcoming book on OCaml from O'Reilly.

C Rose
Friday, February 27, 2004

Regarding D: I'd love to use D instead of C.  It looks like a great language; it's a compiled language that has many of the nifty features of the great scripting languages built right in.

But it's still a work in progress, and doesn't run on the platforms I use.  Some day, maybe.

Will Duquette
Friday, February 27, 2004

I'm surprised that no one has mentioned Tcl.  The syntax of Tcl is remarkably C-like, but you can do a lot of Lisp-like things in it.  My own "Snit" package, for example, adds an object framework to Tcl in such a way that it looks like it's built-in to the language. See http://www.wjduquette.com/snit for more.

Will Duquette
Friday, February 27, 2004

My own take: LISP isn't a wonderful language for modern
programmers, but it _is_ a wonderful language and
representation for complex, ad-hoc structures.

The "S-Expression" data structure is compact, fast,
and can represent arbitrary structures better than
pretty much anything I've ever seen.  A good way to
think of LISP is as a far less wordy ancestor of XML;
in one project, we used XML for the "external" interface
for a system and S-Expressions for the internal interfaces;
we converted XML to S-expressions for passing between
our own modules on an internal network.  It was far
faster and 10% the size of an all-XML solution, and a
simple S-Expression parser can be written in literally 100
lines of C.  This simplicity is why LISP or simplified variants
is quite popular in embedded systems or for things such
as parse-trees or execution/evaluation trees.

So, my own feeling is that programmers wouldn't
hurt themselves by learning about S-Expressions and
some concepts of LISP in general, even if they never
actually write a program in the language.

x
Friday, February 27, 2004

>I think most people also find:
> 3 + 5
>easier to understand than:
>  (3 5 +)"


Yeah, whew, took me years to learn that on my HP 48
calculator.

I must truly be not worthy of lisp.

Let me kneel down and wash your feet with my hair,
oh great Lisp masters.

My language can beat up your language.

Sorry, this has not been a thoughtful contribution to the thread.

jorge fortuno
Friday, February 27, 2004

Wow, what an excellent discussion -- informed, mostly temperate, full of interesting nuggets and located on a useful axis of computer science. I for one am grateful to all the really smart people who contributed to it.

While I'm probably not about to revisit or increase the incomplete understanding of Lisp I gained in college over a decade ago -- any more than I'm going to do any serious coding in the free Haskell clone I keep around out of nostalgia for functional programming -- it's important to be regularly reminded of the depth and breadth of the history of the art, and the folly of (as MW put it) "chasing shiny objects on the horizon."

So: thank you all.

Matthew Caldwell
Friday, February 27, 2004

"I think most people also find:
  3 + 5
easier to understand than:
  (3 5 +)"

Actually, what you've written is postfix notation which is exactly what engineers use every day for solving real math problems.  Once you've used an RPN calculator long enough to get past the initial learning curve you will forever after be extremely annoyed anytime you have to solve anything moderately complicated with a standard calculator because you'll be forced to write down intermediate results on paper instead of just keeping them on the stack.  In Lisp, the expression would be written as:

(+ 3 5)

Incidentally, one of the most popular symbolic math programs, Mathematica, was written in Lisp and uses a Lisp-like syntax, although it also supports infix notation.

anon
Friday, February 27, 2004

My observation on lisp is this:

The pro-lisp side primarily argues through appeal to authority. For example:  Paul Graham says lisp is great, Paul Graham is smart, therefore lisp is great!

The anti-lisp side argues mostly through appeal to poularity. For axample: If lisp is so great, why does VB or Java have 1000 times as many users?

We see that sort of thing within 2 posts here. ;)

Sum Dum Gai
Saturday, February 28, 2004

My opinion of Joel's writings has just gone way, way down, given the number of flat-out wrong things he says in this discussion. He does sound sure of himself though. Maybe all of his other self-assured writing needs to be viewed with a large shaker of salt?

Many of the errors have been addressed by other people, let me address this one:
" I think most of the productivity of lisp came from the fact that it's garbage collected. I don't see any reason why a lisp programmer today would have a productivity advantage over a C# or Java programmer (and yes, I know lisp.) "

OK, here are a few:
- macros
- more powerful object system (CLOS) than any other language
- functional objects (closures)
- incremental compilation; interactive evaluator (other interpretable languages have this, but not C# or Java).

Well, those are four major advantages; the synergy they provide is unmatched yet by any other language.

Yes, Lisp is not that popular today, especially for simple little desktop apps like those your company makes. I know it's a lot of work to make a good desktop app, but frankly, they're all pretty simpleminded. Lisp is used for applications that are HARD --  space telescope experiment scheduling, intelligent CAD systems, bioinformatics, natural language...complex tasks where you actually NEED the extra brain amplification you get from a powerful language.

Anyway, language wars are language wars, but please don't assume you actually "know Lisp" when you have an apparently superficial acquaintance with it.

Mighty Manfred
Saturday, February 28, 2004

Umm, if those little desktop apps are so simple from a Lisp perspective then why isn't every single one of them written in Lisp? Would be much easier, right?

And you do realize that your stupid argument could be turned around to claim that Ada and COBOL are much better than C++ and Java, don't you?

Chris Nahr
Saturday, February 28, 2004

"This is an argument of ignorance. You seem to think (and presumably so
do your millions of brilliant programmers) that linked lists is the
only (good) thing about Lisp. I hear this from a great many people who
claim to know Lisp, and unfortunately this is almost always the result
of the above stated miseducation."

I'd like to defend this opinion. Linked Lists and their handlind in LISP are the one greatest thing about it, at least for mathematicians.
Without trying to get into category theory (Bird and de Moor have written an excellent book about this and its relation to programming: http://www.amazon.com/exec/obidos/ASIN/013507245X/qid%3D918063395/sr%3D1-1/102-0736108-8472119 ), the idea is that with _any_ recursive datatype, like a list (cons or snoc is just a matter of taste), and a 'fold' function (dunno what it's called in LISP, Haskell calls it foldl/foldr, Bird calls it fold, Python calls it reduce) you can trivially transform it into any other recursively defineable datatype.
So basically, you define 'cons' and 'fold' and get anything else for free - if you can twist your brain around enough ;)

Mad Marty
Saturday, February 28, 2004

Also there is (at least) a theoretical possibility to unite Common Lisp and .NET and bring all the libraries exisiting for .NET to Lisp environment.

I've just started playing with this idea recently, and it seems that's possible ( http://www.common-lisp.net/cgi-bin/viewarch.cgi/clotnet@common-lisp.net ). Let's see.

Yurii A. Rashkovskii
Saturday, February 28, 2004

A lot of good discussion here.  As a lisp lover I still admit it's weaknesses...

1. Once you get macros you can't stand writing the 10 lines which equated to with-open-file in another language... and you start seeing more of these patterns the more you become familiar.

2. The lack of libraries is the real killer, if lisp could use the c libraries without effort it would be a spectacular step forward.

3. Changing languages because you don't like the syntax seems silly... assuming two things (which aren't readily available): if you have macros you change the language; if you have a RAD editor which abstracts away from monospaced characters and programmers having to worry about brackets or colons.

I think rewriting VMs to change syntax is impressively dumb - It reminds me that lisp was the first certified object oriented language.

Oliver George
Saturday, February 28, 2004

"Once you get macros you can't stand writing the 10 lines which equated to with-open-file in another language... and you start seeing more of these patterns the more you become familiar."

I always find it interesting that with-open-file and the like are often pulled out as the canonical examples of macros, because their only purpose is to hide Lisp's inexcusably verbose syntax for closures.  Ruby and Smalltalk both do with-open-file equivalents just fine without macros.  Here's Ruby:

File.open("foo"){|file| file.each_line{|line| ...}}

Avi Bryant
Saturday, February 28, 2004

(Aside) Python:

for line in file('foo.txt'):
    ...


(Back on topic). Can any of you Lisp supporters post any examples? There are an infinity of things which would probably be useful if I knew them, that I will never get time to/get around to exploring. What about Lisp(s) means it (they?) should be a high priority?

I prefer not to give my full name.
Saturday, February 28, 2004

I doubt the above python code is equivalent to with-open-file or to the Ruby code I posted.  At the end of the for loop, is the file immediately closed?  What about if an error happens inside the loop?

If so, there must be some ugly special casing going on.  If not, it doesn't do the same thing.

This is my fault for not explaining what with-open-file does.  The idea is that it abstracts away this pattern (written in no particular language):

file = open("foo");
try {
  do_something_with(file);
  do_something_else_with(file);
} finally {
close(file);
}

If you're being careful about file use, you will have that kind of thing all over your code.  The obvious thing to do is to write a function that takes as parameters the only two parts that change, namely the filename and the calls to do_something_with/do_something_else_with.  The filename will obviously be a string, and the do_something_with stuff should be expressed as a function that gets the newly open file handle as its sole argument. 

Ideally, your language supports some kind of anonymous functions, so you don't have to define a helper function each time you use this (and better still would be if these anonymous functions had access to the current lexical scope).  In lisp, using this function might look like this:

(open-file "foo"
    (lambda (file)
      (do-something-with file)
      (do-something-else-with file)))

This would be fine except that lisp hackers get tired of typing "lambda" all the time (which is the keyword to create an anonymous function), so they wrap it in a macro that inserts the lambda for them.  Then it looks something like this:

(with-open-file "foo" (file)
    (do-something-with file)
    (do-something-else-with file))

In Ruby or Smalltalk, anonymous functions are simply blocks of code surrounded by curlies or square brackets, respectively, so there's no need to add sugar with a macro.  In Smalltalk, for example, this might look like

FileStream named: 'foo' do:
  [:file |
  self doSomethingWith: file.
  self doSomethingElseWith: file]

In my opinion, this makes with-open-file a horrible example of why macros are good, because it highlights a problem with Lisp (it takes too much effort to use lambda, though nowhere near as much as, say, anonymous inner classes in Java) rather than an advantage.  It's nice that macros let you work around that problem, but wouldn't it have been better if the problem didn't exist in the first place?

Avi Bryant
Sunday, February 29, 2004

"In my opinion, this makes with-open-file a horrible example of why macros are good, because it highlights a problem with Lisp (it takes too much effort to use lambda, though nowhere near as much as, say, anonymous inner classes in Java) rather than an advantage.  It's nice that macros let you work around that problem, but wouldn't it have been better if the problem didn't exist in the first place?"

Well, that's one way to look at it. You may be right that WITH-OPEN-FILE is not a compelling example to people who are already deeply into a particular style of closure/block based programming. But I think it is wrong to imply that WITH-OPEN-FILE was written to work around a problem of lambda being to verbose. I think it's more accurate to say that macros are an extremely powerful tool that happen to be capable of pretty trivially providing a *different* solution to a problem that can also (as any Lisp programmer knows) be solved with closures. When people use WITH-OPEN-FILE as an example it's not because it's the most powerful macro in the world but because it does illustrate the power that comes from being able to create syntactic abstractions. That there are *other* ways of solving the particular problem that it solves doesn't show anything. Because there are also things that can be done with Lisp macros that are *not* easily translatable into functions on blocks.

In fact it might be more accurate to read causality in the opposite direction--because the set of problems that are nicely solved with concise syntax for blocks/closures is a subset of the problems that macros solve, there was less pressure in Lisp to develop a concise syntax for its closures. Or put another way, it *has* a nice syntax for closures--plenty of them: they're called macros.

And one last observation, Common Lisp is almost infinitely malleable, down to the lexical level. If Lispers thought a concise syntax for closures would add immense power to their language they could define a read macro (different than the macros we've been discussing) to get just such a thing. In fact some folks have witten those read macros  but they hasn't caught on because it doesn't actually add much expressive power.

Peter Seibel
Sunday, February 29, 2004

Avi, after reading your post, I spent a few minutes learning Ruby's blocks, as well as its Smalltalk influence.  Lispers don't think the way a Rubist may think they think. ;)

You seem to know some Common Lisp.  Use the macroexpand() function to determine what exactly with-open-file means to your lisp.  I've asked a few lisps, and they never used "lambda".  No human lisper would use it there either.

To the person who asked for examples of things to look out for:
- the error system is about creating protocols
- numbers are not limited to what fits in a machine word, but you can guarantee they are for speed
- you can do things like replace a function with a caching version of itself, with one command, so it does a table lookup for things it has already seen
- lisp's oop is very general
- "dynamic variables" are an interesting way to handle state and you may find them in things like Aspect-Oriented Programming
- you have enormous latitude in your personal style, you can often mix and match paradigms
- lisp programs are written in a data format that has all the power of xml, and a lot of features exploit that fact; that explains the parentheses
- a compiler/interpreter is just another program

Tayssir John Gabbour
Sunday, February 29, 2004

This is all my gut feeling, I appreciate others thoughts...

Macros make your code more declarative and as such more readable and maintainable.  Interestingly this is the arguement which leads everyone to show examples of how elegant their languages syntax is.  Because of macros Lisp has the potential to be 'readable' in a wider range of problems by *simply* changing the  language to suit the problem.

(As mentioned macros are specifically different from functions because of variable scoping... I agree that inline closures achieve the same goal.  Hell if you want to wrap your closure in a macro so it looks a little nicer... :)

Again I agree a nice example would probably be useful to illustrate this - can someone who uses lisp 'in anger' offer a first hand one?

Also I'm sure this is just one aspect of macros - but it's one which I really pine for when I write java, c or even php.

Oliver George
Sunday, February 29, 2004

Lost in all this competitive language arguing is the requirement for straightforwardness as a basis for reliable and for maintainable software.

For all the goodness that OOP adds to C++, it is still true, for example, that in plain C, you always knew what "a + b" meant.  In the more abstract languages, like C++, you never can be sure unless you start looking things up.

Similarly with powerful macro languages and mutable languages.  The poor code-reader hasn't got much of a chance.

And don't get me started on templates.  Anything that complex can hide bugs.  And hiding bugs is not what you want.

The cost of high-leverage features is obscurity. 

Warren
Sunday, February 29, 2004

Warren, that's such a bunk argument.  You're comparing sclar addition in C to potentially arbitrarily complex "addition" in C++.  Why don't you compare, say, the code for matrix arithmetic in C and in C++?  Yes, operator overloading can hide critical information when it's done wrong, but unneeded verbosity can obscure critical information as well.  When you've got a page of code that, in the correct language, could be expressed on a single line, you're not simplifying things.

One answer is to restrict operators to primitive types and just use functions for custom types.  One problem with this is that you write code that's still too wordy (you want matrix multiplication to be "cheap", and "matMult" is more expensive than "*").  Another problem is that operations which are applicable to types with '+' or '*' defined for them have to be rewritten without operator overloading.  This is the primary reason for templates (and I think that it's compelling -- it's greatly reduced the size and complexity of much of my code).

The answer to bad programming is education, not restriction.

K
Sunday, February 29, 2004

Only a single mention of Objective CAML so far.

My company writes web applications in OCaml because it gives us the best of the LISP world (closures, macros, GC, objects, functors, etc.), plus strong typing for safety, and speed.

We've also gone a long way with getting the database and libraries right - you can now write OCaml programs which call out to Perl or Python libraries, and there's a DBI layer for easy database access.  You can run your OCaml programs standalone or embedded in the Apache or AOL webservers.

http://www.merjis.com/developers/

Richard Jones
Sunday, February 29, 2004

Lisp is still a great language, but its real strength has always been that the representation of programs is the same as the representation of data. They are both lists built of conses. This  identity gives Lisp an extremely powerful macro facility for exploiting algorithmic patterns.

It isn't that Lisp has a with-open-file function. Any language can have a with-open-file function. What Lisp lets you do is recognize that you need something like a with-open-file function, or a pattern matching file processor, or a data space oriented back tracking mechanism and then build the language syntax as you recognize the control structures you need.

A classic example of this was Guy Steele's Lisp array manipulation language that, with some syntactic sugaring, implemented APL. He reprised this idea for the Thinking Machine.

Another good example is Winograd's Planner language for backtracking and data matching that spawned Prolog and a host of other more specialized languages.

I remember an early Smalltalk like package built on Lisp that tried to clarify how the early token oriented version of Smalltalk worked. You could watch how the various ideas in Planner, Smalltalk, and Simula slowly came together to build CLOS.

Lisp is still THE language to use when you have hard control structure problems. Once you understand the control structures you need, you can write a new language or extend an old language to handle them.

Having said this, I should point out that Lisp is not the language for everyone or for every project. Learning Lisp is actually easy. However, it is hard to learn to THINK in Lisp, recognizing and exploiting the data-program identity. Read the reviews of Sussman and Abelson's Structure and Interpretation of Computer Languages on Amazon and you can instantly pick out two classes of readers. You see them in this thread as well.

NOTE 1: Scheme, like Lisp, is powerful because it has a data-program identity, but it is not the same as the one used in Lisp. Lisp programs are lists and Lisp is the list processing language. Scheme programs are closure objects and Scheme is a first class closure and continuation manipulating language. Modern Lisp does support first class closures, but it retains its list processing roots.

NOTE 2: Someone mentioned continuations, so I figured that I'd point out that Alan Bawden demonstrated that reinvoking a continuation is the same as side-effecting data. It wasn't intuitive at the time, but it is another example of the program-data identity captured by a Lisp like language.

Seth Steinberg
Tuesday, March 02, 2004

Maybe it's just me, but this entire discussion seems very far off topic. People have made a lot of great points about the useability of Lisp compared to other languages, but having read all of the posts here, the one thing I haven't seen anyone comment on is the useability of Lisp for a specific function, which really should be the basis for most of the arguments here.

"I've read of few of Paul Graham's essays on Lisp and he seems to think that start-ups today should be using Lisp because it will help them right cleaner code faster."

That's a rather broad statement that anyone should be a bit doubtful of. Let's face it. Different languages are good for different things and are best used in different situations. If you were asked to write a web based application, what would you write it in? I think some obvious choices might be ASP, .NET, PHP, JSP, maybe Perl. I'm betting several others could come up as well, including Lisp. But at the end of the day, you're going to use what you feel is the best tool for the job unless some manager tells you that you don't have a choice (at which point you might want to consider looking for a new job, depending on his reasons).

Lisp is NOT an all encompassing solution to every problem, no matter what Paul Graham says. Nor, for that matter is C, C++, C#, Perl, VB, or any of several hundred other programming languages.

I'm familiar enough with somewhere between 10-20 different languages that I know better than to try and shoehorn C++ into a web app unless it's warranted (and it has been a couple of times). I also know better than to try and use VB for intensive mathematical calculations, or to use C++ for copying log files from one server to another.

In some of these cases, the complexity of doing the task makes choosing a specific language the case. Other times, the ability to modify code in place and run it through the command line or a job scheduler makes it best for the task.

Example: I think copying files from one place on a network to another on a regular basis as a scheduled job is probably best done with Perl. It's simple to write, fast to change, and if something goes wrong or an error occurs, adding additional debugging code is painless, especially if you didn't bother to error check every single little detail because your managers pressed you for time. In any compiled language, you need to find the source code, modify it, compile it, copy it to wherever it is running from, and then see what happens. If it doesn't work, repeat. Editing in place is the key advantage for Perl. (Don't point out archiving the original file. I'd think that would be pretty obvious)

Above all, I definately know better than to choose to use a language just because someone else said it is better for everything (yes, even Paul Graham or Joel), especially when I know over a dozen other tools I can use to accomplish the same task. Productivity is a big factor, but so is maintainability.

If you're productive with a language, and it's easy for others to maintain, then by all means use it. But if there are easier or more logical ways to accomplish your task at hand, then why make your life difficult? I can get to the north side of Boston from the south side by driving through New York, Vermont, and New Hampshire, but have I gained anything more than a scenic route? Chances are, probably not.

I couldn't possibly imagine using Lisp to write a web server, but I would definately consider using it for writing a spam filter, or anything that requires decision making on the part of the application where fuzzy logic needs to be applied or where the input types can change, depending on the meaning or context.

Joel chose to develop City Desk in VB, and has specifically avoided porting his code to .NET and will apparently not do so until his target audience already has the .NET runtimes. Read his reasons for doing so and use the same logic to determine what language(s) your project should use. Look at what you're building and evaluate whether Lisp is a logical choice. If it isn't, then don't use it. Simple as that.

So, back to one of the original questions, "How do you see Lisp being used in the future?". I'd love to hear some more discussion on this. I'm hardly an expert with Lisp, so I'd like to hear about some specific situations where using it would be a really good idea.

Mike Taber
Friday, March 05, 2004

You're right, this was more about those incendiary subjects everyone likes to comment on, and not focussed on the original question.

Two case studies on Common Lisp for practical webapps:
http://homepage.mac.com/svc/RebelWithACause/index.html
http://homepage.mac.com/nickypeeters/lispification.pdf

News, advice and commercial/opensource libraries at:
http://www.lemonodor.com/
http://groups.google.com/groups?group=comp.lang.lisp
http://www.cliki.net

On one hand, I also use any tool for the job, be it Jython, Fortran or lisp.  OTOH, I would like to see more lisp features make their way into other languages, because in many ways it's still very far ahead.

Take the errorhandling system.  Many languages now have exceptions, which destructively unroll the stack.  Joel explained he didn't use them because exceptions can pop out of nowhere.  Well, lisp errorhandling is more general; a protcol between parts of the stack.  Communication takes place over a safe, nondestructive channel parallel to the stack.  If you use a library's API, it will document a number of plans your code can call if something abnormal happens.  So as you use the API for some normal task, it might signal an error or a warning, and your errorhandlers may decide to call these plans.  Of course, your errorhandlers can also ignore them.

In the case of a disk-full error, plans may include deleting temp files or restarting the disk write.  Since the stack is preserved, the disk write can begin right where it left off, so rolling back previous actions is unnecessary.  Because these plans are from the code which does the work, just as if it were your employee, you can treat it as an expert and not break encapsulation.  You don't have to know the precise implementation in order to select the right plans.  So you can try silent methods such as deleting temp files, and if those don't work, you can pop up the abort/retry/save-elsewhere dialog box.  And a big win of this is you don't have to bake in the errorhandling strategy from day one; it can be a refinement when you have the basic sketch down.

This sort of thing is applicable to other languages, not just lisp.  And language designers influence each other; take the Java language spec coauthor who explained, "We were not out to win over the Lisp programmers; we were after the C++ programmers.  We managed to drag a lot of them about halfway to Lisp.  Aren't you happy?"
http://www.ai.mit.edu/~gregs/ll1-discuss-archive-html/msg04045.html

Heijlsberg recently added lexical closures and lambda to the next version of C#.
http://www.msdnaa.net/Resources/display.aspx?ResID=2488
http://www.gotdotnet.com/team/csharp/learn/Future/VCS%20Language%20Changes.aspx

So they're slowly shifting the mainstream over to the lisp direction, in absorbable increments.  In some sense, users are subliminally using it more and more.  Of course, lisp will still survive since it allows experimentation with its features... until a language finally reinvents lisp well.

Tayssir John Gabbour
Friday, March 05, 2004

There's far too much here to reply to, so I'll just reply to Joel's
own comments.  I am not entirely up on the current state of available
Lisp implementations, but I do know something about Lisp.  I wrote the
first Lisp-coded Emacs ever, I was one of the "Gang of Five"
coordinators of the design of Common Lisp, and I was one of the
co-founders of Symbolics.

I have built, in Lisp, not only a powerful Emacs-class editor, but
also an object-oriented database system, an implementation of a
predecessor of TCP/IP, an interactive debugger, and an instant message
facility (lots more but it's been so long that I don't remember any
more).  My colleagues built a window system, an online help system, a
distributed naming/directory system, a "make"-type facility, superb
compilers and syntax-directed editors and debuggers for full standard
Fortran, Pascal, and C, an extremely powerful mail client, a mail
transfer agent, a powerful user-interfaced toolkit, a high-performance
file system, etc, etc.

When choosing a language for a real-world project, the inherent
properties of the language (e.g. does it have real closures) usually
are, and should be, secondary to many practical considerations.  Will
we be able to hire programmers who know the language?  And if not,
will programmers be scared off by the language (e.g. by an unfamiliar
syntax), whether their reasons be "good" or "bad"?  Are there
implementations that are very well-debugged, highly optimized,
available on all relevant platforms, and free?

Chris's original point about availability of libraries is particularly
valid and germane.  And not only "are the libraries available" but are
they available on all platforms and will they continue to be upgraded
as the computing ecosystem upgrades, to handle new revisions of
underlying resources such as hardware, network protocols, window
systems, etc?

I take no position on whether these criteria are or are not true for
Lisp right now.  My point is about which criteria are the appropriate
ones to consider.

Programming language acceptance is a positive-feedback phenomenon, a
snowball effect, a bandwagon effect.  There are lots of people out
there writing libraries for, say, Java, and making fast Java
implementations, and they are putting those efforts into Java because
there are lot of Java users, not because of which programming language
features Java has versus which Lisp has.  And the result of their
efforts is that there are more Java users.

Joel, saying that programmers have "voted with their feet" is
unrealistic.  99.9% of them didn't get a vote to begin with.  Yes,
millions have learned Lisp and do not use it, but is it really because
someone gave them the opportunity to choose which language to use?  I
currently work for BEA, and when a new programmer at BEA is hired,
nobody asks him to choose which language to write in.  And the same is
true for someone joining the Linux kernel effort or joining the
OpenOffice development effort.  Your world of people "choosing" not to
use Lisp is some other planet than the one I live on.  By the way,
that includes me: I have not programmed in Lisp since I left Symbolics
in 1988.

Are there productivity gains to be had from Lisp as compared to other
GC'ed languages?  I could argue either side of that one, but the most
important point is that this just doesn't matter very much compared to
the above practical concerns.

Joel's claim that in order to write fast code in Lisp, you must avoid
"cons", is exactly as valid as the claim that in order to write fast
code in Java or C#, you must avoid "new".  The two scenarios are
exactly analogous.

If I started up my very own major software project now, I would make
my language decision based on the criteria above.  Sure, I love Lisp,
CLOS is awesome, real lexical scoping rocks, Lisp macros are way cool
(when used as directed), and personally I really like Lisp syntax.  I
am rooting for ITA Software, and hope for and expect them to continue
to greater and greater success; more power to them, by all means.  But
it would take a lot, or require special circumstances, to persuade me
to choose Lisp for a major software project, if I were in charge of
making the choice.

Dan Weinreb
Saturday, March 06, 2004

I thought about forking this to another thread, but here goes...

Getting back to Paul Graham's article "Beating the Averages," there are a couple of observations that seem to be missed here, as we drift towards (gentlemanly) language wars.

It sounds like Viaweb was generating and executing code on the fly. This is why Lisp drives beginners nuts with its parentheses and prefix operators, and was so big with the AI crowd. A Lisp program IS a Lisp data structure which is interpreted; it is a list of lists of lists of ... (can you say "recursion"?). The process that reads the Lisp source code is simply a parser that converts each parenthesized input list into an in-memory list. So one piece of Lisp code can build another piece of Lisp code and immediately execute it. And rather than interpretting that Lisp code, you can also write Lisp programs that analyze and modify it. In Lisp, functions and classes are elevated to being first-order, manipulable objects. THAT is what made Lisp such a competitive advantage for Viaweb. Lisp fit not only how they worked, but what they were building. This is also why Lisp macros are unlike, and much more powerful than, macros in other languages and tools. You also learn about

Compare this with most other languages. Java's reflection allows you to write a program that can use any other class without knowing about it explicitly. (Another topic [search on "Semantic Web"] is whether it really understands, or has to use predefined identifiers and hope they mean the same thing; Lisp doesn't really deal with this either.) A little better is .NET and the CodeDOM, but you still need to compile code to disk before you can execute it.

Just as important as the technical capabilities is the matter of style, and how you learn to think about problems and their solutions. Lisp has features for conventional imperitive and procedural programming, but the style emphasizes recursive, functional programming. It also promotes the advantages of that style, such as the fact that it is much easier to read, understand and debug code that does not have any side effects, since you don't need to worry about which are the intended side effects and which are unintended.

I had a friend back in college who took the Computer Languages class a couple years before I did. This was back in the 70's.  Final project for the class was to write an Algol 60 interpreter.  Everyone else in the class tried writing their compiler in IBM S/360 Assembler, because that was the way "real" compilers were written back then.  All turned in ugly piles of non-functional code listing after several months of sweating blood (funny, that was the last year they did a project...).  Except my friend. In the class they learned Lisp and Snobol. My friend spent one day and did a fully functional interpreter in Snobol. The recursive string pattern matching, table data types and dynamic storage structures fitted with the project superbly.

Did my friend cheat? No - this was not a production compiler.
1) Efficiency was not an issue. (Good thing - Snobol on IBM mainframe was not cheap.)
2) No need to maintain.
3) No need to have a team of programmers, or worry about staff turnover.
4) No need to worry about profitability and selling upgrades.

More than anything else, the engineering discipline is about studying the situation, weighing the options and analysing the relevant factors to come up with an effective and feasible solution. That's what my friend did. That's what we all need to do. But please come up with a solution to YOUR problem, not someone else's.

The same trade-offs apply to languages. For example, "sparse" languages trade off built-in features for large support libraries; easy core to learn but a long learning curve for the library. Specialized languages trade convenience in limited scope over generality. While Lisp and Snobol have their strengths that make them extremely powerful in certain situations, these same strengths make them weak in other situations. Perhaps Viaweb was bitten by on the back button by using Lisp's features. They certainly seemed to get plenty of other advantages out of using Lisp. No language will be a total fit for your problem; that's why they still need us programmers to write code.

Unfortunately, one weakness is that in many ways these are "advanced languages." Most programmers don't need and have no use for these advanced capabilities. Because of this, mainstream programmers learn mainstream programming with mainstream languages that only support mainstream concepts and practices. I don't think this is a fault of these languages, its more a problem with our society which places value only on "targeted, practical, specialized expertise" and has little regard for theory, advanced knowledge or broad education. For some interesting comments on this, see www.reciprocality.org and "The Programmer's Stone."

This is where Paul cites Eric Raymond's advice on learning Lisp to learn advanced concepts. Lisp is probably the easiest language to learn recursion in--after all, its based on the Theory of Recursive Functions as one model for the Theory of Computability.

Good science fiction makes a point about the real world; it uses the fictional science to set up an artificial world in which a topic can be clearly explained and examined. Without this perspective, one often cannot address certain issues. Sometimes its because they are just too ingrained in our world. The same applies to the issues in programming; some cannot be expressed in mainstream programming contexts, or are a result of those mainstream contexts. VisualBasic is a visual editor coupled with a textual programming language; it is not a true visual programming language like Prograph or Sirius. Try to explain this to a VisualBasic programmer that has never seen these other languages and environments, and they simply  will not "get it," or understand why anyone would care.

Another point in Paul's paper is that he is looking for competitive advantage in the "startup company" mode. This is quite different from the "big, established company" mode where there are concerns about legacy systems, being able to hire large programmer pools or outside developers and deal with staff turnover. These later require a mainstream, commoditized programming model. This is where the VisualBasic, C++ and Java are really big. COBOL and IBM mainframes enjoyed this position in the 60's and 70's, partly because IBM was smart enough to push these into the schools so the younger workforce knew them already and knew they could get jobs; you had a critical mass built up.

However, in a startup position, you have some flexibility that is not available and can escape from the mainstream. There is a lifecycle to industries, organizations and civilizations. What is appropriate, acceptable and necessary for one phase is not reasonable in other phases. This is a combination of changing size, inertia, and many other factors.

Somewhere in my shelves, I still have a book on Simula-67. This is usually attributed as the first o-o language, and inspired the development of SmallTalk at PARC. Simula (as you might guess from the name) was primarily intended for programming simulations, and could deal with complex behavior from a collection of "objects" in an event-driven siguation. Xerox PARC was trying to do GUI programming with the Alto and Star workstations, and realized that this event-driven programming would work nicely there. So they developed SmallTalk because it supported the work they were trying to do. At Bell Labs, C++ was developed to support the development of verrry large software systems; again o-o seemed like a good fit, so they ran with it. However, o-o was still pretty much a laboratory concept and didn't reach the mainstream until Windows programming became mainstream and demanded a better way of developing. (Yes, it was on the Mac first for the same reason, but Mac programming was not mainstream.)

Unfortunately, most mainstream programmers don't really know o-o principles or think in o-o terms. They also don't know Structured techniques or Data Driven techniques. They know what they picked up in a couple of classes on how to use a certain tool to create screens that end-users can use to create/retrieve/update/delete records in a database. Unfortunately, since they never learned about the concept of transactions as a logical unit of work or information integrity as more than referential integrity of foreign keys, they turn out systems that place most of the burden on the user (and then blame any problems as a "training issue"). While I am comfortable working with recursion, dynamic program invocation, and many other advanced techniques, none of my co-workers are. I've seen some systems where developers did something "really cool and productive," only to find it was unmaintainable because there were no tools and documentation to understand what exactly they did so you could correct or enhance it.

You will always have the mainstream people. And I'm glad we have them. I don't want to be the one who maintains all that legacy code out there, or write the next version of the payroll application. Boring! However, there are always pressures to do more and better with fewer resources, handle greater complexity, and take advantage of that new technology that promises so much better. (Forget that PocketPC, there are <$50 calculators and PC video cards that are more powerful than the mainframe I started programming on 30 years ago.) These pressures will always lead to a need to do something new and different in the programming world. We can try to use a brute force approach with older programming technologies, and that works for quite a while; look at all the COBOL that survived even after the GUI craze forced o-o to the mainstream. But there will always be a point where more and more projects fail using older approaches. The people, like the ones who are interested enough to be carryin on this thread on Lisp and other languages, will need to bring in the innovation that will provide some successes that will eventually lead to a new shift in the mainstream.

In his book "The Fifth Discipline," Peter Senge talks about convergent technologies, using commercial aviation as an example. The Wright brothers flew in 1903, proving that heavier than air flight was possible. However, despite several attempts (many called "hair-brained schemes" or worse), it wasn't until Douglas produced the DC-3 in 1935 that commercial aviation became a reality. This airplane had five key enabling technologies that made it technically and economically viable. Six months earlier, Boeing came out with a nearly identical plane that lacked one of these features and was a failure.

It took over 30 years for commercial aviation to become a reality. However, it still wouldn't have happened without all those early attempts during that period which were ahead of the mainstream. These are the people who identified the problems to be solved. Took less than that for o-o to get into the programming mainstream, so maybe we're not so far behind after all.

I'm aware that Snobol begat the more structured Icon language, also by Ralph Griswold. Icon introduced iterators and other features which has largely been absorbed into o-o languages and libraries.  Dylan has learned from these, Lisp, MP and many more languages. True progress is wonderful.

So I'm all in favor of knowing Lisp, Snobol, SmallTalk, Simula, Java, C++, VisualBasic. I don't use them all. But I am a better programmer for it. And its not just about the features of a language; its about the technical, social, and economic aspects. The former may be the pole, but the later let you recognize the brick wall you need to vault. However, as I normally prefer to walk around obstacles when viable, I'll keep coding those mainstream programs that continue to pay the bills.

Great conversation going. We need more of these in our industry.

David Lathrop
Saturday, March 06, 2004

I didn't read the other comments, but I should say that as a CS / EE student at MIT, everyone learns Scheme (a variant of LISP) in the introductory class, 6.001 <http://sicp.ai.mit.edu/Fall-2003/>.

It's pretty good for learning the basics, but not very deployable.  Although Gerry Sussman and Hal Abelson (two of the most famous CS people alive) use it a lot, e.g. for the online grading system.  I'm sure there's something wrapped around it for the online interface.

Adam Smith
Sunday, March 07, 2004

Authored by: Anonymous on Saturday, March 06 2004 @ 11:57 PM MST
Perhaps the reason that Lisp isn't popular in mainstream software development is because there aren't enough Lisp vendors taking Gartner analysts out for expensive dinners and purchasing full page advertisements in magazines.

In most companies, the decision of which tools the IT department will use is made by MBA's who lack introductory calculus. In general, people can only argue about things they know about, and so we probably shouldn't be surprised that they are swayed more by laminate advertisements in PC Magazine than by appeals to concepts in theoretical computer science.

Is it possible that a model decked out in clothing from the Gap touting software from a flashy company on the West Coast is more appealing than a bearded John McCarthy?

Maybe they just need to repackage Lisp.

Matt
Sunday, March 07, 2004

I came late to this discussion, but have read through it all, and here's first a comment to the discussion about file-open earlier in the thread:

>I always find it interesting that with-open-file and the like
>are often pulled out as the canonical examples of macros,
>because their only purpose is to hide Lisp's inexcusably
>verbose syntax for closures.  Ruby and Smalltalk both do
>with-open-file equivalents just fine without macros.
>Here's Ruby:

>File.open("foo"){|file| file.each_line{|line| ...}}

>(Aside) Python:

>for line in file('foo.txt'):

>This is my fault for not explaining what with-open-file
>does.  The idea is that it abstracts away this pattern
>(written in no particular language):

>file = open("foo");
>try {
>  do_something_with(file);
>  do_something_else_with(file);
>} finally {
>close(file);
>}

None so far have mentioned how to do it in C++:

ifstream file("foo");

// Use "file"

No need for explicit close() or try-finally. Welcome to the world of constructors and destructors. :) "file"'s destructor will close the file at the end of the block (you may create a block anywhere in a function), or if an exception occurres.

However, nowadays I'm doing web apps using PHP. Use the right tool for the job. My thoughts on C++ is my way of avoiding brain rot. ;) It's still my favourite language.

PHP has some nice things - such as dynamical function creation and variable variables and functions (so you can use variables and functions by using their name in a string variable), but it's about as deep as a puddle (a borrowed expression from a C++, C# and Java consultant, about Java).

I recently thought I could do some functional style programming a la STL, in PHP, since the PHP standard library includes similar constructs, such as array_walk (iterate over an array, calling a function for each element), array_filter, array_map and array_reduce (these work like their FP-language counterparts).

I wanted to iterate over an array, calling a function expecting _two_ arguments - the array element, and another fixed variable (a reference to another array). No problem, I thought - using create_function (dynamically creating a function) to create a new function which binds one of the parameters, I could pass the new function to array_walk. This is what you can do in FP-languages (called currying or partial application), and you can do it in C++ using objects as a kind of closures, and overloading operator(), so you get a function object, which behaves like a function, but can also hold state.

Alas, in PHP, it wasn't that easy. In PHP, the functions you create can't hold state... I could store a string as a data value there (as part of the function definition), but not something like a reference to an array. Oh, well.

Terje
Monday, March 08, 2004

Here's another posting, which embodies the unfortunately all too common attitute towards abstraction:

>Lost in all this competitive language arguing is the
>requirement for straightforwardness as a basis for
>reliable and for maintainable software.

>For all the goodness that OOP adds to C++, it is still true,
>for example, that in plain C, you always knew what
>"a + b" meant.  In the more abstract languages, like C++,
>you never can be sure unless you start looking things up.

>Similarly with powerful macro languages and mutable
>languages.  The poor code-reader hasn't got much of a
>chance.

>And don't get me started on templates.  Anything that
>complex can hide bugs.  And hiding bugs is not what you
>want.

>The cost of high-leverage features is obscurity. 

By that argument, the most "clear" code should be assembly - or better yet, machine code, with a complete one-to-one mapping to instructions. No need to look up anything or anywhere. Oh, and avoid functions, as well - or you'll have to look up their definition. Better duplicate the code wherever you need it. In fact, avoid any abstractions at all (except the non-neuman one, then, inherent in the instruction set), to make the code absolutely "clear".

Seriously, though, what is more clear to you:

LDR R0, #1234
ADD R1,R2,R3
JNE $1234

or:

do_something();

You do have a point in that with very mallable languages (including C++), you can essentially create domain-specific languages in them. This is also to some extent true of ordinary libraries ("library design is language design", and vice versa, as it has been said). However, I view that as an advantage, rather than a disadvantage: I allows you to model the application domain (also called problem domain) more closely in the solution domain.

With assembly or something like C, you have way less tools available to shape your own language, so to speak, to model abstractions in the code.

You say that "The cost of high-leverage features is obscurity." I see it rather the other way around: Using a too-low level solution for a given task, means that there may be a rather large gap between the solution, and what it represents. You may see how each statement does something, but not what it does (at a higher level) or why.

You mention overloaded operators and templates. Pop quiz: What is more readable to you:

complex<float> a,b,c;

a=b+c;

or:

struct complex { float real, imaginary; };

complex a,b,c;

a.real=b.real+c.real;
a.imaginary=b.imaginary+c.imaginary;

Not to mention that without templates, you'll have to replicate the above code whenever you want a new data type, or use C-macros (pick your poison).

It comes back to my argument about taking your argument to its logical conclusion: To use assembly language. The point with abstractions is to be able to say _what_ is to be done, and to leave the _how_ to the implementation. It allows you to think at a higher level, and to simplify things, as you don't have to be concerned with all the little details of how something works, but can instead focus on the real job to be done.

As for assembly: I've co-authored an assembler, extASM (for the ARM processor) - the source code was about 500 KB, so I have experience that kind of programming. However, it takes way too much time to do things that way, if you are to create anything moderately sized.

If I were to create that assembler again, I'd probably use something like the Spirit parser framework (a C++ library, using very advanced template and overloaded operators programming, to essentially create a DSL with EBNF-grammer, in pure C++) for the parsing part. Spirit is written by another very competent Joel, ;) Joel Guzman.

By the way, excellent articles, Joe! And good discussions, like this, (mostly) with well-informed people and free of flames.

I've bought and read your GUI for programmers book, as well. Very informative and enjoyable (like the other of Joel's articles) and highly recommended.

This strays quite far from the topic, but abstractions in general has also been discussed in this thread.

Terje
Monday, March 08, 2004

I think the comment about the downside of overloaded operators and identifiers is that a bad programmer can decide to make counter-intuitive overloadings.

For example: (Sorry, my C++ is rusty and my example is sloppy, but should get the point across.)

A poor programmer defines a class (MySet) to represent a set of items (class MyItem). This class overrides the "+" and "-" operators:
MySet MySet::operator '+' (MyItem i) {}
MySet MySet::operator '-' (MyItem i) {}

As expected, "+" adds the item "i" to the set ; this is a natural interpretation of "addition of an item to a set," so a good choice for overloading. Actually, here it means "add and replace an existing item."

Unfortunately, "-" means "add the item to the set unless a matching element already exists, in which case leave the original and return the original set."

Some other programmer reading the code will probably see the "-" and expect it to remove the item from the set. (Remember, poor programmers also tend to do poor documentation.) Only after they have found several inconsistencies will they go back and examine the actual code for the operator; since this many not be an in-line function, it won't be with the specification in the header file, so we didn't see it the first time. Worse yet, the inconsistencies arise in code that they are ADDING to the program, in the form of enhancements.

Now, unless the true intent of the original programmer is spelled out (doubtful), the second programmer is liable to think that the function was not implemented properly and "fix it." Then watch the fireworks fly! Several hours of intense debugging later, it will probably be realized what the "-" operator was actually representing here.

Unfortunately, as one character said "along with great power comes great responsibility." These advanced features shouldn't be taken lightly or used just to show how much of the reference manual you've been able to plow through.  People need to seek clarity in their code, realizing that six months later either they will have forgotten or moved on and left it for somebody else to maintain.

Since I've seen fools make ingenious messes with the simplest and most innocent of programming mechanisms, I don't think this is a reason to remove such valuable mechanisms from the toolbox. The example above could have been the result of using a poor choice of identifier instead of operator overloading. It is an argument for proper training and mentoring (I won't go as far as arguing for certification or capital punishment, but there are times...).

David Lathrop
Tuesday, March 09, 2004

I think someone hit the nail on the head as to why lisp has not been widely adopted - parse trees.

In most other languages, you start with some other syntax, and then the compiler does the work of converting the statements into parse trees.

In lisp, parse trees are all there are. Like pointers, you've immediately shut out about 90% of the programming audience, leaving only compiler designers and l33t hackers who might use lisp.

When people have problems with Java (scary, I know), you know they're not going to stand a chance against writing parse trees from scratch.

As an aside, I've often had temptations to write a GUI editor that let's you build parse trees, a la Visio. Drag a statement box here, connect to this statement over there, with a rubber band line, etc. It would sort be like playing with tinker toys.

And then when you're done, output the whole thing to lisp S-expressions.

Programmer
Wednesday, March 10, 2004

Rich topic. Good reading. Way back it was mentioned that a particular environment's language of choice has probably already been made. And a lot of keys pressed on why it wasn't, (or should have been) Lisp. I'm not really qualified to comment but it seems that languages using infix notation have won out over those using prefix (eg Lisp) or postfix (Forth) for no good reason apart from initial legibility, or perhaps because Basic (I won't mention it again) used infix.

Lisp came in just after Fortran (infix again) but before the B-word and C. It ate core, was interpeted and slow - but smouldered on here and there until Autodesk went looking for a macro language for AutoCAD in the late 80's and by adopting a shareware Lisp multiplied the Lisp cognizant by a large integer. It's still there, alongside VBA and a MSC API. (b.t.w. AutoCAD itself was always written in C). Lisp was a good choice as it lends itself well to the loose data structures describing objects and widgets in CAD. (ASSOC ...) gets used a lot - perhaps it was fortunate that most users weren't programmers, had no prior exposure to other languages and seemed to cope equally with (+ 3 5) in Lisp, 3 5 + on their calculator and 3+5 in their head.

I thought it sucked and wrote (and sold) an infix to prefix pre-processor so x=3+5 produced (setq x (+3 5)) etcetc. It saved me (but too few others, alas) from matching up all those parentheses. The AutoLisp editor is now slicked up beyond compare still providing scripting glue for all purposes swamping any attraction my translater ever offered, whereas the C API, and latterly VBA, handle the heavier chores if that sort of coder can be tempted in.

Compiled C runs more quickly and can hide source more effectively than interpreted Lisp and VBA. These crass commercial attributes sometimes overrode the desirable attributes of AutoLisp  (which of course can now itself be compiled) but may lose their relevance once all languages boil down into CLR. 

Lisp may be well placed for the new runtime. Kelvin Throop (Autodesk) supplied a cool encryptor to allow developers to protect their AutoLisp scripts. From a shareware author's point of view the interpreted bytestream environments provided by Java and .NET are similarly ripe for obfuscation and if all languages are going to be driven into the same hole, it might as well be Lisp. (Well ok, or Java)

Keep it going.

Hugh Adamson
Friday, March 12, 2004

I wish I was more erudite in describing why LISP works.  But I'm not.  So I describe it thus:  LISP was my 6th language academically, my 3rd language for pay.  Admittedly, I had a Symbolics LISP Machine to work with, but I was producing after only two weeks on the job even though I'd had zero exposure to it before.

No language before or since has flowed from thoughts to code as effortlessly or completely as LISP (I think my dialect was known as Zeta-Lisp).  Currently I'm liking Ruby.  And I've laughed when coding Perl.  But no gestalt happened as completely as with LISP.  That was 20 years ago.  One of the things that keeps me coding is thinking I might have that same joy again someday.  Ruby is closing in on it.

I wish I could explain it better.  Maybe someone else can.

Drew Mills
Tuesday, March 16, 2004

OK I've never written LISP. I respect those who master it, but allow me to add my 2¢. I support my dependents and pay the mortgage with MS Visual Studio C++. A while back, an Xtreme Programming evangelist suggested I write a new app in SmallTalk. I investigated Dolphin, Squeak, and the IBM product. What struck me was the absence of support for standard GUI functions.

It's all well and good to rave about the OO Purity of SmallTalk, but in order to deliver consumer applications, it is necessary to generate mucho dialogs, with list boxes, combo boxes, radio buttons ad nauseam. You need to generate a dockable tool bar, floating tool tips, with two clicks.

The look and feel of the 'skin' has to be slick, it has to be something the consumer is familiar with. If the feel of the visual interface is an academic curiosity (Java AWT) or some funky home-grown thing (SmallTalk) with limited support for the gadgets the user has come to take for granted, your app will not sell.

Yes, I know SmallTalk has a GUI interface. What about the printer Device Context? How do I generate a TrueType font? How do I launch an Open File dialog? Print Preview? Device Drivers? Ugly stuff, but this is what makes or breaks an app.

That's reality. And if your development environment doesn't address this, it's just a nacrcissitic academic exercise. This is what I would be concerned with if considering LISP for development.

My benchmark for awfulness is RPG on the IBM AS/400. Yet it sells, because it is well suited for the its customers' business environment. It solves the problem.

Hey, what do I know, I've only been writing code since 1965.

Pierre

Pierre Cloutier
Sunday, March 21, 2004

There was some talk earlier about how the CL lambda syntax was too verbose. 

Well, there is a major difference between Lisp and everything else: You can fix it. No need for a 50-page JCP request and waiting a year or three for it to be implemented, you just need FOUR lines of code:

(set-macro-character #\{
(lambda (stream char)
  `(lambda ,(read-delimited-list #\: stream t)
      ,@(read-delimited-list #\} stream t))))

... and *bam*, there's your new syntax:

(mapcar {x : (* x 2)}
    '(1 2 3 4))

In fact, the '{' character is explicitly reserved for 'the user' for this sort of thing.

alf
Tuesday, March 23, 2004

Pierre: I can understand your complaints about the lack of "standard GUI functions" in Squeak, and I don't know much about IBM's Smalltalk, but if you looked at Dolphin it sounds like you didn't look very hard: Dolphin uses native Windows widgets all the way down, has full access to the Windows APIs, COM, etc.  A Dolphin app looks and acts exactly like any other Windows executable, it's just a whole lot easier to write.

As for any development environment that doesn't support native widgets being a "narcissistic academic exercise", allow me to respectfully disagree; there are a whole lot of apps being written right now that only ever communicate to the outside world through a socket, and GUI support is irrelevant because the web browser/email client/etc takes care of that.

Avi Bryant
Wednesday, March 24, 2004

Picking up on a couple of different posts in this thread:

>>I think someone hit the nail on the head as to why lisp
>>has not been widely adopted - parse trees.

>>In most other languages, you start with some other
>>syntax, and then the compiler does the work of
>>converting the statements into parse trees.

There is another aspect of Lisp that allows a vice to become a virtue in some circumstances. The very general syntax that is made possible by this very simple parser. Actually, it hides some very powerful parsing. For example, "(+ a b)" at first looks more complicated than "a + b", but Lisp actually lets you do "(+ a b c d e f)". It also allows you to write a single function, whose header explains how to interpret the argument list as a positional, optional, keyword and "rest of the list" parameters. (And someone complained about overloading?) This is a lot more powerful than traditional languages. One of those things where it's nice when you need it, but hopefully nobody uses it unwisely.

I've always liked the SmallTalk syntax for multiple parameter methods, but even it doesn't compete with Lisp. You have to define a different method for each signiture. C/C++ does provide some of this capability. Of course, Forth also gives you this, but there you have program it procedurally, not declaratively.

As I recall, another thing that Lisp can do is allow you to enter recursive data structures directly in the input. Most other languages may allow you to enter simple trees of data, but require you to execute a program to connect the pieces into a recursive structure. Same goes for directed graph structures.

>> ... it seems that languages using infix notation have
>>won out over those using prefix (eg Lisp) or postfix
>>(Forth) for no good reason apart from initial legibility,
>>or perhaps because Basic (I won't mention it again)
>>used infix.

Exactly so. Two comments. We've probably all seen those examples of text that contain deliberate errors to show how we skip over stuff without really reading it. The same is true of programming languages. A program that cannot be read intuitively cannot be understood intuitively, which makes it a pain to work with and error prone until you become used to it. Second, most of these languages (and this includes C as well as Lisp and Forth) grew up in the old days of incredibly tiny computers--by todays standards many hand calculators have more memory. So they couldn't afford a fancy compiler, and made the human adapt to the machine rather than the other way around.

The Literate Programming people are quick to point out that programs are read at least 10 times more than written. And that typeset text can be read 10% faster than typewritten text (i.e., courier text). So how much time is wasted trying to read backwards syntax like Lisp's?

>>No language before or since has flowed from thoughts
>>to code as effortlessly or completely as LISP ...

The flip side to the above argument. Lisp was always a structured programming language, even before the whole "go to considered harmful" hit the scene. And it fit very well with the data structures it manipulated in a natural way.

1975 book "Structured Programming" by Michael Jackson was a revelation to many. It said that the structure of the program has to match the structure of the data its processing. Suddenly, it made it clear HOW to develop the structure of a structured program. And his explanation of structure clashes, and ways to deal with them, got us through the rapids where everyone struggled and many capsized.

Of course, Jackson Structured Programming (JSP) as it came to be known is still worth understanding, but isn't a big thing anymore. Like Lisp, it provides insights that must be adapted to today's problems. Jackson was talking primarily about batch programming of sequential files. I still do some of that, but's not frequently and its not what I consider an interesting or high-risk part of a project.

>> (several people touched on this)

Lisp, Forth, SmallTalk, C/C++ all have one thing in common. They are all very simple, minimalist languages that are easy to use. This is counterbalanced by very big, difficult to learn language-specific libraries with a very steep learning curve.

Compare this with PL/I, where things like storage management, multi-tasking, two flavors of I/O, a whole lot more were actually part of the language; unless you went to a highly specialized library for Mathematical functions or similar, there was no library to learn, just the language.

C was partially a reaction to PL/I's complexity. It consciously attempted to minimize the language, and push as much as possible into libraries. This gives you the flexibility of choosing alternative libraries, and getting additional features undreamed of by the language designers simply by adding new libraries.

The so-called Fourth-Generation Languages (4GLs) provided a third way of doing things. They traded simplicity for generality. For example, the Natural programming language was great for (3270-type) interactive programming and report generation working with a database. This made many programming tasks simple, as long as you limited yourself to the things it was good at. The language has grown and become flexible; it has a decent event-driven GUI form-based capability in windowing environments now. However, it just doesn't cut it for XML-based programming, and you don't want to build compilers with it (though the Natural compiler was actually written in Natural for some reason).

>>... Pierre Cloutier on Sunday, March 21, 2004

Too true. We don't want primitive tools that build clunky looking, feature poor programs. AWT made a trade-off: portability via lowest-common-denominator GUI. Swing does better. Overall, we need the tools, either in the language, the library or the IDE tools, that do what today's programming requires. I'm not doing much mainframe 3270 and database programming these days.

We are in a difficult time, because there isn't a really good programming language or system for today's needs. They don't match the need for good looking GUI's (especially Web-based GUIs), while keeping the programming tasks and maintenance easy. ASP.NET helps a lot with the "code-behind" separated from the visual design, but adding a couple new fields to a form is still a lot of work. Maybe the Widbey markup language will get us closer. (No its not innovative; see XUL that was developed as part of Mozzilla.)

And its not just GUI programming. XML processing is hard to do. It reminds me of the way compilers were programmed before people developed good parser generators. Also, distributed compenent based systems are not well supported except for the most basic levels.

David Lathrop
Saturday, March 27, 2004

>>As an aside, I've often had temptations to write a GUI
>>editor that let's you build parse trees, a la Visio. Drag a
>>statement box here, connect to this statement over
>>there, with a rubber band line, etc. It would sort be like
>>playing with tinker toys.

What you're talking about is true Visual Programming, not the Visual Basic "Visual form design with lots of textul programming." Check some old languages: ProGraph and Sirrius(?) that originated on the Macintosh and moved to the Windows environment. I think I found ProGraph with a freeware version some time back.

Like Lisp, these are worth looking at for the lessons. These are descibed as "functional data flow" languages. Like Lisp, they are very far from mainstream.

One aspect of these languages, as well as Visual Basic type programming and especially CASE/UML/Data Modeling tools, is that drag-and-drop programming involves a lot of switching between the mouse filling in dialog boxes. This is not the most efficient to enter informaiton. Some of this is intrinsic to this style of program development, but some is lack of knowledge of good UI development. (I'm often tempted to send a truckload of books on GUI design to Rational. Would Joel volunteer his book?) Also, its hard to get a good linear presentation (i.e., printed form) for learning about a new program or conducting program reviews.

David Lathrop
Saturday, March 27, 2004

The topic is titled, "what do you think of Lisp?"

Well, its a personal opinion but it is my favourite programming language with Scheme and Objective Caml competing for a close second. Smalltalk and Python are also on that list and for good reason. The most important being these languages make writing easy programs very easy and difficult programs possible. When you start thinking about complex algorithms, like those used in AI derived search techniques or character reactions in games etc, you need a language to be able to describe these properties. That is where these dynamic languages excel. Cryptography is another field where I would use Lisp because of the ease of implementing complex algorithms as well as pre-built support for bignums.

For standard desktop based apps, these languages provide no advantage, but have a disadvantage because of lack of standardized GUI support which is tolerable in the commercial variants. Also the commercial Lisp vendors like Franz, Xanalys and Digitool are still very much in business despite having a very expensive product. That is because there is a segment where nothing else will do.

I have worked in an environment where we used Smalltalk as our language of choice despite the "Java hype" and were very happy with it. Now you have some dumb manager who reads Gartner decide that we need to move to Java because its more cost effective. This is obviously not taking into consideration that how difficult it would be to migrate the pre-built frameworks onto Java with its mediocre technology.

I have seen so many conversion projects and Smalltalk to Java failures are probably on top of the list followed closely by Lisp to C++. This is because those languages are still playing catch up. Every year we hear about Java having this new feature in it and the standard comment is - "finally" at least they included that. All a Lisp developer does is, "Hmmph, I'll just write a macro". The whole object system, including CLOS with all its faults is better than many others. There are also many other object systems on Lisp and its easy to write them in. The concept of an image both in Smalltalk and Lisp is probably the most powerful thing going for them. It is no surprise that even after 50 years Lisp is still being used for some very critical projects today.

Functional and dynamic languages are also used in the labs of IBM, Motorola and Intel for chip design validation as well as design. This is especially after Intel's FDIV bug. Lisp is also used in places where natural language processing, cryptography, AI, games etc are being developed.

Lisp was one of the later languages I was exposed to and do not have any intention of looking back. Yes, if I want to develop desktop based GUI apps, there are many choices, but for complex algorithms we need decent languages like Lisp, Scheme, Smalltalk, Ocaml, Erlang etc.

Sanjay Pande
Saturday, April 03, 2004

A great many things have been said already. I may add my voice here: I have probably programmed in between 12 and 20 different programming languages (including C++ and Java), and LISP (my 3rd programming language way back when on my Apple II with LeLisp running with a Z80 card) has alway been my programming language of choice. No other language has ever brought me the same feeling of *enlightment* (possibly Prolog to some extent). When I say Lisp, this would include also other Lisp-family languages such as Scheme, Logo, or Dylan.

The reasons for that are a mix of (in no specific order):

1- Programs are data are programs
2- Macros (whose power is multiplied by point 1)
3- Easy closures and continuations (related to point 1 too)
4- Completely interpreter-like, interactive environments and yet incrementally compiled code (this is not unique to Lisp I know)
5- You can instrument, debug and modify your running system, without restarting it
6- Very simple and consistent syntax (for those who dislike the parenthesized prefix notation, Logo and Dylan propose other ways)
7- With image files, the environment is the program. There is no concept of source files or anything (Smalltalk does it too).

For GUI programming, I am already delighted with the combination of Cocoa, Interface Builder and Objective-C. But if I could replace Objective-C with Dylan or Lisp, I would in an instant.

J-D

Jean-Denis Muys
Monday, April 05, 2004

J.D. says: "For GUI programming, I am already delighted with the combination of Cocoa, Interface Builder and Objective-C. But if I could replace Objective-C with Dylan or Lisp, I would in an instant."

J.D., you might want to stay tuned to developments on OpenMCL. It's an open source Common Lisp implementation for OS X that provides Cocoa bindings. It's probably not as polished an environment as the Apple tools are, just yet, but it's under active development and the Cocoa bindings are getting more mature and robust all the time. (The language part is quite mature as it was a fork off the commercial Macintosh Common Lisp which has been around forever.)

You might also want to check out Clotho, a project to build an IDE for Lisp on OS X using OpenMCL. It's a fairly new project but is also under active development:

  http://www.common-lisp.net/project/clotho/

-Peter

Peter Seibel
Monday, April 05, 2004

Okay, regarding performance -- good Lisp code is fast.

Existence proof: The PS2 games "Jak and Daxter", and "Jak and Daxter 2" were written in a variant of Lisp, called Game Object Assembly Lisp (GOAL). These games are widely cited for having enormous environments, and great graphics.

Using a Lisp-like language allowed them to move seamlessly from assembly-level code to higher-language constructs. All this on  the PS2, which is a multi-processor beast. The ability to change code without restarting the game was also an advantage.

More info here (free registration required):  http://www.gamasutra.com/features/20020710/white_01.htm

The games' developer, Naughty Dog, developed GOAL, based on Scheme. The compiler was written in Allegro Common Lisp.

If you want a cheap Lisp that works well on Win32, (FFI to communicate with C++ code, COM, DLLs, inline assembly), check out Corman Lisp at http://www.cormanlisp.com

Jaba Adams
Wednesday, April 07, 2004

Hello,

I saw a lot of bullshit here, so let me explain WHY there are so many developers writting in C-like languages. Currently we have an IT industry, which requires a lot (A LOT) of developers, which have to produce a large number of lines of code. DO YOU THINK THAT IT'S POSSIBLE TO LEARN lisp (whatever the version is) WITHOUT SERIOUS MATH-KNOWLEDGE?
The C-like languages are simple to UNDERSTAND, because they are imperative, statically-typed languages. They do NOT require deep understanding of what the hell is happening there!
And before you try to write another :LISP is great:, let me tell you something - "In this game is winning not the best, but the most profitable!"
If you like LISP so much, make it profitable. And stop complaining that someone is not smart enough to see the greatness of IT.
Period.

DJ
Thursday, June 17, 2004

i'd like to suggest reading this book for anyone intrested in Scheme; it kicks *** and it's free !

http://mitpress.mit.edu/sicp

(actually this book talks about a lot more then merely Scheme)

Olmo Ziarko
Thursday, June 17, 2004

I am a perl/C programmer who has dabbled with scheme and ocaml. As a newcomer to some of the advanced concepts there is a lot to like.

For me, the nicest thing about ocaml is the type system and the biggest shortcoming is the pre-processor. I find it incredibly hard to change the syntax of the language (and even have to learn a modified version of ocaml to write the pre-processing instructions). However, I still find it an amazing language and dream of it replacing C and C++ for systems type programming (its fast).

Strangely my cognitive problems with the preprocessor  leave me hankering to try lisp again, if only to explore metalinguistic abstraction and changing the language to suit the problem. The more I learn about programming, the better lisp seems to me.

Saul Hazledine
Thursday, June 17, 2004

Funny, I've always thought of the Yahoo stores being some of the worst stores I've ever shopped on. They are very ugly, and not well layed out for a shopper compared to some stores out there.  Most stores are slow and fancy..yahoo stores are slow and not fancy.
Rarely I see a nice fast store that looks good, and has a good search feature, but yahoo stores lack any of this.

Bob
Wednesday, July 07, 2004

All this "theory"  and "talk" about Lisp is great, but what about some real world examples? Does it matter if we can over-analyze and yap about Lisp and how wonderful it is or isn't? Let's see some real world applications that Lisp excels in (obviously it isn't anything special with regard to yahoo).  Let's see some examples of a coder taking only 6 hours to complete the same or a similar project versus 8 hours in c++ or pascal. Where's some real examples?

My guess is that the real world examples are already out there: statistics show that no one is using lisp.. take a look at sourceforge or any website and compare the language statistics.

Jorry T
Wednesday, July 07, 2004

*  Recent Topics

*  Fog Creek Home