Fog Creek Software
Discussion Board




Extensible Programming for the 21st Century

I stumbled onto this fun article this morning (the writer even references a Joel article):

http://www.third-bit.com/~gvwilson/xmlprog.html

I like the idea of a simple common semantic model for all programs with a variety of layers of syntactic abstraction on top of it (at least I've used this technique for several programs in the past).  What we've got now is a common semantic model (at the machine-code level) but 'orthogonal' levels of syntax abstraction.  Will the next big programming language be easier to extend with small sub-languages?

I liked the comments on custom data structure visualization in the debugger too.  It'd be nice to be able to easily decide how you'd like to see an instance of a particular data structure (he uses the example of cyclic graphs), and to easily switch between different views of the structure.  I know I'd like something like that for seeing the states of objects that I usually think of geometrically (like blobs of triangles and so on).

Kalani
Thursday, May 13, 2004

"This leaves the third, and thorniest, problem: the cognitive gap between what programmers write, and what they have to debug. This gap is why so many programmers dislike wizards and other code generators..."

Now recall from another trhead "minimum competency":
[] Does not get hung-up on using "Frameworks"

One of my favorite statements.

Anyhoo.  Its cool that someone out there is thinking way outside the box.  Sometimes really nice things come out of it.  But I'll not hold my breath, I'll probably be dead by then.  The author admits the "big bang" scenario is unlikely.

"We believe that next-generation programming systems will store source code as XML, "

After you've disposed of your inflight barf-bag, you realize by the example that maybe its not such a bad idea. 

"instead of integrated development environments. Instead, the change will probably be generational [Kuhn]: as old programmers retire,.."  Umm, that's me clentching my copy of emacs in my casket.

So what I can distill this down to is an XML representation - kind of like XML as assembler or byte code - of higher level languages.  The author was smart to use C-like syntax to win over converts.  If he had used Pascal like syntax - he would have been ignored.  But you could have this view of the code as Pascal or as C or as BASIC.  The user's language syntax choice would be like the programmer's editor of choice today.

It is certainly interesting.  However, I'm a firm believer that necessity mothers invention - not vice versa.  I don't honestly see the need.  Cool tools are not a need.  Hell as the author mentions, for most programmers, they aren't even a want.

hoser
Thursday, May 13, 2004

Very thought provoking and it's something we should definitely be exploring in the future.

XML? Aaaargh! But I kind of see their point. Guess that's what it will be down at the bottom, just as long as I don't have to look at it!

But I notice they don't attempt to answer the *real*, interesting, criticism :

"what if the underlying representation is too constraining? "

For example, does the underlying structure allow for asynchronous events? Parallel threads? Declarative assertions? Aspects and cross-cutting? Generators? If not, we won't be able to build languages with these features on top of the substrate. It may lock us in to today's thinking.

phil jones
Thursday, May 13, 2004

You don't need XML to do this; as they said:
"And yes, there are better (i.e. more succinct, and hence easier to process) ways to represent the semantics of programs than XML, but we believe that will turn out in practice to be irrelevant. XML can do the job, and is becoming universal; it is therefore difficult to imagine that anything else will be so compelling as to displace it."

So, because everyone is jumping off of a bridge we should do it to? If you're going to suggest, and implement, a new paradigm then you should make the *right* choices, not the *easiest*.

MR
Thursday, May 13, 2004

By "Easiest" I meant *most convenient*.

MR
Thursday, May 13, 2004

MR, he'd probably rather use S-expressions to represent the code (that's essentially what he's saying when he says that XML is overly verbose), but he dismisses the issue that you brought up because that particular syntax isn't important and if the majority of people want another syntax there (no matter how ugly you think it is), that's better than not having a common representation at all.


Thursday, May 13, 2004

You could easily store representations in a relational database, and then use any method you'd like to turn it back into code. That avoids all of the XML-improprieties involved.

MR
Thursday, May 13, 2004

>> If you're going to suggest, and implement, a new paradigm then you should make the *right* choices, not the *easiest*.

Or maybe you should make the choices that allow your paradigm to be realised and also widely adopted.

Matt
Thursday, May 13, 2004

Right, it's easy to look back on languages 50 years ago and say, "oh this is crap, they didn't have continuations, native transaction support, first-class tables, jimmy-on-a-stick" etc, but many of those things only became evident with the concepts introduced by earlier languages.

So rather than doing "the right thing" (c.f. worse is better), it's important to do "the changeable thing".


Thursday, May 13, 2004

Seems to be suggesting storing programs as a sort of syntax-neutral parse tree in an XML format, which can then be edited in a special editor using any number of user-specified syntax options (C-style, Python-style, etc etc)

Very cool, but why not just use Scheme (or Common Lisp) as the underlying language? It's pretty much the same thing just using s-expressions rather than XML. Which happen to be a lot more elegant. All we'd need would be a standard for specifying alternative syntax representations for the Scheme parse tree, and an editor writing that could use these. Sounds like the kind of thing that's probably already been done as part of a 1970s computer science thesis or something.

Matt
Thursday, May 13, 2004

For the next 20 years, dumb shits will keep trying to invent stuff that's already in Common Lisp. This happens because dumb shits like to think they're smart.

U.S. Government
Friday, May 14, 2004

Those that ignore the past are doomed to re-implement it. Poorly. :)

MR
Friday, May 14, 2004

But if you read the article, the guy's not ignoring the past.  He's not ignoring the flexibility that he got out of Scheme either.  The only reason he brought up XML *at all* was that it's an alternative syntax for an old idea, and that insisting on the old syntax is wasted pedantry.

Yes you can make "little languages" in Scheme and everybody knows that's a great idea, as long as your little language can be represented in S-expressions.  That's a fine sub-structure, but it's overly verbose in many many areas (that's the point of "complex syntax" anyway).  Make languages with native support for syntactic extension (and make it easier to muck with the semantics), and the language will adapt better to change.


Friday, May 14, 2004

Two words: prefix operators. Has any Lisp implementation yet managed to provide infix operators for mathematical expressions?

Chris Nahr
Friday, May 14, 2004

Yes. In Common Lisp YOU have control over the system. Syntax is just one facet of that control.
http://www.cliki.net/Infix
(I have not needed that personally, so I haven't tested it yet.)

Now, I ask if you can make your own foreach() loop in your language of choice, without waiting for your language designer. Or write a simple command which automatically turns a function of yours into a caching version of itself. Or if your compiler asks your advice on how to optimize special cases of your functions.

Common Lisp is about being a programmer and not a user.

Tayssir John Gabbour
Friday, May 14, 2004

Years ago Richard Gabriel made a pretty convincing argument for why Common Lisp (and several other technologies) have failed in the marketplace.

I like Scheme, and I think it's a pretty neat language.  Unfortunately, "pretty neat" isn't what drives the market.  Simply saying "uh, Scheme, plus XML!" just isn't going to cut it.

Until you understand and acknowledge why Lisp failed last time, you're wasting your time trying to push it again.

(Go read Gabriel's old papers.  Hint: it didn't fail because it wasn't XML.  Lots of languages which aren't XML have done just fine.)

WIB
Saturday, May 15, 2004

"Yes. In Common Lisp YOU have control over the system."

Or not. This file is a macro that requires the programmer to precede every expression that contains infix operators with a #I prefix so that the expression is correctly "decoded", as in #I(2+3).

Begging your pardon, but I have that much "control" in just about every other programming language... I'd just have to add quotes around the argument!

The macro author recommends getting a Lisp implementation that supports infix notation natively, by the way. Looks like he isn't that convinced of Lisp's alleged flexibility either.

Chris Nahr
Saturday, May 15, 2004

Ah, you're a language warrior, eh Chris? ;) No matter how well I answer your question, you'll twist and turn.

You don't need to precede anything with #I. That was just what I linked to. You can also just cover things in [] brackets. You aren't at the mercy of some library author, or the implementation.

Now answer my questions please. (I took the time to answer your attack, finding a link.) Can you either:
- Build your own foreach() loop.
- Autocache functions with one simple command.
- Compiler asks advice how to optimize special cases of functions.

Tayssir John Gabbour
Saturday, May 15, 2004

WIB, yes indeed, lisp did once fail in the marketplace. But now C is failing in the marketplace. I'm told people used to flame lisp for garbage collection, lexical scoping, etc.

One of the co-creators of Java said, "We were not out to win over the Lisp programmers; we were after the C++ programmers.  We managed to drag a lot of them about halfway to Lisp.  Aren't you happy?"

C# is a further evolution in that direction, as Heilsberg added closures and lambda to C#.

Lisp is back. With a vengeance. And while I understand Chris was annoyed at someone calling lisp-haters "dumb shits," I was annoyed that he attacked lisp rather than the poster.

Tayssir John Gabbour
Saturday, May 15, 2004

In fact Chris, since I'm tired of all the haters & flamers that caused Joel to shut down his forum:
- Can you save your programming environment to a disk image and come back to it like a video game?
- Does YOUR language support infix syntax for functions?
- Can you operate on code with the same powerful muscles you developed for analyzing data?

Tayssir John Gabbour
Saturday, May 15, 2004

Um, Tayssir, I'm not a "language warrior"... but you are, apparently. I'm not the one making grandiose statements about empowering the programmer.

I'm simply asking if I can use standard mathematical infix notation in Lisp, as in any other language (except Forth and Assembler). Is that such an outrageous demand?

Now that this infix macro wasn't so hot you talk about square brackets. I'm sorry, I have no idea what you mean by that -- care to give a brief example?

As to your questions on what "my" language can do, I don't know the answers but I suspect they're all negative. That doesn't bother me much, however, since I never missed any of those features you list. But I would certainly miss infix notation for mathematical operators, which is why I asked about it...

Chris Nahr
Saturday, May 15, 2004

(There are many tools which empower the programmer. Lisp is one such tool, but there are others. I sometimes get passionate about them and write that way.)

- The infix macro issue.
The way to understand this issue is to realize that lisp is about opening up the innards of the lexer/compiler/etc to the programmer. A lexer for any language just tokenizes the sourcecode en route to the compiler. In lisp, you may customize this lexer through "read macros."

When the lexer sees the (, it realizes it's a macro character which says, "Stuff the next tokens into a handy list." The ; is a character which says, "Ignore the rest of the stuff on this line, I'm a comment."

When you read that infix macro, he used the perhaps ungainly #I(3+2) infix syntax. Common Lisp reserves [] for you, so you can make it [3+2] as you wish. Just define [ to whatever #I( is.


- Missing features.
I didn't miss them either until I used them. But I am aware of the strengths and power of other languages, and study them to be a better programmer. Lisp did not support OOP until someone wrote some macros. Now it is one of the most powerful OOP systems. Most lisps do not include a codewalker for static typing, but a couple weeks ago in the Amsterdam lecture, the lecturer wrote a quick typechecking system.
http://alu.cliki.net/lisp-user-meeting-amsterdam-april-2004


- Reading Gabriel's works.
I might be seeing him next month. ;)

Tayssir John Gabbour
Saturday, May 15, 2004

You jerks are still arguing about this?  If you both went and read up about worse-is-better you could probably skip about a week's worth of this low-level bickering and get to something that's actually new and/or useful.

This crap is straight out of Gabriel's "requirements for language acceptance" -- "must not require users to have 'mathematical sophistication'", "must be similar to existing popular languages", "must be a minimally acceptable language", ...

Yes, C/C++/Java is technically worse than Lisp.  No, that isn't what drives the market and isn't worth a bucket of warm spit.  Are you trying to make a point here that wasn't made 10 years ago?

/me presses fast-forward on this thread

WIB
Saturday, May 15, 2004

Why don't you prove that you're not the total moron that you sound like by providing a link to that paper? The documents that I found for "Richard Gabriel" and "Worse is Better" don't contain those quotes.

Chris Nahr
Saturday, May 15, 2004

Perhaps us jerks are. ;) Sorry for hijacking the thread.

I'd add Kernighan & Mashey's "The Unix Programming Environment" paper. Unix spread like Sun's Java virus, because of weak legal hassles and not trying to cash in hard. Plus an underground snapshot of Unix sourcecode spread with the Lions book.

The lisps that survived were also viruses. One didn't require proprietary hardware and survives happily now, charging thousands per seat even amid competition.

Gabriel's paper is old, he's taken pseudonyms to argue for/against it, trying to answer his internal doubts. Programmers are users too, and we often need a gradual slope to things like garbage collection. Anyway, that is what I learned in the last months.

http://www.jwz.org/doc/worse-is-better.html
http://www.dreamsongs.com/WorseIsBetter.html

Tayssir John Gabbour
Saturday, May 15, 2004

Chris: See the second URL Tayssir listed.  (He probably googled for "richard gabriel worse is better"; the first result is Gabriel's web page about worse-is-better, containing several of his papers on worse-is-better, including the one I quoted from.)

If you need to be spoon-fed the URL of the paper that contains the quotes, it's at http://www.dreamsongs.com/NewFiles/AcceptanceModels.pdf

Tayssir: Yes, he's debated both sides of this issue.  That's one thing that makes be believe it even more.  :-)  Even if it's not strictly true (I'm a bit undecided myself sometimes), it's an interesting model that usually seems plausible and accurately models my observations.  That's good enough for me (as my friend C.M. is fond of saying).

WIB
Saturday, May 15, 2004

*  Recent Topics

*  Fog Creek Home