Fog Creek Software
Discussion Board

Developing a full blown compiler ?

Is it possible nowadays for a single person to write a full blown compiler in a year or two ?

Will it be stupid to not use the available tools
like Lexx & Yaxx to decrease the dev process of a compiler

Saturday, May 10, 2003

It depends. There are still people who build their own cars. Of course re-inventing the wheel will take more time but if you are not in a hurry and you enjoy the process it worths it.

Saturday, May 10, 2003

"Is it possible nowadays for a single person to write a full blown compiler in a year or two ?"

Depends on the language you want to compile. You should be able to write a subset Pascal compiler in a few weeks. Writing a full C++ compiler on the other hand would take many man-years.

"Will it be stupid to not use the available tools
like Lexx & Yaxx to decrease the dev process of a compiler"

Once again, it depends on the language. Some are better suited to application of techniques such as recursve descent. Most commercial compilers use a combination of techniques.

Neil Butterworth
Saturday, May 10, 2003

Having made a C compiler at university using flex and bison I found it very useful knowledge. You understand the difficulties involved in compilation, and you are not so fast blaming the compiler for everything. Complaining that the compilerdevelopers should do a better job.

Anyway I stumbled onto this article at ACM Crossroads

I will not build a compiler, it is way to complex and requires many man years, but occationally I encounter the need to build some simple parsing tools.

I can use flex/bison without too much problems but do anyone have any experience of the newer tools discussed in the article  ?

I have still an intrest in compilers and read papers when I find them. The compiler course had the Dragon Book, but are there any other book recomendations ?

I work sometimes closely to our companys compiler group, but they have no recomendations outside Japaneese books and papers.

I have also read some of the Intel whitepapers about their compiler.

At the end it boils down to that knowing the internals of a compiler is not my core task and time does not always allow to studdy odd subjects like this. Also there are so many more subject to studdy.

Anyone know about some kind of online reading group where say a paper a month is released and discussed. Theoretical Computer Science. Just to keep your mind on its toes. I try to read DEK when I find time. But it is also hard, since you have no guidance or discussions like at University.

What it boils down to is that I miss some theory from Uni but I will not trade my job for a PhD program just yet. I still have theoretical challenges with engineering codes etc. more so than normal software development. Sorry for the rant, the compiler started it..

Fredrik Svensson
Saturday, May 10, 2003

I've never found lex and yacc to be worth the trouble. Hand coding lexers and parsers just ain't that hard.

Joel Spolsky
Saturday, May 10, 2003

Several years ago I wrote a C compiler in C.  The target architecture was a 68000 and I got it to the point where it could compile itself.  Fun project, and I learned a lot about coding.

I hand-coded the parser (recursive descent).  Lex and yacc didn't have much to motivate me.

Eric W. Sink
Saturday, May 10, 2003

Regarding books on the subject, I would recommend the book "Modern Compiler Implementation in Java" (ISBN 0-521-58388-8) by Andrew W. Appel, if the dragon book seems to rigid or old-fashioned.

When it comes to tools, the only thing I have to say is to try to stay clear of lex && yacc and anything that is related to these, if possible, as the code they generate really cannot be said to be designed as much as "thrown together", IMHO.

Unfortunately, they seem to have assimilated the field. For instance, the book above uses JLex and CUP.

Roland Kaufmann
Sunday, May 11, 2003

I built several compilers using JLex and CUP after reading the Appleman book.  The book was a basic introduction to compilers and had very practical basic compiler coding tips.  It was a nice contrast to the Dragon book which covers important topics, but is more theoretical and leaves much of the coding "as an exercise for the reader".

JLex and CUP worked okay with two exceptions (1) the error handling capabilities were poor, and (2) I had trouble getting JLex to handle unstructured text (e.g. documentation of equations.) 

I'm considering switching to JavaCC, which seems to be a standard in the Java world and is possibly more flexible. Any comments from the field?

By the way-- in response to Joel-- the reason I like using a tool instead of direct coding (and he's right, it's not hard) is maintenance effort.  I've frequently made small changes to the language syntax targeted by the compiler.  To implement the change I make a small edit to the language descriptor file and run an ant script which calls JLex, CUP, and javac to generate and compile the java code.  Time elapsed -- very little.

Will Glass-Husain
Sunday, May 11, 2003

You ask, "1-2 years?"

I would say define "full-blown".

Are you inventing a new language?

Doesn't awk, perl, python, ruby, haskell, ocaml, asm, effiel, fortran, lisp, XSL, snobol, C# (tm), java, PHP, COBOL or SmallTalk work well enough? :)


p.s. If you are going open source, please let me know, I am interested in helping.

Monday, May 12, 2003

Developing a full blown compiler was a semester course at my college for undergrad students.

Yes, this also involved implementing our own versions of lex and yacc, however were allowed to use these tools for the "final" version of the compiler.

It's said that if an undergrad lives through that course, they can handle anything the world can throw at them. :p

S. Gwizdak
Monday, May 12, 2003

I highly recomment antlr ( for generating parsers and lexers over Lexx and Yacc (bison).

It generates recursive decent parsers for Java, C++ and now C#.

I developed a small VisualBasic-like compiler and Java-like virtual machine for my Compiler Development course.  One of my never-off-the-ground pet projects was to write a programming language and compiler.

Wayne Venables
Monday, May 12, 2003

It is possible to design a compiler in a year or two. From experience, writing the compiler is the easier part. The harder parts are designing a useful language that is not just a syntactic variation on something that exists. The most tedious part is designing the runtime library. In addition to coding the runtime, there are legal issues such as the redistributability of the code. This is important, if you want the compiler to be commercially useful.

I did all the above and it took a long time. If you're interested in the result look at which is a compilable VHLL (very high level language). Designing the language was definitely the hardest part.

Tuesday, May 13, 2003


Lingo looks interesting.

Are you aware that there already is a lingo language?

I'm guessing you do, since you look like you've done your homework.

Oh, and the postal address is missing in the payment details.

Ged Byrne
Wednesday, May 14, 2003

I've found about 3 Lingos so far. There's the Recursiv one  you found, Macromedia (far larger) and another one I forget now. And my one! Also Lingo is trademarked in various places, but not by Macromedia.

Anyway I've 'parameterized' the name so it should be possible to change it. Maybe it will become 'Visual Fred'.

"Oh, and the postal address is missing in the payment details. " You're right - I felt uneasy about putting my address on the internet. I don't know if that's paranoid. There should be a phone number though.

Bill Rayer
Wednesday, May 14, 2003

I'm downloading as I type.  I think I've been looking for something like this for some time. 

I see the fax number.  Have you considered excepting payments by PayPal?

Ged Byrne
Wednesday, May 14, 2003

The fax comes through to the phone line at the moment. The phone is a UK number where I live.

I've used Paypal about once to buy something. I recall it was useful after I set it up, but the initial setup was a pain since I had to wait for a credit card statement. This was about 2 years ago and I did this from the UK. It may have got better since.

I'm still trying to get the hang of this on-line thing as I'm sure you can see :). It's been enough of an effort getting the software to work. The d/load is a late beta and any feedback is welcome.

Thursday, May 15, 2003


Tuesday, January 27, 2004

*  Recent Topics

*  Fog Creek Home