Searching for Signal

the n01se blog

My path to Clojure

Someone asked me about the path that lead me to Clojure. On the off chance that he's not the only one curious, I thought I'd post the answer here.

Sometime after college I started hunting for the "best language", not with any formal definition but as a personal quest for what I liked the best. A primary criteria was something I called "expressiveness". I don't know if I can define that accurately, but it has something to do with being able to provide APIs that concisely model whatever specific domain you're dealing with. A common example is how much easier it is to work with text in perl (because of strongly integrated regex support) than it is in C.

My "best language" before Clojure was Ruby, because of its clean object system and iterators, but I hadn't experienced any functional or immutable languages yet.

I had heard a lot about how amazing LISP was, but I didn't get it. I couldn't figure out why anyone liked LISP's annoying syntax, or why anyone (who wasn't deeply into Artificial Intellegence) would want to write programs that write programs. So one big turning point was reading "On Lisp" by Paul Graham (free download). By the time I was done with that book, I was convinced that any "best language" had to support macros.

So I tried to use Common Lisp, but just couldn't make it "stick". I had learned plenty of other languages, and usually enjoyed the process, but Common Lisp's lack of libraries really drove me nuts. I could do network, file or text stuff in 4 or 5 lines of ruby that took a page of code in Common Lisp. So I shelved Common Lisp for a while.

I do a lot of C++ and JavaScript at work, and was learning to really like JavaScript -- clean syntax, closures, direct support for hash-maps and regex, a builtin "gui" system. So it was easy for Yegge to lure me into trying JavaScript on the JVM, especially with the promise of optional static typing, which I was pretty sure I wanted.

Around the same time I learned about Project Euler, and thought it would be a great way to get used to using JavaScript for general purpose programming rather than just browser stuff. I did the first five PE problems in JavaScript before giving up. It was terrible. Not only was there no static typing yet, but things that had been minor annoyances in the browser now seemed to be giant obstacles to solving the problem at hand. Re-reading some of Yegge's cheerleading brought me to the conclusion that he's actually endorsing some private language based roughly on JavaScript. None of the other languages that Google lets him use are as malleable, and Rhino lets him code up whatever new features he wants in Java and morph the language to his will. This is great for him, I'm sure, but was lousy for me.

I hadn't used Java in any capacity for years, but now thanks to Yegge I had it installed and ready to go. This made me more willing to try Scala. I don't remember how I heard about it, but I picked it up to work on the rest of the PE problems, and this is when I started to learn about immutability. Scala encourages the use of immutable locals, and has a few immutable collections. I expended a lot of energy bending my problem-solving thought processes to this new context. Scala also has lazy streams, which was another big hurdle. Both of these were challenging to learn, but once I got it, they were very rewarding tools to have at my fingertips. Project Euler was my primary teaching tool, forcing me to learn new concepts a little at a time. The folks on the IRC channel helped me when I got stuck, though in exchange, I frequently had to endure condescension from some of them.

So this was going along okay, until I thought I had enough Scala under my belt to take a crack at a tricky little API I'd had on my mind. I'd been wanting a way of picking data out of HTML documents, something with the expressive power of XPath, but that embedded cleanly in the host language. I wanted to be able to provide new functions, filters, etc. (have you tried to work with dates in XPath and XSLT??), and for the return values to be host language data structures, not more XML documents. I'd actually built a beastly little project using xlstproc and ruby, but that's another story altogether.

Anyway, I began sculpting this API in Scala. I spent a day or two experimenting with syntax, the right combinations of traits, implicits, etc. to create the kind of XPath-like expressions I wanted. I finally got the beginnings of something that I thought would work, and compiled a minimal test. Well, tried to compile it, but the types weren't quite right. I then spent the another day or two trying to get the type declarations correct enough to even compile, and finally gave up in frustration. I'm sure it would have been possible to make it work, but I wanted to spend my time solving actual problems, not solving problems the compiler was making up for me. Maybe I didn't want static typing after all.

This is when I started learning Clojure. Paul Graham had convinced me I wanted a LISP. Yegge's hype about JavaScript had prepared me for the JVM. Scala had introduced me to immutability and lazy streams. Clojure let me use the best properties of each of these, so it was a very natural fit right from the beginning. I've since learned more -- how to use more immutable collections, software transaction memory, etc., and the discussion group and IRC channel have always been friendly and encouraging.

That's how I got to where I am. I don't think I'd recommend the JavaScript or Scala tangents to anyone else, but "On Lisp" may still be worth reading, at least until Programming Clojure comes out. Since bailing on Scala, I've done another 25 or so Project Euler problems in Clojure, and had so much fun I even went back and did the first 14 all over again. I also took another stab at XPath-inspired document querying, resulting in a zip-filter lib that I'll write about some other time.

Clojure's definitely closer to the "best language" than any other I've tried. If you haven't tried it yet, you should!