I spent most of the last couple of weeks embroiled in a project for work that provided me with a magnificent (or maybe terrible) distraction: building a toy programming language. One I'm deluded enough to think might be more than a toy someday.
Here's how this went down. Several of the major programming languages I cover for my day job — Rust, Swift, and a few others — are built atop a framework called LLVM. LLVM can be commanded to generate code programmatically. Pick an existing language that has an interface to the LLVM library, and you can use that language to build a compiler for a new language with LLVM doing the heavy lifting. Go far enough, and eventually you can build the compiler in the new language. (This is roughly how most new language development takes place.)
To get my hands as dirty as possible, I checked out a tutorial the LLVM folks wrote that demonstrates how to implement a simple language called Kaleidoscope. Turns out someone else had implemented the Kaleidoscope demo in Python, a language I know intimately well, so I took that demo, disassembled it, studied it, and used it as my way into LLVM's innards.
And then I did something that was, again, either magnificent or terrible: I continued working on it. As in, I started following some of the what-to-do-next suggestions in the tutorial. Adding type definitions, allowing the user to build standalone binaries ...
One week later, I had enough of a language that I was able to build Conway's Game Of Life:
Two weeks later, I was writing documentation for the language. That's when I knew I was in trouble.
I have this bad tendency — or maybe a good one? — to get involved in projects that force me to play over my own head, but I don't always follow through as consistently as I ought. When I created the software that runs this blog, for instance, I did so because I had an immense distaste for WordPress, and also because I didn't want to continue running Movable Type. I did manage to create enough of a blog application to suit my needs, but it's still too hard to deploy well. Some of that I blame on the awful state of web hosting generally, but most of it is my own fault, I admit. Building an installer for a web application just isn't something you can get terribly excited about, especially when the majority of the itch you wanted to scratch with the project in question has not only already been scratched but had cortisone applied to it.
But I also have this other tendency to seek out challenges just to see what comes of my meeting with them, and not because I want to achieve some specific end result. One of my watchphrases for life has long been "You only get better by playing over your own head." Developing a programming language, even a tiny one, forced me to play over my own head in several way:
Again, my purpose here wasn't to create something that would replace any of the existing languages, because such things are virtually impossible to fulfill by design. People use some things instead of others not because they are technically superior, but because they are omnipresent, because they have a low barrier to entry, because they have a whole slew of social factors around them that are difficult if not outright impossible to engineer. If you build something that has an appeal to someone, somewhere, an audience will form around it, and you can then figure out the best way to form a relationship with that audience.
Where's this going to go? No idea. But if my skills ramp up enough with it that I produce something capable of production work, watch out.
New York City
Other Lives Of The Mind