Jeffrey I. Schiller
Networking, Security and Cavies

Learn a new Language

This is a post about programming languages... really. You just may have to read a bit to get there...

So one of the things that I do is write data driven websites. They range from the simple content management system I wrote for the Essex County Rabbit and Cavy Breeders to on-line surveys for MIT's Web survey service. At their heart all websites have two basic components. A “static” component that serves up static content and a dynamic component that serves up content that varies by context. Generally dynamic content is derived from a database query.

None of the sites I write have to scale to the size of something like Twitter or Facebook. However they do have to handle some amount of load. I am also a cheapskate, so I would like to throw a minimal amount of hardware at these solutions (particularly for the Rabbit Club which doesn't really have a budget for any IT stuff at all!).

Some of these sites, particularly surveys, have to handle load spikes when the survey is released. This is particularly true when we offer a time based incentive. Something like “Everyone who fills out the survey gets a $5 gift certificate for ice cream, the first X people get two!” As an aside, it is amazing how bribe-able college students are in this fashion.

So... I have been looking at how to make sites scale. Cloud services such as Amazon's EC2 are really cool for something like this. You can add web application servers to your heart's content (or wallet's capacity!). It is easy to scale web applications. But the database backend... now that is trickier.

Enter CouchDB. CouchDB is one of the new generation of “NoSQL” databases. Although I had been using SQL (via Django most recently) I had no inherent desire to continue to use it. Its a tool, there are lots of tools... What I found useful about CouchDB is its replication strategy which is very flexible. In particular I can have multiple writable copies of my database. One on each server I deploy if I so choose.

Now CouchDB is open source and free. This also means that I need to be able to support it myself. OK, I am the kind of person who likes to understand how the tools I use actually work, be they hardware or software. So I downloaded the source code for CouchDB and looked at it.... and gagged!

I was staring at the “Blub Paradox” – right in its face. Now there is this really good paper by Paul Graham called “Beating the Averages.” Go read it.... In it he asserts that programming languages have different amounts of expressive power (he also asserts that Lisp is the most expressive, something I am not sure I completely agree with... and yes, I have programmed in Lisp). He also postulates that most programmers settle on a favorite programming language before they turn 25 and often do not want to change after that. I guess I am the exception to the rule. But then again, I studied and taught “Structure and Interpretation of Computer Languages,” MIT course 6.031, in the late 1970's and early 1980's. The lessons learned stick with me to this day.

So back to the “Blub Paradox.” (go read the paper for details). Basically when a programmer familiar with some language looks at languages that are more expressive then their chosen language, see languages that are weird! Well, I was looking at Erlang … and it was weird. I can usually figure out what a program is doing, even if I am not 100% familiar with the language it is written in. After all, a loop is a loop and most programming constructs are pretty recognizable. But with Erlang I was seeing things I had never seen before. So I decided I needed to learn it. This desire was increased when I decided to use a program called “Membase” for a distributed memory cache and discovered that it too was written in Erlang (honest, I didn't know that when I was looking for solutions).

So what is so special about Erlang. Well it is a functional programming language, which makes it very different. Variables can only be assigned once in a function. There isn't really iteration. Recursion (and tail recursion) are one of its fundamental paradigms. Back in my 6.031 days we thought recursion was pretty cool, so this appeals to me. So what and why bother you ask? Well the goal of Erlang is to facilitate the writing of robust, concurrent, distributed programs. And with systems being run on multiple instances of an OS (virtual or real) and with individual systems running on multi-core CPU's. A language the can facilitate concurrency is definitely interesting!

So what's the point of this post... Well, I could say it is “learn Erlang” which isn't a bad investment of your time if you ask me. But the underlying point is that keep on eye on new and interesting programming languages (p.s. Erlang isn't new, it is just becoming more popular as the problem it was designed for is becoming more common!). You will be glad you did.

Oh.. and go read “Beating the Averages” now, you will be glad you did, really!