Lessons Learned while Introducing a New Programming Language

I've used a lot of languages (professionally) over the years: (off the top of my head) Cold Fusion, HTML, Javascript, php, SQL, CSS, ASP(classic & .net), C#, Ruby, Flex, Java, & Clojure. Each language has pros and cons. Being a programmer, it's easiest to discuss the cons - and in general I believe it was best said:

I hate all programming languages - Matt Foemmel

I think it's important to start with this in mind. At some point you're going to hate what you're advocating, so imagine how other people feel about it.

In 2008 I introduced Clojure into a DRW codebase. This blog entry focuses on the adoption lessons I've learned in the past few years.

Language Selection
Introducing a new language to an organization isn't an easy task; if you're going to succeed you're probably going to need to pick a language that satisfies a large technical need and is also considered socially acceptable. I was writing Java 100% of the time when I joined DRW, despite the fact that a large portion of the code I was writing only needed to run in eye-ball time (250ms). Java was absolutely the right choice for the Faster Than Eye-Ball Time code we were writing, but using Java for the other code always made me feel like I was unnecessarily paying the Java verbosity tax.

On occasion I would mention the Java verbosity tax, and I found that my boss was actually interested in looking more closely at JRuby. I think JRuby would have been a win for us, but for me it was more interesting to hear that I had an ally if I wanted to explore non-Java options. If JRuby was on the table, then I knew that any high-level, dynamically typed language would also likely be on the table.

However, before I'd even discovered any JRuby curiosity, I'd already begun looking into Haskell. Generally, software in trading firms needs to run "fast." If I was going to successfully introduce a new language it was going to need to run "almost as fast as Java." I'd heard good things about Haskell's speed, and it also fulfilled another adoption desire of mine:

A language that doesn't affect the way you think about programming, is not worth knowing - Alan Perlis

I believed that if I found a language that was "performant enough", allowed us to deliver at a faster pace, and improved our programming skills then the level of effort required for adoption would be justified.

I toyed with Haskell for a bit, but the adoption path seemed a bit too steep. There's the effort it takes to learn Haskell, but more importantly: we were already on the JVM. If I was going to get any support, I needed something that was easy to sneak into our existing infrastructure. Enter Clojure. Clojure is performant enough, more succinct than Java, and sufficiently different to anything else I'd previously worked with. Clojure is also dynamically typed and high level (like Ruby), so I was hoping to get some support from my boss.

Causing my teammates as little pain as possible was a very large requirement - I believed this was key to gaining adoption. Clojure appeared to be the best choice due to the fact that we were already using:

• IntelliJ all day
• JUnit to run all of our tests
• TeamCity for CI & artifact creation
• The JVM on our servers
• Yourkit for profiling

Clojure was the language that gave me everything I was looking for and had the easiest adoption path for the rest of the team.

I would have preferred Haskell or OCaml from a learning perspective, but they didn't seem like practical choices - and I doubt I would have had as much success getting them in production. While I need to be an expert in many things Clojure related, I relied on the JVM server settings that someone else has determined to be "the best". If I had chosen Haskell or OCaml I'd have had to become an expert on a much larger scope of topics (e.g. deployment, memory model, libraries, new tools, etc).

I believed then, and I still believe today that Clojure was the best choice given our technical needs and the social context.

Hello World
Introducing a language is a delicate act. There's a bunch of valid concerns that you're going to need to address. I wasn't exactly sure how my teammates were going to react to the introduction of Clojure, so I wrote the original code in my own time at home. I knew that we needed some integration tests for our application; however, no one was actively working on implementing anything. I began by writing them in Java and then wrote Clojure versions as well. I knew enough Clojure that I was able to showcase it's succinctness - something I knew the team would value in integration tests. Additionally, since the tests aren't part of the production running code there were no real speed concerns to consider.

Integration tests are a good place to introduce a new language, but any non-production code will probably be an equally good choice. For example, you could also choose database migration scripts, log file parsers, 3rd party software simulators, or deployment software. As long as you pick something that can fail without too much immediate pain you should be able to easily recover from any adoption issues.

After I had both sets of tests complete I showed both versions to the other developers on the team. I pointed out why I preferred the Clojure versions and asked if they would be willing to give Clojure a chance. I also made commitments that made it hard for them to say no to the experiment.

Your Commitment
I eased my teammates adoption fears by making the following commitments.

• If you want to work on the code I'll work with you (if you want me to work with you).
• If you don't want to work on the code I'll fix anything that's broken.
• If the initial pain of working with a new language becomes unbearable to you, I'll rewrite everything in Java on my own time.

Obviously you'll need to get team buy-in before writing too much in your new language - otherwise you could be setting yourself up for working a lot of nights and weekends.

Tool Support
Chances are your team already has a tool-chain that they are happy with. Whatever that tool-chain is, your new language is going to need to play well within it. For me this meant being able to execute Clojure code within IntelliJ as easily as Java. For the most part the La Clojure plugin does the heavy lifting; however, I did need to write a testing framework that allowed me to run focused tests and seamlessly integrated with our existing JUnit test suite. The main point here is to remove any adoption friction that your team may be feeling. Learning a new language is a reasonable request, but changing the way a team works simply to accommodate an unproven (on that team) language choice is probably too much to ask.

You may also need to make some sacrifices as well. I prefer to write Clojure in emacs; however, I'd rather be writing Clojure(where appropriate) in IntelliJ than writing Java in IntelliJ. During the early/fragile adoption time, you're the one who's going to need to do the majority of the compromising.

Find allies
Chances are you'll have varying levels of interest in your new language. During the early days you should do anything you can to encourage others when they're interested; however, you don't want to push anything on anyone - that's the easiest way to find enemies. Hopefully you'll have a few teammates who are also as excited about a new language as you are - work closely with them on improving both of your skills. You'll want to get as many advocates as you can, otherwise you'll end up looking like the lone team member forcing the team to do something they aren't comfortable with.

It's also inevitable that you'll end up needing more research time, needing tool support, and dealing with more production issues than you originally anticipated. You're going to need a few other people to pick up some of the slack when you find yourself overextended. Even when things are going well you'll find yourself in need of allies to help you support the growing code written in a new language.

Lastly, the worst case scenario is you leaving a team and no one is left on the team that wants to support that code. It's pretty easy to see how a staffing situation can be portrayed as an adoption problem.

Know Everything
Obviously you can't actually know everything, but you're going to need to have an answer or be able to quickly get an answer to anything that comes up. You'll definitely want to read a few books on your new language before putting it in any codebase that your team members also work with or rely on. That's likely not enough though, you'll also need to know where to go if you run into issues. For Clojure this was the IRC channel for immediate answers and the mailing list for less time sensitive issues or issues that required a bit more explanation. If you're really looking to cover your bases you'll want to have some type of relationship with the creator or one of the community leaders.

Once people start adopting the language you introduced they're going to start doing things you didn't anticipate. You're going to need to know the dark corners of the language and the associated corner cases that exist. You'll also need to be an expert on issues such as memory allocation, performance, deployment, tool integration, library support, upgrade schedules, and everything else that is outside of the language's syntax.

The more allies you have, the less you'll need to 'know everything'; however, at the end of the day you're going to need to know as much as possible. If things ever go wrong, all eyes are going to be on you. That's the level of commitment you're making by introducing a language, so you better know what you're getting into.

Get Help
If your company is willing to let you introduce languages then they are probably a fairly supportive organization. Hopefully you'll have a bit of a training budget. See what opportunities you have for bringing in the language creator or the community leaders to work with you or provide training. If you're having issues with anything, having the creator of a language work with you is obviously a huge advantage. However, if things are going well it will probably benefit you to give up your training budget to contribute to a pool that could cover some training for other team mates who are interested in learning from the language's creator (or a community leader). Whether it's for you or interested allies, utilize your companies training budget to encourage adoption.

Be an Advocate, not a Zealot
At the end of the day it's not likely that everyone is going to have a compatible opinion. That's fine. Don't push your opinions on people who aren't interested. On most occasions the 'right' choice is the one that someone is passionate about. You might be passionate about your language, but a teammate of yours might be passionate about the old language. Neither of you needs to be right or wrong. People should work with what they are passionate about, and any attempt to make them work in another way is likely to do more harm than good. People who want to work with the new language will find a way to organize themselves together and people who don't will do the same thing. There's no reason to force adoption.

Originally, I approached the problem as "If I create a clearly superior solution then everyone will want to come along." This definitely turned out not to be the case, thus the blog entry on compatible opinions on software. I learned that one person's "obviously better" is another person's "obviously worse." In the end the team ended up organically dividing into people who worked on the Clojure code and those that didn't. This worked out well for both parties, as the people who wanted to work with Clojure had enough available to them, and the people who didn't weren't forced to.

The divide was more of a practice than an official split. We were all still part of "one team"; however, we tended to work on separate applications that communicated through messaging or not at all. I advocated strongly for a team split based on incompatible opinions and size (we were at 7, and I thought 4 and 3 would be fine), but we never ended up making anything official. Eventually other factors changed and the team shrunk to a size where a split was no longer necessary. I still believe splitting would have been the best solution if the team hadn't reorganized for other reasons.

The End
Introducing a new language is likely a multi-year affair for any moderately sized organization. There's not likely an "end" where your responsibilities go back to what they were before introducing a new language. On the flip-side, you get to use what you consider to be the best tool for the job. Hopefully it's worth it when it's all said and done. Personally, I'm happy with my choice, but I expect to be learning new lessons on this topic for at least the next few years as well.

High Level Testing with a High Level Language

In the early days of my project we made the decision to high-level test our Java application with Clojure*. One and a half years later, we're still following that path. It seemed worthwhile to document the progress so far.

My current preferred style of testing is rigorous unit testing and less than a dozen high level tests.

This style of testing doesn't catch everything; however, context is king. In my context, we constantly have to balance the number of bugs against the speed at which we deliver. We could test more, but it would slow down delivery. Since we don't want to drastically impact delivery, we try to get the most we can out of the tests that we do write.

A few more notes on context. Our high level tests are written by programmers for programmers. The application is fairly large and complex. We use Java, Clojure, Ruby, C#, & others. We take advantage of open-source frameworks as well as vendor and in-house frameworks. The result is a user-facing application used exclusively in-house. It's not a service or a 'for-sale' product.

The Bad
Context: The team knows Java fairly well and writes the majority of the domain code in Java. The team didn't have any previous experience with Clojure.
Result: Happiness with using a high level language was often impacted by no knowledge of that high level language.

Context: The vast majority of the code is written in Java and is able to be manipulated with IntelliJ.
Result: Some members of the team felt that using an additional language hampered their ability to use automated refactoring tools, and thus the rewards were not worth the cost. Other members believe the powerful language features provide benefits that out-weigh the costs.

Context: The tests need to run on developer boxes and build machines. One and a half years ago, there was no easy way to run Clojure tests in JUnit.
Result: The team hacked together something custom. It didn't take long to write, but the integration with IntelliJ and JUnit is not nearly as nice as using pure Java.

The Interesting
Context: The team has plenty of experience with Object Oriented (OO) based, C-style languages. The team didn't have any previous experience with Functional Programming (FP). Tests are procedural. However, the team was much more experienced writing procedural code in OO than FP.
Result: The paradigm shift impacted the speed at which the team was able to learn Clojure, but the team was able to peek into the world of FP without writing an entire application using an FP language.

The Good
Context: The team needed to build several utilities that allowed them to high level test the application.
Result: It was trivial to create a REPL that allowed us to communicate at a high level with our application. We practically got a command line interface to our application for free.

Context: The team was curious if Clojure would be the right language for solving other problems on the project. The team knew that with less than a dozen high level tests, rewriting the tests in another language would not be a large time investment.
Result: The team was able to determine that Clojure is a viable language choice for several tasks without having to take the leap on a mission critical component whose size may be variable.

Context: High level tests often require a significant amount of setup code. Clojure provides language features (duck-typing, macros, etc) that allow you to reduce the noise often generated by setup code.
Result: The tests are easier to read and maintain (assuming you understand what and how things are being hidden) and the amount of utility code required to run high level tests was significantly reduced.

*We still use Java to unit test our Java, for now.

Sacrificing some IDE capabilities

When discussing adopting Clojure or JRuby there are often heated discussions concerning the pros and cons of working with a language that has less IDE support. This blog entry will focus on the common concerns I hear.

I've never met anyone who has mastered Ruby or Clojure and had the opinion that they are more productive in Java. I think that's the best "proof" that the pros outweigh the cons.*

Other than that, things get complicated. The problem is, I understand where reluctant adopters are coming from. But, I just don't see any way to convince them to make the leap.

Many Java programmers do a bit of Ruby or Clojure and make a determination. I think their concerns are inflated by 3 factors: api/library fear, shallow understanding of powerful language features, and inability to optimize problem resolution.

A common concern is that learning a new language requires learning new libraries. Even though Java's libraries are available in Clojure or JRuby, both languages still have their own apis/libs to learn. In Java, you often use auto-completion. In Ruby and Clojure some auto-completion is available; however, it's often suspect. Few people use a Java REPL, so they don't understand how helpful/powerful a REPL is. In Clojure and Ruby, the REPL is often the most effective way to learn new apis/tools. Since most Java developers haven't experienced the value of the REPL, they exaggerate the overhead of learning apis/libs.

Writing idiomatic Java using Ruby or Clojure will produce painful and likely unmaintainable code. Writing idiomatic Ruby and Clojure often require understanding the powerful aspects of a language (metaprogramming, method_missing, macros, duck-typing). Understanding those aspects of a language can greatly change the design of a system. The result of mastering language features is often concise and significantly more maintainable code. However, if you aren't able to see the concise/maintainable version, you will picture a design that may likely be hard to manipulate without IDE support. Enter unnecessary concern and doubt.

Lastly, when things go wrong you need to know how to fix them. Fixing issues is often a mix of tweaking the language and mixing in a few tools. In Java it's often Ignore the noise in the stacktrace, Click a few links and open a few files, Drop in some break points or print lines or change a few tests. Getting to the source of the problem efficiently requires an IDE. In Clojure and Ruby I have those tools (since I do my Clojure and Ruby in IntelliJ), but I also have the REPLs. I know what noise to ignore. Even without those tools (when I used TextMate for Ruby), I know what patterns generally represent what issues. I can see the signal in the noise, and I have an additional tool (the REPL) to help me determine what is signal and what is noise.

However, if you don't know what is noise and what is signal, and (once again) you don't understand the value of a REPL then you exaggerate the cost of problem resolution.

Cosmin Stejerean recently mentioned "To me good IDE support for Clojure is very much like using training wheels on a bicycle."

That's almost true. I'll freely admit that I'd love some free refactoring. It would make me more productive. However, I'm already more productive so it's icing on the cake, not a reason to stop me from using the language.

Unfortunately, those training wheels are very much a deal-breaker to conservative adopters.

* I have recently heard of a few projects starting as Python and switching to Java, but I haven't ever done any Python, so I wont speculate as to why that happened.

Polyglot World

At QCon 2008 Steve Vinoski told me he uses at least 4 languages pretty much every day. At the time I thought 4 seemed like a lot of languages to use. Are we ready for a world where programmers need to know so many languages?

If you think about building a web application, you probably need to know a server-side language, HTML, Javascript, CSS, SQL, etc. Of course, there's no easy way to draw a line and say that those are, or are not all languages. I'm not sure the distinction matters, what does matter is having effective tools to get your job done. Maybe 4 languages isn't surprising.

I've worked on projects where C# and SQL were all we used; however, I think those days are coming to an end.

As an example, let's start with books. I'm currently reviewing Martin Fowler's upcoming DSL book. For his examples, Martin chose Java and C# where possible, and Ruby where a dynamic language was necessary. I think his language choices are pragmatic and wise. It's easy to agree and not give the idea another thought. However, his choice is a departure from his traditional usage of Java and C# exclusively.

Martin didn't add a new language because he changed his mind about dynamic languages, he added a new language because our industry changed it's mind about dynamic languages. Dynamic languages are now an acceptable choice for many projects, and an appropriate language addition for authors looking to appeal to the widest audience.

If Martin ever publishes another version of Refactoring, I expect it will have examples in several different languages. Refactoring: Ruby Edition contains several refactorings that are impossible to do in Java and C#. I have no idea what languages will be popular at the time of another Refactoring release; however, I think we will see a diverse set of languages and refactorings that target specific languages.

However, the polyglot movement can be found in the industry even more so than in books. Twitter is known to use both Ruby and Scala. Google uses C++, Java, Python, and Javascript. At DRW Trading we use any language that will help us get to market faster. Any given day, working solely on my project, I could be working with Java, C#, Clojure, Ruby or other even more interesting options. Steve's "4 language" statement doesn't surprise me at all these days.

You can also look at the multiple languages on the CLR and the JVM as further proof that as an industry we are willing to look to multiple languages to solve individual problems. The interoperability allows us to use the languages and paradigms we prefer for specific tasks, while working toward a greater solution. From personal experience, using Java and Clojure together has been a big productivity boost on several occasions.

As I already mentioned, in the web world you often need to master several languages to be effective. As a young developer, learning each language is probably a daunting task. However, I think having several targeted languages is part of the reason that web application development has been so successful. Targeted languages often give you the power and simplicity that allow you to get your job done without thinking about overly complex abstractions.

People often forget that "one language to rule them all" also implies that the language will attempt normalize everything. However, there are obviously efficiency gains to be had if you choose languages that play to the strengths of a paradigm or platform. The web proves this by having languages that target and solve specific problems. I wouldn't want to write CSS or Javascript in another language, even if it allowed me to solve all my problems with only 1 language. I prefer languages that increase my effectiveness to languages that reduce my learning requirements.

Ruby took a bold and productive step by introducing features that worked exclusively on Linux. As a result, several tasks are very simple to do on Linux and impossible on Windows.I applaud the choice. It makes my life easier when I'm on Linux, and I know I have to find another solution when I'm working on Windows. I'll take those options over some poor abstraction that feels painful on both platforms.

And, that's where I see our industry moving. Languages targeted at specific problems and run on specific platforms. The languages will be easier to create and use. There's been a movement in our industry towards simplicity for some time. This is just another logical step in that direction.

The days of one language for all problems and platforms are dwindling, which is nice. I prefer the polygot world.

Kill Your Darlings

When I worked with Zak Tamsen, one of his favorite (software development) sayings was: Kill Your Darlings. The idea was simple, don't get too attached to code.

I'm not sure why so many developers become so attached to code they've written. Perhaps developers have the same attachment to code that artists have to their paintings or music. Or, perhaps they view their legacy code as job security. Whatever the reason, developers seem to have some illogical desire to extend the life of code they've produced.

Of course, the desire to make legacy code live-on isn't universal. I tend to live in the opposite world, where I can't delete code fast enough. As much as I'd like to believe that people who share my views are evolved in some way, I know it isn't true. At least in my case, I like to delete code because I have ADD. Plain and simple.

Someone recently asked me how I deal with developers who are over-attached to their code. I've only found one successful way: create a finished solution that is simpler, and then share it with the original developer. Unfortunately, this path doesn't exactly promote collaboration, so it can definitely lead to other problems.

George Malamidis taught me something about code attachment a few years ago: You always gain by allowing someone to show you an alternative solution. If someone wants to solve a problem in a different way, there are several gains to be had. If their way is inferior, you have an opportunity to mentor a team-mate. If their way is equally elegant, you've gained another solution, or point of view that may be superior in the future. If their way is superior you learn something new and the codebase improves. In exchange for these gains you only need to give up time. Time is valuable, but it's also well spent on improving the ability of a team-mate or your personal ability.

Michael Feathers has also written on this topic, specifically focusing on frameworks. In Stunting a Framework, Michael discusses creating small focused frameworks and then letting them go. I think Michael really nailed it with that entry, it's definitely worth a quick read.

I think killing your darlings extends beyond codebases and frameworks to languages themselves. At SpeakerConf 2009, I floated the idea that we should more actively seek to kill languages. Perhaps, after 3 versions of a language, it's time for that language to be retired. Imagine what we could create if the resources dedicated to Java were instead focused on creating a successor to Java. Think of all the time that would be saved if backwards compatibility became a non-issue.

I can imagine the horror of developers who have written and used frameworks for Java. What would we do without all those frameworks? The reality is, we'd port those frameworks and improve them. There's thousands of developers who are dying to port an existing framework to a new language. And, frameworks should be stunted anyway, if you agree with Michael (which I do).

We are already evolving. New languages are appearing all around us. Frameworks are born and killed at a rapid pace. However, the attachment to code, frameworks, and languages only slows our maturing process. Be aware of, and support progress. Kill your darlings.

Targeted Languages

The vast majority of actively evolving business software is written in Java these days. Java has long enjoyed the title of One Language to Rule Them All. However, in a previous post, The Next Big Language, I mention that I'm skeptical that there will be one language that is perfect for solving all possible problems in the future.

I might be overestimating the speed at which our profession is maturing. One of the reasons Java became the enterprise standard was because the wrong people were making decisions based on inadequate information and swarms of terrible programmers. I like to believe that we're moving away from those days, but then again, we definitely still have far too many NNPPs that need to be encouraged to find other employment. However, I do think it's inevitable that we move to languages targeted at specific domains and problems eventually.

We've already taken the first step. Games are a form of business software. Games are consumer products. However, game development has never been dominated by Java. I won't pretend to know about the game industry, but from what I hear it's been largely dominated by C++. However, these days Lua has enjoyed great popularity in the videogames industry.

The game industry has been using the right tool for the job for some time; however, other business are also starting to catch on. Many companies are using Ruby and Rails to build websites. There's no question that having a good group of programmers building your website using Ruby and Rails can provide a significant competitive advantage. Erlang is another language that targets a specific class of applications, and it provides huge advantages over the alternatives for those applications.

I think targeted languages raise some new interesting questions.

For the professional software developer, targeted languages may make it harder to switch domains. These days it's easy to get a job doing Java development in insurance, banking, advertising, and many other domains. If in the future all the banks are using a functional language focused on low-latency, you may find it harder to make the switch to an advertising agency with a shop that's using an object oriented dynamic language. It may not be long before selecting a language implies selecting a domain as well.

There will be implications for firms also. The pools from which companies hire from will get much smaller. It's very hard to hire a good Java developer these days, imagine what the picture is going to look like when there are 5-7 targeted languages that are widely accepted and the developer pool is split 5-7 ways.

There will also be questions about how many languages your organization will support. If you're a bank and you write your trading applications using the best language for the job, would you allow the intranet dev team to use a completely different language?

Companies are already facing these questions. At DRW Trading we use C#, Java, C++, Python, Ruby, Perl, etc. I hear that Google only allows C++, Java, Python, and Javascript. There's definitely a balance between leveraging existing knowledge and using the right tool for the job. However, finding that balance isn't an easy task.

Like it or not, we're headed towards more targeted languages. It's probably worth considering the implications sooner rather than later.

The Language Question

My first job was working with Cold Fusion for a year. For the next two years I worked primarily with Javascript and a proprietary language. Next was a brief stretch with PHP and ASP, followed by about 3 years of .net. Finally, for the past 2 years I've been doing Ruby. I do not consider myself a Ruby developer. I prefer to just be a developer.

I don't expect that Ruby will be the last language I ever work with. In fact, I expect I'll probably be doing something new in the near future. The question is what language should I work with next? Should I go for a classic like Lisp or Smalltalk? Should I give Scala or Erlang a shot and ride the concurrency wave?

The problem is, I'm asking the wrong question. Rarely when discussing languages do the languages themselves dominate the conversation. I can't remember ever being in a conversation where someone won a language debate with things like closures, metaprogramming or static typing. Instead, the primary focus of language debates are usually around frameworks, performance, IDE support, developer availability, what OS does it run on, and any number of other factors that are only related to the language.

The question extends even beyond factors related to the language.

At Google I’ll work with C++ rather than (for example) Ruby but I do get to be part of changing the world. -- Jon Tirsen

Jon gave up a job using his language of choice to work with another language he likes far less, but for a cause he's very interested in. I have another friend who recently starting working with an investment company because he was interested in the domain.

Two years ago I started looking at Ruby because several of my colleagues were giving it a look. I preferred Rake to NAnt, and starting using it on my .net projects. Before long, someone asked me to be part of a Ruby project, because of my limited exposure to Rake.

I got introduced to Ruby by coworkers who were interested. I got involved with Ruby because I prefer build files that do not require XML programming. I stuck with Ruby because we had a steady stream of Ruby projects that needed experienced developers, I got plenty of blog content, I liked the composition of the Ruby teams I got to work with, and I liked working with clients who are comfortable with early adoption.

Notice, none of the reasons I use Ruby have anything to do with the Ruby language itself.

I'm interested in doing something new because I feel like I've been doing the same thing for about a year now. I'm also interested in traveling to other ThoughtWorks offices and getting some fresh ideas for innovation.

Again, none of my desires have anything to do with language.

I'm not giving you the tired "no silver bullet" or the hand waving "the right tool for the right job". I'm asserting that people use those phrases to justify their language choice, but you'd be better off asking what the real motivations for choosing a language are. What other factors does the language introduce that make it their choice.

It's also helpful to have this understanding when considering criticizing someone's language choice. My friends aren't using Java because they like Java, they are using it because they like IntelliJ, high performance on a few boxes, simple deployment, Hibernate, String Template, Spring, and a hundred other factors. Therefore, criticizing Java as a language doesn't really do anyone any good. Even if I convinced them that Lisp is a better language than Java, I still wouldn't have convinced them to use Lisp on their next project.

Erlang Interest

Why do the Alpha Geeks care about Erlang? The obvious conclusion points to processor speeds remaining static, and multi-core computers becoming mainstream. As of October 31st 2007 it is quite complex to create a well designed desktop application that takes advantage of multi-core processors using Java, C#, Ruby, etc. Alpha Geeks like to have several tools at their disposal. Having several tools available can ease solving a complex problem if the tool is designed with that problem in mind. Therefore, looking to see if Erlang provides a better solution for concurrent programming is an easy choice.

Why do CIOs care about Erlang? CIOs know that computing power is going in the direction of more cores, not faster processors. Knowing that Erlang is a language designed to take advantage of multi-cores is enough to drive interest. Other questions that I would expect from CIOs are: Reliability? Nine nines uptime[1]. Maturity? It's been around for 20 years. Who else is using it? Ericsson, Nortel, T-Mobile. The argument sounds compelling.

Why do I care about Erlang? I quite like the idea of running an entire test suite in parallel. Additionally, Procedural languages were mainstream years ago and several best practices emerged. These days, OO languages are mainstream and more lessons have been leared; however, the masses have never exploited the power of functional languages. I believe that if programmers start writing applications in Erlang to address the concurrency issues the best practices from the functional community will be given more attention. I hope that everyone can benefit from the diversified experience.

[1] The AXD301 has achieved a NINE nines reliability (yes, you read that right, 99.9999999%). Let’s put this in context: 5 nines is reckoned to be good (5.2 minutes of downtime/year). 7 nines almost unachievable ... but we did 9. (http://www.pragmaticprogrammer.com/articles/erlang.html)

The next big language

I believe the large success that Ruby has enjoyed has made programmers eager to find the next big language (NBL). The NBL holds a lot of promise for the aspiring software developer. Become an expert in the NBL and you'll be able to command a premium while riding the next wave. If your lucky you may also design the testing or build framework that will ensure stardom in the NBL community.

So, how can you be ahead of the curve? How can you be one of the first to master the NBL? Tim O'Reilly already told us how to start: watch the Alpha Geeks.

The NBL might be a language that already exists. Dave Thomas recently blogged a bit about Erlang. Avi Bryant continues to invest his time in Smalltalk. And, it would be foolish to ignore Paul Graham and the Lisp community.

Then again, the next big language might be Ruby with YARV, Rubinius, JRuby, XRuby, or IronRuby. It could also be a language that exists but has a small community such as IO or Boo. Or, it could be a new language with more or less rope.

I think part of the problem with guessing what the next big language will be is that the question assumes there will be one language that can satisfy all your needs. The truth is, differing systems have very different needs. How many web applications have you worked on that actually needed an object oriented language? Because of the request/response nature of the Internet, most web applications could easily be built using a procedural language. There's tons of PHP sites that work just fine without objects. Sure, some sites are complicated and need a bit more than PHP4. For example, building Yahoo Store must require an Object Oriented language, right? While Yahoo Store was probably a bit complicated for PHP4, Paul Graham lists Lisp as their secret weapon of success. I'm not implying that OO languages are better than Lisp or vice versa. I'm simply pointing out that a language choice is probably not something worth making at an Enterprise level. The same is true when looking for the NBL, you aren't likely to find one that is perfect for solving all possible problems.

So then the question becomes, of the various NBLs of the future, which one is designed to address the problems you are interested in? If multi-processors and concurrency interest you, Erlang might be the ticket. If you love building web applications, Ruby/Rails isn't likely to go away any time soon. Or, if you want to build on years of experience with the same language, Smalltalk and Lisp have plenty of lessons for you to learn.

In my opinion it doesn't matter what NBL is. Most languages have unique solutions that allow you see problems in an entirely new way. That's why the Pragmatic Programmers suggest that you learn a new language every year. So instead of looking for the NBL, learn a language with a paradigm that differs from the one you use at your day job. For example, if you are currently using an OO language, learn a functional language. The established languages of today have plenty of lessons for you to learn. Knowledge of Ruby, Smalltalk, C#, Java, Lisp, Erlang, PHP, Perl, etc will ensure that you will quickly become proficient in any language you adopt in the future.

If you still insist on trying to predict the future, here's something to consider. As an industry we spent many years learning the lessons of procedural languages. Next, the majority learned the lessons of statically typed object oriented languages. Now, more than ever, we are learning the lessons of dynamically typed object oriented languages. If I had to guess, I think sooner or later the masses are going to figure out that functional languages also have power to offer.