Testacular - Spectacular Test Runner for JavaScript
Tuesday, November 27, 2012
We found that we were struggling with existing tools, so we decided to write our own test runner. We wanted a test runner that would meet all of our needs for both quick development and continuous integration -- a truly spectacular test runner. We've called it Testacular.
Let's walk through some mental tests for what we believe makes for an ideal test runner...
it(‘should be fast’)
In order to be productive and creative you need instant feedback. Testacular watches the files in your application. Whenever you change any of them, it immediately executes the specified tests and reports the results. You never have to leave your text editor.
This enables a new way of developing. Instead of moving back and forth between the editor and the browser, you can simply stay in the editor and experiment. You instantly see the results at the command line whenever your changes are saved.
Besides that, our experience says that if test execution is slow, people don’t write tests. Testacular eliminates many barriers that keep folks from writing tests. When developers get instant feedback from their tests, the tests become an asset rather than annoyance.
it(‘should use real browsers’)
JavaScript itself is pretty consistent between different browsers, so one could potentially test browser code in non-browser environments like Node.js. Unfortunately, that’s not the case with the DOM APIs. AngularJS does a lot of DOM manipulation, and we need to be sure that it works across browsers. Executing tests on real browsers is a must.
And because Testacular communicates with browsers through a common protocols (eg. HTTP or WebSocket), you can test not only on desktop browsers but also on other devices such as mobile phones and tablets. For instance, the YouTube team uses Testacular to run continuous integration builds on PlayStation 3.
Another advantage of using real browsers is that you can use any of the tools that the browser provides. For example, you can jump into a debugger and step through your test.
it(‘should be reliable/stable’)
To be honest, most of these ideas were already implemented in JsTD almost three years ago. I think these are truly great ideas. Unfortunately, the implementation was flaky. It’s very easy to get JsTD into an inconsistent state, so you end up restarting it pretty much all day.
Testacular solves that. It can run for days, without restarting. That’s because every test run is executed in a fresh iframe. It reconnects browsers that have lost their connection to the server. And yep, it can gracefully recover from other issues, like syntax errors in the code under test.
We'd like to invite you to take Testacular for a spin. You can learn a bit more in this screencast. Please let us know what you think!
The project is open sourced and developed on GitHub.
The Kirkland event marked the 7th iteration of the program which was also recently done in NYC. Kirkland however was the first time that the game had been customized to encourage exploratory testing. In the case of "Ship Wars the Test Edition," we planted 4 bugs that the engineering participants were awarded for finding. Well, we ran out of prizes and were quickly reminded that when you put a lot of testing minded people in a room, many bugs will be unveiled! One of the best unveiled bugs was not one of the four planted in the simulator. When turning your ship 90 degrees, the ship actually turned -90 degrees. Oops!
Participants were encouraged to test their spaceship built on their own machine or a Google Chromebook. While the coding was done in the browser, the simulator and web server were run on 
Sounds fun, huh? We sure hope our participants, including our first place winner shown receiving the Nexus 7 from Garret, enjoyed the evening! Beyond the battles, our guests were introduced to the revived 
