You are using an outdated browser which puts all net citizens at risk. As an incentive to upgrade to a current and thus much more secure product (we recommend the free Firefox browser), you won't be able to visit this site in its cute design, but in this rather boring printer-ready version only. Thank you for considering a browser update!

Pimp Your .autotest

This blog article is out of date, check out Mac-friendly Autotest instead.

BDD is Fun

Besides the positive impact on code quality, BDD has a fun factor of it’s own. It adds some immediate and colorful feedback to the work cycles and makes me feel like being the pilot of my code.

Setting things up is quite easy nowadays, but as many howtos out there have come to a certain age, here’s one for RSpec 1.2 and Cucumber 0.3. Test::Unit is supported as well. (This is work in progress, so please feel free to post your comments and check back as I’m going to update this post for a while.)


This howto assumes that you are working on a Rails application and that your workstation is running Mac OS X with the latest and greatest version of Growl installed.


First install the gems necessary to work with Ruby on Rails applications:

gem install rspec rspec-rails cucumber webrat ZenTest

Webrat is optional, but you don’t want to miss it. Now define some environment variables:

echo "export AUTOFEATURE=true" >>~/.profile
echo "export RSPEC=true" >>~/.profile
source .profile

If you prefer to set these in your application, add the following lines to the top of config/environments/test.rb instead:

ENV['RSPEC'] = 'true'

Next are the images which will be displayed in Growl notifications:

mv autotest_images ~/.autotest_images

And finally create the configuration file ~/.autotest:

module Autotest::Growl

  def self.growl title, msg, img, pri=0, stick=""
    system "growlnotify -H localhost -n autotest --image #{img} -p #{pri} -m #{msg.inspect} #{title} #{stick}"

  Autotest.add_hook :run_command do
    @label = File.basename(Dir.pwd).upcase
    @run_scenarios = false
    print "\n"*2 + '-'*80 + "\n"*2
    print "\e[2J\e[f"   # clear the terminal

  Autotest.add_hook :ran_command do |autotest|
    gist = autotest.results.grep(/\d+\s+(example|test)s?/).map {|s| s.gsub(/(\e.*?m|\n)/, '') }.join(" / ")
    if gist == ''
      growl "#{@label} cannot run tests", '', "~/.autotest_images/ruby_grey.png"
      if gist.match /[1-9]\d*\s+(failure|error)/
        growl "#{@label} fails some tests", "#{gist}", "~/.autotest_images/ruby_red.png"
      elsif gist.match /pending/
        growl "#{@label} has pending tests", "#{gist}", "~/.autotest_images/ruby_yellow.png"
        @run_scenarios = true
        growl "#{@label} passes all tests", "#{gist}", "~/.autotest_images/ruby_green.png"
        @run_scenarios = true

  # FIXME: This is a temporary workaround until Cucumber is properly integrated!
  Autotest.add_hook :waiting do |autotest|
    if @run_scenarios
      gist = autotest.results.grep(/\d+\s+(scenario|step)s?/).map {|s| s.gsub(/(\e.*?m|\n)/, '') }.join(" / ")
      if gist == ''
        growl "#{@label} cannot run scenarios", '', "~/.autotest_images/ruby_grey.png"
        if gist.match /failed/
          growl "#{@label} fails some scenarios", "#{gist}", "~/.autotest_images/ruby_red.png"
        elsif gist.match /undefined/
          growl "#{@label} has undefined scenarios", "#{gist}", "~/.autotest_images/ruby_yellow.png"
          growl "#{@label} passes all scenarios", "#{gist}", "~/.autotest_images/ruby_green.png"


Autotest.add_hook :initialize do |autotest|
    %w{.svn .hg .git vendor}.each {|exception| autotest.add_exception(exception) }

A few things to note here:

  • ZenTest does not yet really support Cucumber. However, this workaround does the trick if you use RSpec or Test::Unit as well.
  • The Growl message titles may be too long for some display styles, a good choice is “Music Video”.
  • The run_command hook clears the terminal every time before tests are ran, however, you can still scroll up to previous output. Every test run is separated with a line of dashes.
  • The “cannot run tests/scenarios” message is shown e.g. when you have a serious syntax error in your specs like do-end nesting typos.
  • The -H localhost argument is a workaround for a glitch which lets Growl 1.1.4 on Mac OS X 10.5 randomly ignore some notifications. However, it will only work if you check “System Preferences -> Growl -> Network -> Listen for incoming notifications”. (You’ll get a NSPortTimeoutException from growlnotify if you don’t do this.) Furthermore, when using this setup for the first time, you have to check “System Preferenes → Growl → Network → Allow remote application registration” as well. Once autotest is in the list of Growl applications, you can uncheck this again.


Just change into the RAILS_ROOT directory and fire up autotest:


Whenever you save a file of your Rails project, all relevant tests will be ran and you get both a Growl feedback and detailed test results in the terminal.

(Sven Schwyn)


Peter Hellberg said on Wednesday, April 15, 2009:

I also like to add:

if name == 'trunk'
  name = Dir.pwd.split(/\//)[-2]

So that I get the correct name if working from the subversion trunk directory

Mathieu said on Thursday, April 16, 2009:

I needed to remove the “-n autotest” option in the .autotest script to have it working (ruby 1.8.6 growl 1.1.4 and Leopard fully updated).

Re: The “-n autotest” part tells Growl to pretend the messages from growlnotify are coming from autotest directly. This means if you want to change the behavior of Growl only for autotest messages, go to “System Preferences -> Growl -> Applications” and look for “autotest” in the list. If it’s not there or the specific settings are faulty, things will misbehave.

MarkB said on Thursday, April 16, 2009:

This is all fantastic! The export path was exactly what I was looking for . . . And I was so excited to finally hear there might be a way to stop that random notification ignoring – but I am having a small problem with the notifications throwing an exception:

2009-04-15 22:44:00.703 growlnotify[48688:613] Exception: NSPortTimeoutException

Any idea what is going on?

Re: You have to check “System Preferences -> Growl -> Network -> Listen for incoming notifications” (as described above), otherwise Growl is deaf on that port.

Rafi Jacoby said on Tuesday, April 21, 2009:

@MarkB: I got Growl to work when I also checked “Allow Remote Application Registration”

Re: Indeed, when you use this setup for the first time, remote application registration must be checked. I’ve added it to the post, thanks a bunch for the hint.

Rafi Jacoby said on Tuesday, April 21, 2009:

This is a pretty useful post, but I found for our setup it doesn’t throw up a red growl when there are errors in the tests.

Made the following changes:

output = filtered.empty? ? '' : filtered.last.slice(/(\d+)\s.*tests?,\s(\d+)\s.*failures?,\s(\d+)\s.*errors?(?:,\s(\d+)\s.*pending)?/) if output =~ /[1-9]\d*\s(failure|error)/

Re: I’ve just updated .autotest to support Test::Unit as well – using slightly more elegant expressions.

LukeC said on Thursday, April 23, 2009:

What is the relevance of this line ? filtered = autotest.results.grep(/\d+\s.*examples?/)

I’m guessing you’re not using test::unit

Re: That’s right, however, I’ve just updated .autotest to support Test::Unit as well.

Luke Cowell said on Monday, April 27, 2009:

Thanks! That works for Test::Unit perfectly.

(We are remaking our web presence and therefore comments are temporary disabled.)