Taming the autotest Beast with FSEvents
autotest is a great tool and all, but it is simply resource intensive. Due to autotest's implementation, it eats up CPU resources, not because the tests are always running (only after you modify your file), but because autotest continually polls each file in your directory, and sub directories, and checks to see if it has been modified.
This continual polling isn't good for CPU resources. Now I may have some spare cycles to let autotest do its thang, but it also doesn't sound too healthy for my hard drive.
Note: This is Mac OS X 10.5 Leopard specific.
Along Comes a Leopard
Whilst reading the Arstechnica review on Leopard, I came upon the section on the File System Events (FSEvents) that was introduced in 10.4 actually (for Spotlight), but used once again for Time Machine. In Mac OS X 10.5 Leopard, the API was opened up for the public to consume.
File System Events (FSEvents)
In its simplest level, your application will notify FSEvents, that it wants to listen to a particular directory, and when that directory (or its sub directories) are modified, FSEvents basically triggers a callback in your application. This allows you to hook to, listen to file system changes, and react to accordingly.
This is exactly what I needed to calm the fury of the autotest beast.
Install RubyCocoa
Note: It looks like the code works on the stock Ruby and RubyCocoa out of the box actually. You don't need to install the latest version.
First of all, you need to install RubyCocoa, as this provides us with the bindings required to communicate with FSEvents. As I installed Ruby via MacPorts, I opted to do a source install (the MacPorts at present, is one version behind). I did run into trouble, encountering this error "file is not of required architecture".
If you ever needed a reason to actually use RubyCocoa, let this be your reason!
Taming the beast
Once installed, dump this little gem into your ~/.autotest file:
class Autotest
def run
hook :run
reset
run_tests
require 'osx/foundation'
OSX.require_framework '/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework'
callback = proc do |stream, ctx, numEvents, paths, marks, eventIDs|
paths.regard_as('*')
rpaths = []
numEvents.times { |i| rpaths << paths[i] }
run_tests_in_paths(*rpaths)
end
allocator = OSX::KCFAllocatorDefault
context = nil
path = [Dir.pwd]
sinceWhen = OSX::KFSEventStreamEventIdSinceNow
latency = 1.0
flags = 0
stream = OSX::FSEventStreamCreate(allocator, callback, context, path, sinceWhen, latency, flags)
unless stream
puts "Failed to create stream"
exit
end
OSX::FSEventStreamScheduleWithRunLoop(stream, OSX::CFRunLoopGetCurrent(), OSX::KCFRunLoopDefaultMode)
unless OSX::FSEventStreamStart(stream)
puts "Failed to start stream"
exit
end
OSX::CFRunLoopRun()
rescue Interrupt
OSX::FSEventStreamStop(stream)
OSX::FSEventStreamInvalidate(stream)
OSX::FSEventStreamRelease(stream)
end
def find_files_in_paths(*paths)
current_dir = Dir.pwd.length + 1
result = {}
paths.each do |path|
# Largely copied from autotest
Find.find path do |f|
Find.prune if @exceptions and f =~ @exceptions and test ?d, f
next if test ?d, f
next if f =~ /(swp|~|rej|orig)$/
next if f =~ /\/\.?#/
filename = f[current_dir..-1]
result[filename] = File.stat(filename).mtime rescue next
end
end
return result
end
def run_tests_in_paths(*paths)
find_files_to_test(find_files_in_paths(*paths))
return if @files_to_test.empty?
# Copied from autotest
cmd = make_test_cmd @files_to_test
hook :run_command
puts cmd
old_sync = $stdout.sync
$stdout.sync = true
@results = []
line = []
begin
open("| #{cmd}", "r") do |f|
until f.eof? do
c = f.getc
putc c
line << c
if c == ?\n then
@results << line.pack("c*")
line.clear
end
end
end
ensure
$stdout.sync = old_sync
end
hook :ran_command
@results = @results.join
handle_results(@results)
end
end
Now when you run autotest, you'll run into something like this:
/Users/aizat/.autotest:4: warning: method redefined; discarding old run
Don't worry about it, and feel free to ignore it.
With this code, you can also tame the beast. There you have it, a much saner autotest, only for Mac OS X 10.5 Leopard.
autotest-ting your Rails Application with Visual and Audio Feedback using Growl and mpg321
autotest is a commandline tool that comes packaged together with ZenTest. Designed to continually test your Rails application, autotest makes a slight notch easier by removing the need to repeatedly call the rake task to run your application's various tests.
Note: autotest is not limited to the default Ruby Test::Unit::TestCase. It will even work out of the box with RSpec.
Installing autotest
autotest is actually in the ZenTest gem.
sudo gem install -y ZenTest
Go to your application root directory and run autotest and watch it go!
autotest
Dead simple right? But still slightly irritating having to switch back to the terminal to see the results of test. So lets spruce it up a little, lets make it notify us of the results.
Visual Feedback with Growl
Growl is a notification that is over layed on top of your desktop, so that other applications are able to notify/inform you of anything, generally updates. For example Adium uses it to notify you of people logging in or out.
Note: Growl is a Mac OS X application. For other platforms you'll have to look at integration with their notification apps, for example knotify, or gnome-notification.
Combined together with Growl, you will continuously be notified of the current status of your test suite through a nice non disruptive interface. Thus helping to ensure the integrity of your code base.
Installing Growl
Download Growl and install it. But don't eject the Disc Image yet. We have to install the growlnotify command as well. This has to be done via the command line, so pull up your Terminal again.
We need to find out where Growl has been mounted to.
mount | grep -i growl
Possible Result:
/dev/disk2s2 on /Volumes/Growl 1.1.2 (local, nodev, nosuid, read-only, mounted by aizat)
From here you can see it has been mounted on /Volumes/Growl 1.1.2. Now go back to your Terminal, and we'll install growlnotify.
cd "/Volumes/Growl 1.1.2/" cd Extras/growlnotify sudo ./install.sh
By default growlnotify is installed into /usr/local/bin, your applications may not be able to see that this exists. So let's find out.
Execute:
which growlnotify
Desired Result:
/usr/local/bin/growlnotify
Possible Result:
no growlnotify in /opt/local/bin /opt/local/sbin /usr/local/bin /bin /sbin /usr/bin /usr/sbin
We want the desired result, so what do you do if you don't get it?
echo "export PATH=/usr/local/bin:$PATH" >> ~/.bashrc
Now growlnotify is accessible by all new Terminal sessions.
Let's give it a shot, and test out Growl!
growlnotify -m "Hey <a href="http://en.wikipedia.org/wiki/Tony_the_Tiger">Tony</a>, isn't this just grrrrrrreat?
Integrating Growl with autotest
We need to create a .autotest (yes with the period) file in your home directory.
touch ~/.autotest open ~/.autotest
Now stuff this in there!
module Autotest::Growl
def self.growl title, msg, img, pri=0, sticky=""
system "growlnotify -n autotest --image #{img.inspect} -p #{pri} -m #{msg.inspect} #{title} #{sticky}"
end
Autotest.add_hook :ran_command do |at|
results = [at.results].flatten.join("\n")
output = results.slice(/(\d+)\s+examples?,\s*(\d+)\s+failures?(,\s*(\d+)\s+not implemented)?/)
if output
if $~[2].to_i > 0
growl "Test Results", "#{output}", File.join(ENV['HOME'], %w[Library Application\ Support autotest rails_fail.png]), 2
else
growl "Test Results", "#{output}", File.join(ENV['HOME'], %w[Library Application\ Support autotest rails_ok.png])
end
end
end
end
Note: Adapted from Wincent Knowledge Base. I also took the personal liability to move files to the ~/Library/Application Support/ directory as I thought it would be more appropriate. Your choice, just change as desired.
Now for the final touch, the elusive images!
mkdir -p ~/Library/Application\ Support/autotest cd ~/Library/Application\ Support/autotest curl -O http://blog.internautdesign.com/files/rails_fail.png curl -O http://blog.internautdesign.com/files/rails_ok.png
Note: If you want, you can even change the images to whatever you want yourself, just change lines 12 and 14 to point to the image location.
autotest-ing
Now go to the root directory of your Rails application, and simply execute:
autotest
and you should soon be notified of the results of your test.
Note: You can further customize Growl in the System Preferences!
Audio Feedback
Visual feedback is cool, but it would be even more awesome if it had audio feedback to accompany it.
Note: This method is cross platform.
This was first described on FozWorks (read on how to Install)
As I aggregated my autotest images into the ~/Library/Application Support/autotest directory, I thought I'd dump the sound files in there as well. Just pay attention when you have to modify your ~/.autotest to accommodate the different path.
The only disappointment with the default sounds they provide is that, they are a little bit soft, and is often drowned out by my music player. But no worries, you can decide to use your own effects, or if your like me, increase the gain with Audacity.
In the mean time, anyone have some interesting replacement sound effects?
autotest-ing your Rails Plugin
autotest is a great tool to easily test your Rails application. autotest runs in the background and continuously test your app, and notify you of the results, thus leaving you to build your app with the confidence of knowing that it isn't going to break without your knowledge, and as soon as possible. It makes writing your tests easier, and the easier it is, the more likely you'll end up doing it.
Note: I have only used autotest with RSpec, and all details are based on that. I also assume that your plugin is installed in vendor/plugins
I have been working on a plugin that uses RSpec to help me test the plugin's integrity. After a while I got a little tired of continuously running a rake task to test it out.
Sadly by default, autotest doesn't test your plugin directory. What a shame, but it also provides a challenge!
Enabling autotest on your Rails plugin
Now go into your plugin directory and create a folder called autotest.
cd vendor/plugins/secret_sauce/ mkdir autotest
Inside the autotest directory, create a file called discover.rb and dump this little gem inside:
$:.push(File.join(File.dirname(__FILE__), %w[.. .. rspec])) Autotest.add_discovery do "rspec" end
While your in the root directory of your plugin, in my case its secret_sauce, just run autotest.
autotest
Boom! You are now autotest-ing your Rails plugin! Sweet.
Using your Application's RSpec Options
You'll notice that your autotests lack a bit of color... Or perhaps you want it to run the same options as your application. Have no worry soldier! First go to your spec directory in your plugin, and create a symbolic link back to the original spec.opts file.
Using your application's RSpec options:
cd vendor/plugins/secret_sauce/spec ln -s ../../../../spec/spec.opts
Note: Note: This only works for Unix-like operating systems, thats Mac OS X, Linux, and FreeBSD to name a few. For you Windows folks, you will have to just create a spec.opts file, or change your OS.
If you want to run under another set of options, just create a spec.opts file in the spec directory of your plugin, and fill in the details.
One shortcoming
Sure one shortcoming is that it's not integrated when calling autotest in your application root directory, but something is always better than nothing.
