Strange Symphonies The best way to predict the future is to invent it

Taming the autotest Beast with FSEvents

autotest is a great tool and all, but it is simply resource intensive. Due to autotest's implementation, it eats up CPU resources, not because the tests are always running (only after you modify your file), but because autotest continually polls each file in your directory, and sub directories, and checks to see if it has been modified.

This continual polling isn't good for CPU resources. Now I may have some spare cycles to let autotest do its thang, but it also doesn't sound too healthy for my hard drive.

Note: This is Mac OS X 10.5 Leopard specific.

Along Comes a Leopard

Whilst reading the Arstechnica review on Leopard, I came upon the section on the File System Events (FSEvents) that was introduced in 10.4 actually (for Spotlight), but used once again for Time Machine. In Mac OS X 10.5 Leopard, the API was opened up for the public to consume.

File System Events (FSEvents)

In its simplest level, your application will notify FSEvents, that it wants to listen to a particular directory, and when that directory (or its sub directories) are modified, FSEvents basically triggers a callback in your application. This allows you to hook to, listen to file system changes, and react to accordingly.

This is exactly what I needed to calm the fury of the autotest beast.

Install RubyCocoa

Note: It looks like the code works on the stock Ruby and RubyCocoa out of the box actually. You don't need to install the latest version.

First of all, you need to install RubyCocoa, as this provides us with the bindings required to communicate with FSEvents. As I installed Ruby via MacPorts, I opted to do a source install (the MacPorts at present, is one version behind). I did run into trouble, encountering this error "file is not of required architecture".

If you ever needed a reason to actually use RubyCocoa, let this be your reason!

Taming the beast

Once installed, dump this little gem into your ~/.autotest file:

class Autotest
  def run
    hook :run
    reset

    run_tests

    require 'osx/foundation'
    OSX.require_framework '/System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework'

    callback = proc do |stream, ctx, numEvents, paths, marks, eventIDs|
      paths.regard_as('*')
      rpaths = []

      numEvents.times { |i| rpaths << paths[i] }

      run_tests_in_paths(*rpaths)
    end

    allocator = OSX::KCFAllocatorDefault
    context   = nil
    path      = [Dir.pwd]
    sinceWhen = OSX::KFSEventStreamEventIdSinceNow
    latency   = 1.0
    flags     = 0

    stream   = OSX::FSEventStreamCreate(allocator, callback, context, path, sinceWhen, latency, flags)
    unless stream
      puts "Failed to create stream"
      exit
    end

    OSX::FSEventStreamScheduleWithRunLoop(stream, OSX::CFRunLoopGetCurrent(), OSX::KCFRunLoopDefaultMode)
    unless OSX::FSEventStreamStart(stream)
      puts "Failed to start stream"
      exit
    end

    OSX::CFRunLoopRun()
  rescue Interrupt
    OSX::FSEventStreamStop(stream)
    OSX::FSEventStreamInvalidate(stream)
    OSX::FSEventStreamRelease(stream)
  end

  def find_files_in_paths(*paths)
    current_dir = Dir.pwd.length + 1
    result = {}
    paths.each do |path|
      # Largely copied from autotest
      Find.find path do |f|
        Find.prune if @exceptions and f =~ @exceptions and test ?d, f
        next if test ?d, f
        next if f =~ /(swp|~|rej|orig)$/
        next if f =~ /\/\.?#/

        filename = f[current_dir..-1]
        result[filename] = File.stat(filename).mtime rescue next
      end
    end
    return result
  end

  def run_tests_in_paths(*paths)
    find_files_to_test(find_files_in_paths(*paths))
    return if @files_to_test.empty?
    # Copied from autotest
    cmd = make_test_cmd @files_to_test

    hook :run_command
    puts cmd

    old_sync = $stdout.sync
    $stdout.sync = true
    @results = []
    line = []
    begin
      open("| #{cmd}", "r") do |f|
        until f.eof? do
          c = f.getc
          putc c
          line << c
          if c == ?\n then
            @results << line.pack("c*")
            line.clear
          end
        end
      end
    ensure
      $stdout.sync = old_sync
    end
    hook :ran_command
    @results = @results.join

    handle_results(@results)
  end
end

Now when you run autotest, you'll run into something like this:

/Users/aizat/.autotest:4: warning: method redefined; discarding old run

Don't worry about it, and feel free to ignore it.

With this code, you can also tame the beast. There you have it, a much saner autotest, only for Mac OS X 10.5 Leopard.