Strange Symphonies The best way to predict the future is to invent it

Related tags

15May/101

Beginning WordPress Automated Testing

I've been a heavy user of WordPress. This blog has been using WordPress for almost the past five years. I've developed multiple plugins and themes for WordPress. Plus I've been a heavy advocate of WordPress for many of my clients work as that is all they really require.

Though I've been a heavy user, I haven't really contributed code to WordPress.

Coming from a Ruby on Rails background, I was familiar with testing. Whilst working with WordPress, I felt a bit insecure that there were no available tests out there.

Lucky for me, with a bit of research I found WordPress did have Automated Testing. When first running the tests, I didn't get 100% success rate. Now this freaked me out.

Soon I learned that sadly not all the tests pass, and the test are outdated. Hopefully there will be plans to update the tests, but in the mean time, here is how to get the tests started.

Dependencies:

  • SubVersion
  • PHPUnit
  • Empty MySQL Database

Installing PHPUnit

WordPress Automated Testing has a dependency on PHPUnit, so we are required to install the package.

Open up your terminal, and as root execute:

pear channel-discover pear.phpunit.de
pear channel-discover pear.symfony-project.com
pear install phpunit/PHPUnit

Installing WordPress Automated Testing

Pull down the Automated Testing code like so:

svn http://svn.automattic.com/wordpress-tests/

This should create a wordpress-tests folder. Now go inside it.

Create a MySQL Database

The WordPress Automated Testing requires an empty database. That means the database should have no records, no tables, nothing.

I simply created a new MySQL Database called wordpress-tests:

mysqladmin -u root -p create wordpress-tests

Now we are almost good to go.

Configuring WordPress Tests

Inside the WordPress Tests directory, copy the wp-config.sample.php to wp-config.php. This configuration is similar to normal WordPress installations. Fill in the necessary details, and you are almost good to go.

Running Your Tests

Once your tests are configured, you can run the tests via executing:

php wp-test.php

Failing Tests

Don't be surprised that some tests fail. Apparently its a normal thing. I messaged the WordPress Testers Mailing List about the tests failing. Judging from the reply I got, apparently its "right".

My test failure rate is:

FAILURES!
Tests: 489, Assertions: 2919, Failures: 136, Errors: 32, Skipped: 18.

Overall it takes about 6 and a half minutes to run.

Conclusion

There you have it, automated testing for WordPress. Hopefully by posting this, more and more people will contribute back to WordPress, and improve the automated testing.

19Nov/070

autotest-ting your Rails Application with Visual and Audio Feedback using Growl and mpg321

autotest is a commandline tool that comes packaged together with ZenTest. Designed to continually test your Rails application, autotest makes a slight notch easier by removing the need to repeatedly call the rake task to run your application's various tests.

Note: autotest is not limited to the default Ruby Test::Unit::TestCase. It will even work out of the box with RSpec.

Installing autotest

autotest is actually in the ZenTest gem.

sudo gem install -y ZenTest

Go to your application root directory and run autotest and watch it go!

autotest

Dead simple right? But still slightly irritating having to switch back to the terminal to see the results of test. So lets spruce it up a little, lets make it notify us of the results.

Visual Feedback with Growl

Growl is a notification that is over layed on top of your desktop, so that other applications are able to notify/inform you of anything, generally updates. For example Adium uses it to notify you of people logging in or out.

Note: Growl is a Mac OS X application. For other platforms you'll have to look at integration with their notification apps, for example knotify, or gnome-notification.

Combined together with Growl, you will continuously be notified of the current status of your test suite through a nice non disruptive interface. Thus helping to ensure the integrity of your code base.

Installing Growl

Download Growl and install it. But don't eject the Disc Image yet. We have to install the growlnotify command as well. This has to be done via the command line, so pull up your Terminal again.

We need to find out where Growl has been mounted to.

mount | grep -i growl

Possible Result:

/dev/disk2s2 on /Volumes/Growl 1.1.2 (local, nodev, nosuid, read-only, mounted by aizat)

From here you can see it has been mounted on /Volumes/Growl 1.1.2. Now go back to your Terminal, and we'll install growlnotify.

cd "/Volumes/Growl 1.1.2/"
cd Extras/growlnotify
sudo ./install.sh

By default growlnotify is installed into /usr/local/bin, your applications may not be able to see that this exists. So let's find out.

Execute:

which growlnotify

Desired Result:

/usr/local/bin/growlnotify

Possible Result:

no growlnotify in /opt/local/bin /opt/local/sbin /usr/local/bin /bin /sbin /usr/bin /usr/sbin

We want the desired result, so what do you do if you don't get it?

echo "export PATH=/usr/local/bin:$PATH" >> ~/.bashrc

Now growlnotify is accessible by all new Terminal sessions.

Let's give it a shot, and test out Growl!

growlnotify -m "Hey <a href="http://en.wikipedia.org/wiki/Tony_the_Tiger">Tony</a>, isn't this just grrrrrrreat?

Integrating Growl with autotest

We need to create a .autotest (yes with the period) file in your home directory.

touch ~/.autotest
open ~/.autotest

Now stuff this in there!

module Autotest::Growl

  def self.growl title, msg, img, pri=0, sticky=""
    system "growlnotify -n autotest --image #{img.inspect} -p #{pri} -m #{msg.inspect} #{title} #{sticky}"
  end

  Autotest.add_hook :ran_command do |at|
    results = [at.results].flatten.join("\n")
    output = results.slice(/(\d+)\s+examples?,\s*(\d+)\s+failures?(,\s*(\d+)\s+not implemented)?/)
    if output
      if $~[2].to_i > 0
        growl "Test Results", "#{output}", File.join(ENV['HOME'], %w[Library Application\ Support autotest rails_fail.png]), 2
      else
        growl "Test Results", "#{output}", File.join(ENV['HOME'], %w[Library Application\ Support autotest rails_ok.png])
      end
    end
  end

end

Note: Adapted from Wincent Knowledge Base. I also took the personal liability to move files to the ~/Library/Application Support/ directory as I thought it would be more appropriate. Your choice, just change as desired.

Now for the final touch, the elusive images!

mkdir -p ~/Library/Application\ Support/autotest
cd ~/Library/Application\ Support/autotest
curl -O http://blog.internautdesign.com/files/rails_fail.png
curl -O http://blog.internautdesign.com/files/rails_ok.png

Note: If you want, you can even change the images to whatever you want yourself, just change lines 12 and 14 to point to the image location.

autotest-ing

Now go to the root directory of your Rails application, and simply execute:

autotest

and you should soon be notified of the results of your test.

Note: You can further customize Growl in the System Preferences!

Audio Feedback

Visual feedback is cool, but it would be even more awesome if it had audio feedback to accompany it.

Note: This method is cross platform.

This was first described on FozWorks (read on how to Install)

As I aggregated my autotest images into the ~/Library/Application Support/autotest directory, I thought I'd dump the sound files in there as well. Just pay attention when you have to modify your ~/.autotest to accommodate the different path.

The only disappointment with the default sounds they provide is that, they are a little bit soft, and is often drowned out by my music player. But no worries, you can decide to use your own effects, or if your like me, increase the gain with Audacity.

In the mean time, anyone have some interesting replacement sound effects?

19Nov/070

autotest-ing your Rails Plugin

autotest is a great tool to easily test your Rails application. autotest runs in the background and continuously test your app, and notify you of the results, thus leaving you to build your app with the confidence of knowing that it isn't going to break without your knowledge, and as soon as possible. It makes writing your tests easier, and the easier it is, the more likely you'll end up doing it.

Note: I have only used autotest with RSpec, and all details are based on that. I also assume that your plugin is installed in vendor/plugins

I have been working on a plugin that uses RSpec to help me test the plugin's integrity. After a while I got a little tired of continuously running a rake task to test it out.

Sadly by default, autotest doesn't test your plugin directory. What a shame, but it also provides a challenge!

Enabling autotest on your Rails plugin

Now go into your plugin directory and create a folder called autotest.

cd vendor/plugins/secret_sauce/
mkdir autotest

Inside the autotest directory, create a file called discover.rb and dump this little gem inside:

$:.push(File.join(File.dirname(__FILE__), %w[.. .. rspec]))

Autotest.add_discovery do
  "rspec"
end

While your in the root directory of your plugin, in my case its secret_sauce, just run autotest.

autotest

Boom! You are now autotest-ing your Rails plugin! Sweet.

Using your Application's RSpec Options

You'll notice that your autotests lack a bit of color... Or perhaps you want it to run the same options as your application. Have no worry soldier! First go to your spec directory in your plugin, and create a symbolic link back to the original spec.opts file.

Using your application's RSpec options:

cd vendor/plugins/secret_sauce/spec
ln -s ../../../../spec/spec.opts

Note: Note: This only works for Unix-like operating systems, thats Mac OS X, Linux, and FreeBSD to name a few. For you Windows folks, you will have to just create a spec.opts file, or change your OS.

If you want to run under another set of options, just create a spec.opts file in the spec directory of your plugin, and fill in the details.

One shortcoming

Sure one shortcoming is that it's not integrated when calling autotest in your application root directory, but something is always better than nothing.