Simple Thoughts Saimon Moore's occasional musings RSS

saimonmoore.net

Posts

Archive

Sep
3rd
Fri
2010
permalink

fairtilizer.com is now known as official.fm

I've been working on fairtilizer.com pretty much full time for the last 2 years. This august the team flipped the switch on a new branding Official.fm, the idea being massiveattack.official.fm. As I was off on holidays, huge congratulations to the officialfm team are due. Hopefully will be blogging more often now (I think I said that about 2 years ago too :)
Jan
24th
Thu
2008
permalink

Created mongrel dont-serve-static patch ticket

If anyone’s interested, I’ve updated the patch for mongrel trunk (r949 / REL_1-1-3) and posted it here to the mongrel project

I’m hoping this get’s in. Am I too hopefull? :)

Jan
23rd
Wed
2008
permalink

Mocking ActiveRecord::RecordInvalid with mocha

For future reference

1
          2
          3
          4
          5
          6
          7
          8
          

            def mock_record_invalid(model)
              user_errors, record_invalid_error = mock('errors'), mock('record_invalid_error')
              user_errors.stubs(:full_messages).returns([])
              model.stubs(:errors).returns(user_errors)
              record_invalid_error.stubs(:exception).returns(ActiveRecord::RecordInvalid.new(model))
              record_invalid_error
            end
Jan
18th
Fri
2008
permalink

RSpec Story Runner Driven (Browser) Acceptance Testing

I’ve been using rspec now for quite a while now (thanks to chrissturm) and have been loving it. It feels a lot more natural and intuitive and I’m even getting the hand of learning when/how to mock/stub (though I still have some fixtures lying around). I’ve been meaning to learn up on using the new story runner feature and while googling I came upon a post by Kerry Buckley in which he provides a quick overview of how to setup story runner and also describes how he got story runner to drive selenium acceptance testing.

Just a quick blurb about what story runner is.

Story runner basically allows you to write specifications in a plain text file, written in natural language. You basically write a story (paraphrasing Dan North “a description of a requirement and its business benefit, and a set of criteria by which we all agree that it is done”.)

For each story you can then write a number of different scenarios (imagine that feature in different situations) and for each scenario you write a set of criteria which determines how that scenario can be completed successfully.

e.g. 
1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          

           Story: UI
            As a developer                                                  #
            I want to go to the uimockups page                              # Description of intent
            So that I can implement the mockup                              #
          
             Scenario: Going to the /uimockups page when not logged in      <= Scenario Description
               Given an anonymous user                                      #
               When the user goes to /uimockups                             #
               Then the document title should be 'personal'                 # criteria, actions & expectations
               And the page should contain the text 'done by Webtypes'      #  
               And the page should have a field named 'strip-search-input'  #
               And the page should have a form named 'strip-search'         #

Given a text file like this, you then write a small ruby script (see /stories/stories/project.rb below) which then takes the text, parses it look for the highlighted keywords. Each Given, When and Then is a Step. The Ands are each the same kind as the previous Step.

Run as is, you’ll get this same story output back to you but each of the lines under Scenario will be marked with “pending” which basically means that the story is yet to be implemented. e.g.

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          

          saimon@artemis~/dev/projects/myrailsapp$ ruby stories/stories/project.rb
          (in /Users/saimon/dev/projects/myrailsapp)
          Running 1 scenarios
          
          Story: UI
          
            As a developer
            I want to go to the ui page
            So that I can see the mockup
          
            Scenario: Going to the /ui page when not logged in
          
              Given an anonymous user (PENDING)
          
              When the user goes to /ui (PENDING)
          
              Then the document title should be 'personal' (PENDING)
              And the page should contain the text 'done by Webtypes' (PENDING)
              And the page should have a field named 'strip-search-input' (PENDING)
              And the page should have a form named 'strip-search' (PENDING)
          
          1 scenarios: 0 succeeded, 0 failed, 1 pending
          
          Pending Steps:
          1) UI (Going to the /ui page when not logged in): Unimplemented step: an anonymous user

To actually get the story to pass, you need to implement each of the Steps in ruby. i.e. Here’s an example of the implementation of the 2nd step:

  • “When the user goes to /ui (PENDING)”
1
          2
          3
          4
          5
          6
          

          steps_for(:project) do
            When "the user goes to $path" do |path|
              get path
            end
          end

As you can see it’s basically parsing the line for a step keyword (in this case ‘When’), and then takes the rest of the line and tries to match it against any of the When steps it knows about. It also goes one step further and allows you to add in variables so that you can extract dynamic criteria directly from the story line ($path) in this case.

So once, it has been matched, it ends up executing :

1
          2
          

          get /uimockups

The cool thing about it is that once you’ve implemented a step, it’s just reused every time it’s matched in the story. You can also have a stable set of steps which you use in multiple stories. You could even conceivably build up a library of them to be used in other applications.

I was at the BCN Ruby/Rails group meeting last night and one of the attendants expressed concerns about the brittleness of the syntax. In fact, there’s no problem because story runner will mark any line that it hasn’t been able to match against any of the steps known to it as pending so you can easily determine a syntax problem. And if an exception is raised by anything it has matched then it’ll provide the appropriate stack trace pointing you to the step that caused the exception.

After watching Pat Maddox’s screen-cast I’m convinced that using story runner is a good way of starting out your speccing. You can start by writing a story that describes a feature and then drill into it as you implement the steps. Along the way you’ll find you need to implement controllers, models, helpers and views and before you do you can then implement the appropriate specs (only enough to get the functionality in the story passing) which in turn drives the implementation of the object in question.

Now, finally, I can get to the real reason I wrote this post.

I’m interested in being able to do my integration tests via story runner and occasionally do the odd browser acceptance testing and as I also had been meaning to play with FireWatir & SafariWatir I decided to adapt his code to using watir.

But I added another requirement to the mix. What I really wanted was seamless integration between normal integration tests using plain rspec and browser acceptance testing hen running the same scenarios or even be able to mix and match.

After a bit I come up with this setup (very similar to Kerry’s original setup):

Directory Structure:

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          

          +-- lib
          | +-- tasks/
          |   +-- acceptance.rake
          +-- stories/
          | +-- all.rb
          | +-- helper.rb
          | +-- steps/
          | | +-- project.rb
          | | +-- watir.rb
          | +-- stories/
          | | +-- project.rb
          | | +-- project.txt

Note: As per Kerry’s article I’ve further subdivided the top-level stories directory into stories and steps subdirectories. You don’t have to if you don’t have that many stories to write but I like the organized feeling it provides.

/stories/all.rb 
1
          2
          3
          4
          5
          6
          

          dir = File.dirname(__FILE__)
          require "#{dir}/helper"
          Dir[File.expand_path("#{dir}/stories/**/*.rb")].uniq.each do |file|
            require file
          end

/stories/stories/project.txt

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          

          Story: UI
            As a developer
            I want to go to the ui page
            So that I can see the mockup
          
            Scenario: Going to the /ui page when not logged in
              When the user goes to /ui
              Then the document title should be 'personal'
              And the page should contain the text 'done by Webtypes'
              And the page should have a field named 'strip-search-input'
              And the page should have a form named 'strip-search'

/stories/stories/project.rb

1
          2
          3
          4
          

          #Call me with: [BROWSER=firefox|safari|ie] ruby stories/stories/project.rb
          require File.join(File.dirname(__FILE__), "../helper") 
          run_story_with_steps_for (browser ? [:watir_project, :project] : [:project])

/stories/steps/project.rb

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          27
          28
          29
          30
          31
          32
          

          steps_for(:project) do
            Given "a test user" do
              User.delete_all
              User.create!(:name => 'test', :openid_url => 'http://dummy.openid/',
                          :email => 'test@example.com')
            end
            
            When "the user goes to $path" do |path|
              get path
            end
            
            Then "the document title should be '$title'" do |title|
              response.should have_tag('title', title)
            end
          
            Then "the page should contain the text '$text'" do |text|
              response.should have_text(/#{text}/)
            end
            
            Then "the page should have a field named '$field'" do |field|
              response.should have_tag("input[type=text][id=?]", field)
            end
            
            Then "the page should have a form named '$form'" do |form|
              response.should have_tag("form[id=?]", form)
            end  
          
            Then "the page should have a submit button named '$name', with the label '$label'" do |name, label|
              response.should have_tag("input[type=submit][id=?][value=?]", name,label)
            end  
          end

/stories/steps/watir_project.rb

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          27
          28
          29
          30
          31
          32
          33
          34
          35
          

          steps_for(:watir_project) do
            When "the user goes to $path" do |path|
              browser.goto "http://localhost#{path}"
            end
            
            When "the user types '$text' into the $field field" do |text, field|
              browser.text_field(:name,field).set(text)
            end
            
            When "the user clicks the $button button" do |button|
              browser.button(:value, button).click
            end
          
            Then "the document title should be '$title'" do |title|
              browser.title.should == title
            end
          
            Then "the page should contain the text '$text'" do |text|
              browser.text.include?(text).should be_true
            end
            
            Then "the page should have a field named '$field'" do |field|
              (browser.text_field(:name, field).exists? || browser.text_field(:id, field).exists?).should be_true
            end
            
            Then "the page should have a form named '$form'" do |form|
              (browser.form(:name, form).exists? || browser.form(:id, form).exists?).should be_true
            end  
          
            Then "the page should have a submit button named '$name', with the label '$label'" do |name, label|
              tf = (browser.text_field(:name, field) || browser.text_field(:id, field)).exists?().should be_true
              tf.value.should == label
            end
          end

/stories/helper.rb :

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          27
          28
          29
          30
          31
          32
          33
          34
          35
          36
          37
          38
          39
          40
          41
          42
          43
          44
          45
          46
          47
          48
          49
          50
          51
          52
          53
          54
          55
          56
          57
          58
          59
          60
          

          ENV["RAILS_ENV"] = "test"
          require File.expand_path(File.dirname(__FILE__) + "/../config/environment")
          require 'spec/rails/story_adapter'
          
          # watir gem
          require 'firewatir' if ENV['BROWSER'] && ENV['BROWSER'] == 'firefox'
          require 'safariwatir' if ENV['BROWSER'] && ENV['BROWSER'] == 'safari'
          
          def start_ff
            FireWatir::Firefox.new
          end
          
          def start_safari
            safari = Watir::Safari.new
          end
          
          
          #Require steps in steps dir
          Dir[File.dirname(__FILE__) + "/steps/*.rb"].uniq.each { |file| require file }
          
          #Require appropriate watir browser object
          if !$ff && ENV['BROWSER'] == 'firefox'
            $ff = start_ff_with_logger
          end
          
          if !$sf && ENV['BROWSER'] == 'safari'
            $sf = start_safari_with_logger
          end
          
          #Choose which browser to use in steps
          def browser
            $ff || $sf
          end
          
          def run_story_with_steps_for *steps
            with_steps_for *(steps.flatten) do
              # Pull the filename of the caller out of the stack. Must be a better way.
              run caller[3].sub(/\.rb:.*/, '.txt'), :type => RailsStory
            end
          end
          
          # By default, RSpec adds an ActiveRecordSafetyListener to the story runner. 
          # This rolls back database changes between scenarios, which is great if your calling your code directly, 
          # but obviously means that if you write to the database, the server that Selenium's talking to can't see them. There's probably a cleaner way of disabling it.
          class Spec::Story::Runner::ScenarioRunner
            def initialize
              @listeners = []
            end
          end
          
          module ::ActionController #:nodoc:
            module TestProcess
              # Work around Rails ticket http://dev.rubyonrails.org/ticket/1937
              # Helps to remove annoying html parser warnings
              def html_document
                @html_document ||= HTML::Document.new(@response.body, true, true)
              end
            end
          end

So once you’ve got all that setup, you can then run:

1
          2
          3
          

          saimon@artemis~/dev/projects/myrailsapp$ 
          ruby stories/stories/project.rb

to execute the project story using basic rspec. It provides the following output:

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          

          Running 1 scenarios
          
          Story: UI
          
            As a developer
            I want to go to the ui page
            So that I can see the mockup
          
            Scenario: Going to the /ui page when not logged in
          
              Given an anonymous user
          
              When the user goes to /ui
          
              Then the document title should be 'personal'
              And the page should contain the text 'done by Webtypes'
              And the page should have a field named 'strip-search-input'
              And the page should have a form named 'strip-search'
          
          1 scenarios: 1 succeeded, 0 failed, 0 pending

Woot! You can know just take that project.txt and send it to a client, a fellow developer, a project mailing list etc…

But, let’s go the extra step and run that same scenario against Firefox (go to FireWatir and follow the instructions. They’re pretty simple.)

Start firefox with -jssh 
1
          2
          3
          4
          5
          6
          7
          8
          

          saimon@artemis~/dev/projects/myrailsapp$ 
          /Applications/Firefox.app/Contents/MacOS/firefox -jssh
          
          run the story with the BROWSER environment variable:
          
          saimon@artemis~/dev/projects/myrailsapp$ 
          BROWSER=firefox ruby stories/stories/project.rb

and watch how FF is magically commanded to go through your stories scenarios. In the end it’s run the story against FF and provides the same passing output as the previous run.

One further step is to write a few rake commands to simplify running all your stories, with or without browser acceptance testing.

Add this file:

/lib/tasks/acceptance.rake

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          12
          13
          14
          15
          16
          17
          18
          19
          20
          21
          22
          23
          24
          25
          26
          27
          28
          29
          30
          31
          32
          33
          34
          35
          36
          37
          38
          39
          40
          41
          42
          43
          44
          45
          46
          47
          

          desc "Run the acceptance tests, starting/stopping the test server."
          task :acceptance_with_browser => ['acceptance:server:start'] do
            begin
              Rake::Task['acceptance:run'].invoke
            ensure
              Rake::Task['acceptance:server:stop'].invoke
            end
          end
          %w(firefox safari).each do |browser|
            Object.class_eval <<-EOS
              desc "Run the acceptance tests using the #{browser} browser."
              task :acceptance_with_#{browser} do
                $browser = '#{browser}'
                Rake::Task['acceptance_with_browser'].invoke
              end
            EOS
          end
           
          namespace :acceptance do
            desc "Run the acceptance tests."
            task :run do
              system "#{$browser ? "BROWSER='#{$browser}' " : ''}ruby stories/all.rb"
            end
          
            namespace :server do
              desc "Start the mongrel server"
              task :start do
                system 'script/server -e test -d'
                sleep 5
              end
          
              desc "Stop the mongrel server"
              task :stop do
                if File.exist? MONGREL_SERVER_PID_FILE
                  pid = File.read(MONGREL_SERVER_PID_FILE).to_i
                  Process.kill 'TERM', pid
                  FileUtils.rm MONGREL_SERVER_PID_FILE
                else
                  puts "#{MONGREL_SERVER_PID_FILE} not found"
                end
              end
            end
          end
          
          MONGREL_SERVER_PID_FILE = 'tmp/pids/mongrel.pid'

Then you can do:

1
          2
          3
          4
          5
          6
          7
          8
          9
          10
          11
          

          saimon@artemis~/dev/projects/myrailsapp$ 
          rake acceptance:run
          
          or 
          
          rake acceptance_with_firefox
          
          or 
          
          rake acceptance_with_safari

I thoroughly enjoyed getting that setup and though I only plan on writing browser specs for specific issues/features it’s nice to have the choice and the geek factor is way up high :)

Have fun speccing…