Ruby: Cucumber + Rake + some useful stuff

Cucumber is just another Ruby-based test engine which gives the ability to write human language automated tests. But this is only one particular aspect. But there\'s another aspect. Cucumber is just another one auxilary utility as many others. As many others Ruby-based solutions the Cucumber can be integrated into some infrastructure. In particular there\'s ability to define some specific tasks to be done. In addition to this sometimes it\'s necessary to check if overall solution is built without any errors, get the list of the tests marked with some specific tag or list the steps which do not have step definitions implemented or so. All this stuff is quite useful for tests development.

First of all it would be great to run some specific tasks in more convenient and compact way than standard Cucumber command line. In particular, in case there\'s necessity to specify some tags in addition to features folder, it  would be nice to have some command like:

some_command [task_name [parameters]]

For this purpose we can use such Ruby gem as Rake. It can be installed by the following command:

gem install rake

Rake is some kind of analog of Make utility (if we use C++), ANT ( if we use Java)  etc. This utility uses files of specific type where some tasks to execute are defined. Even more, miscellaneous Ruby gems have specificly reserved class for Rake where the we can specify the execution options which Rake will use while executing this gem. Cucumber is not an exception for this. It has Cucumber::Rake::Task class which is base class for tasks executing Cucumber tests. The only thing left is to set command line options we need for each particular task.

So, let\'s say we have some working directory named working_dir containing  features sub-folder where our tests are stored. Let\'s create file named Rakefile (it\'s one of the default file names used by rake) in the  working_dir folder and update recently created file with the following contents:

require "cucumber/rake/task"

Cucumber::Rake::Task.new(:run) do |task|
  task.cucumber_opts = ["features"]
end

OK, we\'ve created new task running all the tests inside features folder. Let\'s go to the working_dir folder and start the following command line:

rake run

This command line will run the Rake task defined in Rakefile file and named as  run. In this example the command line above does the same as standard Cucumber command line like::

cucumber features

At the moment the optimization is not so obvious but it would be seen more clearly in case we use some additional options to run the tests. For instance, let\'s say all our tests are marked with specific tags corresponding to some groups/sub-groups of tests. And we have some basic tag which marks all the tests we use for execution. Let\'s name this tag as @all. Then the Cucumber command line starting all the tests is represented in the following way:

cucumber -t @all features

After such modifications let\'s update our Rakefile file with the following:

require "cucumber/rake/task"

Cucumber::Rake::Task.new(:run) do |task|
  task.cucumber_opts = ["-t","@#{ENV["TAG"] || "all" }","features"]
end

The changes are highlighted with the bold. We\'ve just added 2 options:

1) option specifying that the next few options are the list of tags to be used

2) the actual tag name.

The command line running tests under @all tag is still the same:

rake run

But if we want to run the tests under some tag different from @all ( for example, @test ) then it just needs to type command line like:

rake run tag=test

And this would be the same as the command line like

cucumber -t @test features

And this is only beginning. Let\'s see how we can verify if all the tests in the  features folder are compilable and ready for execution. In most cases we can expect the compilation errors when different keywords we define match several patterns in step definitions (redundant steps). Actually, Cucumber picks up all the features defined in the specified folder and processes everything. But only tests marked with the tags specified in cucumber command line are executed. So, we have to create some feature-file containing only 1 empty instruction. This test will fail only in case some compilation errors or redundant steps take place. But if it passed it means that all the set of tests is ready to use. At least it can be executed.

OK. Let\'s create compile.feature file in the working_dir/features folder and fill it in with the following content:

@compile
Feature: Compilation tests
    Scenario: Compile Test
        Given Build is OK

After that let\'s create compile.rb file in the working_dir/features/step_definitions folder and put the follwoing lines into it:

Given /Build is OK/ do
end


That\'s all. Now we can go back to working_dir folder and start the command line like:

rake run tag=compile

If there\'re no compilation errors, well, at least the tests can be run. Taking into account the fact that the task verifying compilation correctness is quite specific as well as frequently used (especially during new tests development), there\'s a reason to make separate Rake task for activity like that. Let\'s update our Rakefile file with the lines like:

require "cucumber/rake/task"

Cucumber::Rake::Task.new(:run) do |task|
  task.cucumber_opts = ["-t","@#{ENV["TAG"] || "all" }","features"]
end

Cucumber::Rake::Task.new(:compile) do |task|
  task.cucumber_opts = ["-t","@compile","features"]
end

After these changes ( the changes are highlighted with the bold) the following command line can start the compilation test:

rake compile

Well, optimization is clearly seen if we compare the line above with corresponding Cucumber command line.

Another useful feature is the ability to list all the tests under specified tag as well as the list of steps which do not have Ruby implementation in step definitions.

For this purpose we need to create our own reporter class. First of all let\'s make an agreement that we use Cucumber of  0.3.101, version. In the later versions the Visitor class we\'re going to use became deprecated so the code we\'ll create later wouldn\'t work on the later versions.

I\'d like to point out that all the user-defined classes for Cucumber formatted output should be inherited from Cucumber::Ast::Visitor class. Inherited class should contain some set of base methods which are supposed to be executed before/after feature file processing, before/after step execution etc. For our task we need to override only 4 methods:

• initialize
• visit_feature
• visit_feature_element
• visit_step_result

Let\'s create print_steps file working_dir/features/support folder and implement our report formatter in the following way:

class PrintSteps < Cucumber::Ast::Visitor

  def initialize(step_mother, io, options)
    super(step_mother)
  end

  def visit_feature(feature)
    puts feature.name
    super
  end

  def visit_feature_element(feature_element)
    puts "\\\\t#{feature_element.name}"
    super
  end

  def visit_step_result(keyword, step_match, multiline_arg, status, exception, source_indent, background)
    puts "\\\\t\\\\t#{step_match.name}" if status == :undefined
  end
end

After that we go back to working_dir folder and run the command line like:

cucumber -d -f PrintSteps features

And after this the console will display the list of features, scenarios and undefined steps. And each component will be displayed with specific outline.

Such feature is useful in order to control the overall amount of the tests as well as always keep an eye if all the steps are implemented. Also, we can check if the changes we made didn\'t corrupted the tests.

Let\'s create Rake task listing features, scenarios and undefined steps for some speciifc tag. Rakefile file should be updated in the following way:

require "cucumber/rake/task"

Cucumber::Rake::Task.new(:run) do |task|
  task.cucumber_opts = ["-t","@#{ENV["TAG"] || "all" }","features"]
end

Cucumber::Rake::Task.new(:compile) do |task|
  task.cucumber_opts = ["-t","@compile","features"]
end

Cucumber::Rake::Task.new(:list) do |task|
  task.cucumber_opts = ["-d", "-f", "PrintNames", "-t","@#{ENV["TAG"] || "all" }","features"]
end

After the changes are made we can list the features, scenarios and undefined steps for the tests under @all tag by means the following command line:

rake list

In case we have to do the same thing but for some specific tag (for instance, @test ) it can be easily done by the following command:

rake list tag=test

That\'s all for now. We have auxilary solutions helping us to keep an eye on efficiency of our Cucumber tests. The tasks we\'ve created can be easily used either locally or on some auto-build server. For instance the rake compile compile command execution may be setup to be triggered by each check-in operation.