In this how-to article, you will learn how to bake your own automation tests framework with ruby to test your application’s APIs. The main framework features are:

1. The ability to run the tests in parallel.
2. Automatic test report generation.
3. Multi-environments supports
4. Run the your test packs locally, and in CD/CI
5. Write your tests in a BDD style with cucumber

I will also share some pro tips to help you maintain your automation test framework and keep your test pack healthy.

Prerequisites

Along side a general knowledge on Ruby language. You will also need the following:

1. Ruby v.3 or higher

You will need Ruby installed on your development environment. I recommend using RVM - Ruby Version Manager which allows you to easily install, manage, and work with multiple ruby environments from interpreters to sets of gems.

2. Code editor or an IDE

Because Ruby is a script language you can get away with using any code editor you like, for example; Visual Studio. That’s been said, Rubymine IDE comes with many useful features out of the box.

3. Coffee

You need a good cup of coffee! that’s non negotiable!

Setting up your environment

Start by creating a new directory and initialize the Gemfile. In this article, I’ll name the automation test directory: automation-test. However, you can name it as you see fit to your project.

# make a new folder
mkdir automation-test

# move to the new folder
cd automation-test

# create a new gemfile
bundle init

There are many delicious gems you can add to your gemfile, I have picked a few from which I consider to be the bare bones for any Ruby automation test framework. Feel free to experiment with other alternatives.

bundle add config block_repeater cucumber faker httparty rake logger parallel_tests rspec rspec-expectations rubocop

We can use cucumber to automatically generate the framework folder structure:

cucumber --init

Your files/folder structure should look like this:

Your feature files (.feature) can be kept under the the root directory /features.

Pro tips:

1. Consider organizing your feature files (.feature) in a sub folders under the root directory /features.
2. You might want to add a ReadMe file. ReadMe file can contain some general information about your framework.
3. It’s a good practice to include .gitignore file. A gitignore file specifies intentionally untracked files that Git should ignore.

Configuring your framework

To save yourself the hassle of requiring each gem in the gemfile manually, we will use Bundler.require which will automatically require all gems. Go to ./support/env.rb file, where we will keep all of the framework configurations, and add the following:

require 'bundler/setup'
Bundler.require

It’s a good practice to separate your configurations from your code. To do so, create a new folder in the root project folder and call it config. In side the config folder, create a new file settings.yml which will be used to store all the default configurations.

Lastly, in ./support/env.rb file, you need to load the configurations from the settings.yml:

setting_files = Config.setting_files('./config', ENV['ENVIRONMENT'] || 'local')
Config.load_and_set_settings(setting_files)

If you need to set up a new environment, for example: CI, you can create a new setting file within the config folder. Be sure to set the environment variable name to be the same name as your new setting file.

Logging

Logging in the automation test framework is as important as logging in the application under the test. To set up the logging in the automation framework, you first need to go to ./support/env.rb file, and add the following:

$log = Logger.new(STDOUT)

$log.formatter = proc do |severity, datetime, _, message|
  date_format = datetime.strftime('%Y-%m-%d %H:%M:%S')
  log_message = "\e[31m#{message}\e[0m"

  "#{severity}: #{date_format}: #{@scenario&.name}: #{log_message} \n"
end

# A simple logger function
#
# @return [Logger] logger
def logger; $log end

Note that we are pulling the scenario name into our logging, to get the scenario name you will need to create a new hook. Create a new hooks folder under the directory /features. Inside the /hooks folder, create a new Ruby file test_config_hooks.rb. Then create a new hook:

Before do |scenario|
  @scenario = scenario
end

In order to use the logger, you will need to call the logger function:

logger.info('something')

# OUTPUT:
# INFO: 2023-09-11 11:46:21: ExampleTest: something

Pro Tip To keep your terminal clean, you can also silent http request logs:

 # In the ./support/env.rb file
HttpLog.configure { |config| config.enabled = false }

Running Tests In Parallel

We will be creating Rake tasks to run the tests. However, you can also use Rake tasks to inject your test data, or to run lint, etc. To get started, create a new file in the root folder and name it Rakefile - without any file extensions. In the Rakefile:

require 'parallel_tests'

namespace :test do
  number_of_processes = '11'
  number_of_retries = '3'
  
  desc 'Run the functional test suite locally'
  task local: %i[chutney lint] do
    ParallelTests::CLI.new.run(%W(-n #{number_of_processes} --type cucumber -- -f progress --retry #{number_of_retries} -- features))
  end
  
end

You can change the number_of_processes and number_of_retries. Note that the number_of_processes is the number of tests that will run in parallel. Where the number_of_retries is how many times the test will re-run if it fails.

Pro Tip

1. Make sure your tests are independent from each other, Otherwise you might end up with a deadlock or a flaky tests
2. In most cases, the more number of processes, the faster your tests will run and more resources your test will consume.
3. Be aware that you might face flaky tests if you run out of resources.

In order to run your rake task, open your terminal:

rake test:local

Wrapping Up

Congratulation! You have a fully working automation framework. But do not stop there, the automation test framework is juts like any other piece of software, it requires maintenance and and continues improvement.