Test Automation Infrastructure Setup

In DevOps and test automation era there are questions in minds of those who would like to adapt newer environment and like to perform test automation using available tools. Here I am talking about how to build test automation infrastructure using latest technologies like, I am not test automation expert but being an SRE experienced how to setup your own test automation infrastructure without depending on third party licensed tool. It takes some time to mature such infrastructure by continuous testing but once it becomes ready whole organization can use same or similar infrastructure and similarly other third party organisation like BrowserStack developed their own infrastructure where now they are selling it as PaaS solution in testing domain. Following are the tools they you would like to interact if you are building your solution on AWS cloud:

·       Terraform

·       Docker

·       AWS ECR (Elastic Compute Repository)

·       Selenium Web Drivers

·       Browser Stack

·       NightWatch

·       Phantom

·       NodeJs

·       Bash and Jenkins jobs.

This blog talks about how to setup Test infrastructure as PaaS and client-side Node Package Manager(npm) code that you need to write/modify so that you can write your test cases according to your org. requirement so that you can run them against UI PaaS test automation infrastructure.

Benefits of Test automation and why we require it:

As we know that organizations doing development in agile framework and on daily basis they are releasing new code and even some time with in a day they are doing multiple releases in their test environment and assume a situation where you have multiple components in your web app that require independent testing after every release so here such automation test cases comes in picture that automatically runs those tests cases after successfully build and deployment phase and inform users in form of success or failure with details report for reporting purpose.

1.      It saves your number of hours.

2.      It can perform repetitive UI testing that a human can ignore or can-do mistakes while testing due to its cumbersome test nature.

3.      It keeps testing records for reporting purpose

Scenario: you would like to enter user name and password on site and would like to see if user is successfully able to login or not. and you need to perform web UI testing using available test automation frameworks like Selenium Webdriver, BrowserStack and nightwatch. There are other tools available in the market but we found these more suitable those can easily be integrated with Selenium Web drivers

Tools that we are going to use:

1.      Test Automation Framework:      Selenium Web driver

2.      Infrastructure as code tool:          Terraform

3.      Container Technology:                  Docker and AWS ECS

4.      Deployment Tool:                           Jenkins

5.      Paid Test Automation portal:       Browser Stack

6.      Programming language:               Node.Js and related npm modules

Pre- work for Test Automation infrastructure:

Download docker Selenium image from Selenium web site and upload it to AWS ECR as Web Automation service infrastructure and that service will listen on specific port. This instance will be having capability to launch number of internal sessions for testing that can be viewed on session detail page and screen shot can be captured from specific session. It helps in getting details like on which step auto testing is happening. This service will be available organization wide and multiple teams can use this SaaS solution for their testing and you can choose among browsers that you would like to use for testing and that can be set in your node js client.

Selenium based solution require tuning and configuration according to organization requirement where as some paid portals like BrowserStack are available that simplifies test automation service but cost is associated with it and if you are planning for long term and matured test infrastructure according to your requirement then you have to invest time in building your own test infrastructure.

So, to get above work done you require

Local docker setup on which you will run Selenium image and finally upload that image to ECR so that further terraform can build this server infra. Download one of the Firefox Dockerfile from github:

·        selenium/standalone-firefox-debug (For debugging purpose VNC service is installed)

·        selenium/standalone-firefox (For Production use this image)

#Login into your linux VM or container and perform following steps to download Dockerfile to build new selenium webdriver image and modify it as per your setup(IP,Port) and finally upload it to AWS ECR:

$ git clone https://github.com/SeleniumHQ/docker-selenium.git
# Cd into correct brower folder
$ cd docker-selenium/StandaloneFirefoxDebug/

$ docker build - < Dockerfile  (Build reference)

$ docker run $ docker run -d -p 4444:4444 --shm-size 2g selenium/standalone-firefox:3.11.0-californium

Read complete details on selenium page if you are using other browser or setting as it is mentioned on that page and finally upload modified image to AWS ECR and it will look like on your local docker setup:

$ docker images

REPOSITORY                                                                                                  TAG                 IMAGE ID            CREATED             SIZE
XX219486XXXX.dkr.ecr.us-west-2.amazonaws.com/develop/user-interface-automation.selenium                    <none>              77af4d6b9913        19 hours ago        1.089 GB

 Above steps have completed your server side pre setup work so that your concerned Jenkins jobs and build correct infrastructure as per uploaded customized image.

Following is the workflow of UI testing where you will be running :

Client-Side Test Scripts To Test Functionality ->  Test Automation Infrastructure Service(Paas/Seleinum Webdriver)

Selenium Server Side Infrastructure Architecture:

Jenkins Job to start build Infra -> Terraform Code to build ECR, ECS -> Server will be running on port 80 to provide automation test services . We are not covering how to build terraform code as you can learn it from Terraform side but we are giving an overview about how things will work once you have everything in place.

1.      Jenkins job

2.      Terraform code to create ECR

3.      Terraform Code to create ECS cluster,service and task and service will start on port 80 and final end point that you will get will be in following format https://ui-testing.example.com

Client Side Test Automation infrastructure:

Jenkins job to start test job -> Docker container will start that will contain npm command (points to URL)->exit status of test cases

Here in client configuration we can use following services to start testing.

1.      Selenium Webdriver: We can take session snapshots manually but no video available

2.      BrowserStack: Enterprise level web portal that provides recorded video of testing for record reference purpose.

3.      NightWatch (Is the NPM module in which we can configure to use selenium, browserverstack or phantomJS)

4.      PhantomJs – Not much mature but it works. There are some compatibility issues with JavaScript.

Below we mentioned one of the available Node.Js code to configure above mentioned client-side libraries so that concerned server platform can be used for testing. Things that we need to make sure before testing.

Test case examples that you need to write in your node.js code.

a. /ui-automation/tests/test1.js

b. /ui-automation/tests/test2.js

root@99dafd878829:/application-ui-automation/tests# vim web-application-login.js

const config = require(“../conf/configFile”);

const loginActions = require(“../pages/login”);

const waitTime = config.GlobalWaits.PageLoadWait;

 module.exports = {

 “@tags”: [“Smoke”, “Login”],

  “Login To WebApplication”: function(browser) {







    let navigationBar = browser.page.navigationBar();


     .waitForElementVisible(“@home”, waitTime)



 } };

Git clone of your repo that contains node.js client code:

Modify important files like:

·        Project.json – Contains repository,scripts,directories,keyword,author,devDependencies and npm install command installs required packages according to it.

·        Config/configFile.json: Environments, Credentials, GlobalWait, BSCredentials details

·        /UI-automation/node_modules/nightwatch/bin/nightwatch.json – Is the most important file in which we need to modify endpoint for our UI test server infrastructure(https://ui-testing.example.com). Example like selenium_host, selenium_port and in this same file we can mention whether we would like to use Selenium, Browserstack or any other such test infra SaaS

·       Npm install

·        npm run localExecution

So Jenkins job that contains terraform code will spin up selenium server and another client side Jenkins job will spin up a docker client side container that will run your test cases corresponding to selenium server. Full Story is another user behavior recording tool so that you can get information about how user is accessing your website and which link they are clicking and when they are dropping specific cart purchase but we are not covering it under this blog.

So once above job will complete you will get artifacts or reports in proper format as Jenkins is having required report plugin that will help you in creating those reports and finally you can refer those reports to take further decision to make your deployment with new code as PASS or FAILED. I know you will face some challenges in setting up this infrastructure but this blog at least gives you direct about how to setup and pre requisite resource that you should have before building such massive infrastructure for your organization.