Running Automated Selenium Tests from Jenkins on Linux

Our mission is to build quality, efficient and cutting edge software solutions for our clients while building strong, well-rounded careers for our employees.

19 June 2017 alan.monson@stgconsulting.com Comments Off on Running Automated Selenium Tests from Jenkins on Linux QA

By: Bill Witt
Introduction
As a Quality Assurance Engineer or Software Development Engineer in Test, there is high demand for those who are not only able to create automated tests using Selenium WebDriver, but there is also a demand for engineers who can integrate selenium testing into a continuous integration server like Jenkins.
Using tools like Jenkins provides the ability to create and maintain automated test runs and provide test result reports.  Mastering Jenkins can be a challenge to those who may be unfamiliar with it and its many configuration options and plugins, but the challenges presented by Jenkins can be even more discouraging when trying to set it up for a Linux environment.

Problem Background
Having been familiar with configuring WebDriver automation tests to run on an instance of Jenkins from within a Windows environment, I had hoped that configuring Jenkins might be as straight-forward when configured in a Linux environment, in this case, Ubuntu.  Upon beginning this endeavor, I quickly found that setting up a Linux based Jenkins configuration was quite a bit more difficult, especially in a non-GUI Linux Server environment.  
The following article has been written with the intention of assisting others who may have come across the same difficulties I had encountered, or those who may have been tasked with setting up a Jenkins automated test project who may not be sure where to begin.

Solution
After several days of trial and error attempting to run automated Firefox and Chrome WebDriver tests from Jenkins in a Linux (Ubuntu) environment, I was finally able to implement a solution that works.  This solution will work in Linux environments that do and do not have a GUI, both desktop and server versions.
In the following sections of this article I will outline the various steps required to successfully configure the Linux environment and Continuous Integration Server, in this case Jenkins, to run WebDriver automated tests using either the Firefox or Chrome browsers.

Linux Setup
The first thing to do is to prepare your Linux environment by adding the required Chrome and Firefox drivers.  Although somewhat obvious that chromedriver will need to be setup separately, due to recent changes in how WebDriver works, it will also be necessary to separately configure Firefox’s driver, geckodriver.  Run the following commands in the Linux terminal, outlined in each of the driver configuration sections.
Chromedriver Setup:

    1. sudo apt-get -y install unzip
    1. wget -N http://chromedriver.storage.googleapis.com/2.10/chromedriver_linux64.zip -P ~/Downloads
    1. unzip ~/Downloads/chromedriver_linux64.zip -d ~/Downloads
    1. chmod 777 ~/Downloads/chromedriver
    1. sudo mv -f ~/Downloads/chromedriver /usr/bin/chromedriver
  1. sudo apt-get install libxi6 libgconf-2-4

Geckodriver (Firefox) Setup:

    1. sudo apt-get -y install unzip (skip this step if done previously)
    1. wget -N https://github.com/mozilla/geckodriver/releases/download/v0.16.1/geckodriver-v0.16.1-linux64.tar.gz -P ~/Downloads
    1. tar -xvzf ~/Downloads/geckodriver* -C ~/Downloads
    1. unzip ~/Downloads/geckodriver* -d ~/Downloads
    1. chmod 777 ~/Downloads/geckodriver
  1. sudo mv -f ~/Downloads/geckodriver /usr/bin/geckodriver

Xvfb Setup:

    1. sudo apt-get -y install xvfb
  1. sudo chmod 777 /run/user/

Git Setup:

  1. sudo apt-get -y install git-all


Jenkins Setup
After completing the installation of the Chrome and Gecko (Firefox) drivers, you will need to configure Jenkins to properly run the WebDriver tests in a semi-headless browser type of execution.  The following configuration sections will provide the steps for installing the required Jenkins plugins and Jenkins build environment setup.  This article assumes that Jenkins has already been installed, but if this is not the case, an additional section has been added to the end of this article that provides the steps needed to install Jenkins
Jenkins Plugin Installations:

    1. Open the plugin manager by doing the following, navigate to: Jenkins > Manage Jenkins > Manage Plugins, then click the “Available” tab
  1. Select the following plugins by selecting from the alphabetized list, or by typing the plugin name in the “Filter” textbox, then click the associated checkbox and click the “Install without restart” button (repeat for each plugin)
      1. GitHub Organization Folder
      1. Pipeline
      1. Xvfb
    1. Safe Restart (this is optional, but recommended)

Jenkins Global Tool Configuration:

    1. Open the Jenkins Global Tool Configuration window by navigating to: Jenkins > Manage Jenkins > Global Tool Configuration
    1. Scroll to the “JDK” section and do the following:
        1. Click the “Add JDK” button
        1. For the name textbox, just enter: JDK 8 (this is whatever Java version you happen to be using, this could be 7, 8, 9, etc.)
        1. Click the “Install automatically” checkbox
        1. Select the “Install from java.sun.com” option, then select the appropriate version
      1. Click the “I agree…” checkbox. You may have to enter the Java/Oracle username credentials, which can be setup for free on their website
    1. Scroll to the “Git” section and do the following:
        1. Click the “Add Git” button
        1. Name the Git Installation Default or Git
      1. In the “Path to Git executable” textbox, enter: /usr/bin/git
    1. Scroll to the “Maven” section and do the following:
        1. Click the “Add Maven” button
        1. Just name the installation Maven or Maven <version number>
        1. Click the “Install automatically” checkbox
      1. Select the version installed (usually the latest version available) in the dropdown box under “Install form Apache”
    1. Scroll to the “Xvfb installation” section and do the following:
        1. Click the “Add Xvfb installation” button
        1. Name the installation simply, Xvfb
      1. Click the checkbox by “Install automatically”
  1. Click the “Save” button

Jenkins System Configuration:

    1. Open the Jenkins System Configuration by navigating to: Jenkins > Manage Jenkins > Configure System
    1. Scroll to the “Global properties” section
    1. Click the checkbox by “Environment variables”
    1. Add a new environment variable by clicking the “Add” button
    1. Name the variable: XDG_RUNTIME_DIR
    1. For the value, enter: /run/usr/1001
    1. Scroll to the “Jenkins Location” section
    1. In the “Jenkins URL” textbox, enter: http://127.0.0.1:8081
  1. Click the “Save” button

Create the Job for the Automation Project:

  1. To create a new project for your automated tests, do the following:
      1. From the main Jenkins vertical menu, select: “New Item”
      1. Name the project in the “Enter an item name” textbox
      1. Select the “Freestyle project” option
    1. Click the “OK” button

Configure Jenkins Automated Test Project/Job:

    1. Select the newly created project, then click “Configure” in the left vertical menu
    1. Scroll to the “Source Code Management” section
        1. Click the “Git” radio button
        1. Set the repository URL in the associate textbox (e.g., https://github.com/<username>/<projectname>)
        1. If the GitHub repo is Private, rather than Public, add the necessary credentials
        1. Scroll to the “Branches to build” section
      1. In the “Branch Specifier” textbox enter: */master
    1. Scroll to the “Build Environment” section
        1. Click the checkbox by the “Start Xvfb before the build, and shut it down after.” Option
        1. Click the “Advanced…” button
      1. Select the Xvfb installation previously created from the drop down menu
    1. Scroll to the “Build” section
        1. Click the “Add build step” button
        1. Select the “Invoke top-level Maven targets” option
        1. Select the previously named Maven installation from the dropdown menu
        1. In the “Goals” textbox enter: clean test
        1. Click the “Advanced…” button
      1. In the “POM” textbox enter: $workspace/pom.xml
    1. Scroll to the “Post-build Actions” section
        1. Click the “Add post-build action” button
        1. Select the “Publish TestNG Results” option
      1. In the “TestNG XML report pattern” textbox enter: **/testng-results.xml
  1. Click the “Save” button

Jenkins Linux Installation
In the case that Jenkins has not yet been installed on whichever Linux environment you may be setting up your tests in (preferably in Ubuntu), these are the steps for installing the Jenkins continuous integration server in your environment, run from the terminal window:

    1. wget -q -O – https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add –
    1. sudo sh -c ‘echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list’
    1. sudo apt-get update
    1. sudo apt-get install jenkins
  1. Edit the /etc/default/jenkins config file to replace the line, “HTTP_PORT=8080” with “HTTP_PORT=8081”

Sample WebDriver Setup for Automation Project
Initializing the Java WebDriver instance for Chrome or Firefox in your project can be accomplished by doing the following:
public static String chromeDriverPath = “/usr/bin/chromedriver”;
System.setProperty(“webdriver.chrome.driver”, chromeDriverPath);
WebDriver driver = new ChromeDriver();

public static String geckoDriverPath = “/usr/bin/geckodriver”;
System.setProperty(“webdriver.gecko.driver”, geckoDriverPath);
WebDriver driver = new FirefoxDriver();

Additional Email Configuration Options
Another essential set of features that should be configured in your Jenkins setup includes collecting test result data into a test report, displaying test result data, and enabling the ability for Jenkins to email the test result report.

TestNG Emailable Report Setup
Before test reports can be sent out after a test run, the settings for both the native Jenkins email publisher and the “Email Extension Plugin” will need to be configured.  This configuration will require that the SMTP Server setting be setup from within Jenkins in two separate locations.
Once the Email notification settings have been configured, the next step will be to install and configure the “TestNG Results Plugin” and optionally, the “Test Results Analyzer Plugin.”  The TestNG Results Plugin will compile all the test run and result data into an emailable report that can be automatically sent off once a test fun has completed.  The Test Results Analyzer Plugin is not required, but it adds an additional presentation of the test result data.
Install Plugins

    1. Open the plugin manager by doing the following, navigate to: Jenkins > Manage Jenkins > Manage Plugins, then click the “Available” tab
  1. Select the following plugins by selecting from the alphabetized list, or by typing the plugin name in the “Filter” textbox, then click the associated checkbox and click the “Install without restart” button (repeat for each plugin)
      1. Email Extension Plugin
      1. Test Results Analyzer
    1. TestNG Results

System Email Notification Configuration

    1. Open the Jenkins System Configuration by navigating to: Jenkins > Manage Jenkins > Configure System
    1. Scroll to the “Extended E-mail Notification” section
    1. Click the “Advanced…” button
    1. Fill in the SMTP server details as follows (using the STG Consulting Gmail as an example):
        1. In the “SMTP server” textbox, enter: smtp.gmail.com
        1. In the “Default user E-mail suffix” textbox, enter: @stgconsulting.com
        1. In the Advanced section, check the box by “Use SMTP Authentication”
        1. Enter the STG Gmail username (email address)
        1. Enter the STG Gmail password
        1. Click the “Use SSL” checkbox
        1. In the “SMTP port” textbox, enter: 465
      1. Change the Default Content Type option to “HTML (text/html)”
    1. Scroll to the “E-mail Notification” section
    1. Click the “Advanced…” button
    1. Fill in the SMTP server details with the same information that was added previously in step 4
  1. Click the “Save” button

TestNG Results Configuration

    1. Navigate to the Job/Project page for the test automation
    1. Select the “Configure” option
    1. Click the “Post-build Actions” tab at the top, or scroll down to that section
    1. Configure the TestNG Results Plugin by doing the following:
        1. Click the “Add post-build action” button
        1. Select the “Publish TestNG Results” option
        1. In the “TestNG XML report pattern” textbox enter: **/testng-results.xml
      1. For additional, optional TestNG Results options, click the “Advanced…” button
    1. Configure the Email Notifications Plugin by doing the following:
        1. Click the “Add post-build action” button again
        1. Select the “Editable Email Notification” option
        1. Change the Content Type option to “HTML (text/html)”
        1. Click the “Advanced Settings…” button
        1. In the “Triggers” section, in the “Failures – Any” box, do the following
            1. Click the red ‘X’ on the “Developer’s” box to remove it
            1. Click the “Add” button in the “Send To” portion of the box
            1. Select the “Recipients List” option
            1. Click the “Advanced…” button
            1. In the “Recipient List” textbox, enter the email(s) that the test reports will be sent to
            1. If desired, a reply-to email address may be entered in the “Reply-To List” textbox
            1. Change the Content Type option to “HTML (text/html)”
          1. In the “Attachments” textbox, enter: **/emailable-report.html, **/Run Tests.html, or **/*.html
        1. Click the “Add Trigger” button
        1. Select the “Success” trigger option and do the following
      1. Follow the same sub-steps provided in step 5e for the “Success” trigger
  1. Click the “Save” button

Test Results Analyzer Configuration
Once the Test Results Analyzer Plugin has been installed, most of its functionality will be automatic.  On the automation project’s page, a new menu item will be added under the new “TestNG Results” option, named “Test Results Analyzer” in the left side menu.  Additional test result analysis can be found by clicking this menu option.  For further configuration, do the following:

    1. Open the Jenkins System Configuration by navigating to: Jenkins > Manage Jenkins > Configure System
    1. Scroll down to the “Test Results Analyzer” section
  1. Configure the settings as desired within this section, such as specifying the graph and chart types and thresholds

I hope these steps were helpful to better understand how to successfully configure the Linux environment and a Continuous Integration Server, like Jenkins, to run automated tests using Selenium webdriver on either Firefox or Chrome browsers.

For more information regarding selenium testing, jenkins integration, or automated testing check out our other blog posts.

To learn how you can work with a STG Consultant for your software technology needs, contact us— we’d love to hear from you.

Tags:

© STG Consulting 2019. All rights reserved.

Terms and Conditions - Privacy Policy