TestingBot GitHub Action

GitHub Secrets

To get started, you'll need to configure 2 secrets in your GitHub repository.
These 2 secrets are TB_KEY (your TestingBot Key) and TB_SECRET (TestingBot secret) which can be obtained from the TestingBot member area.

In your GitHub repository, go to Settings and find the Secrets option in the left-hand pane.

Add both the TB_KEY and TB_SECRET as Repository Secrets:

Set up a GitHub workflow

Next, we'll create a new GitHub workflow. Create a workflow file test-workflow.yml in the .github/workflows directory of your GitHub repository.
You can find out more about workflow files on the GitHub documentation pages.

In your workflow file, you can indicate when to run the GitHub Action.
We recommend doing this on both push and pull_request for maximum test coverage:

name: "TestingBot Test"
on: [push, pull_request]

jobs:
    test:
        runs-on: ubuntu-latest
        name: Sample Test
...

You can specify a runs-on parameter, to indicate the OS of the container that should run your test.

The most important part is the steps section, which we'll cover in the example below.

Example workflow with TestingBot

Let's set up a workflow that will do the following actions:

• Checkout the current GitHub repository code
• Install dependencies for the project
• Run the tests in the repository

name: "PR Checks"
on: [push, pull_request]

jobs:
    test:
        runs-on: ubuntu-latest
        name: Action Test
        steps:
            - uses: actions/checkout@v4
            - uses: testingbot/testingbot-tunnel-action@v1
              with:
                key: ${{ secrets.TB_KEY }}
                secret: ${{ secrets.TB_SECRET }}
                tunnelIdentifier: github-action-tunnel

            - name: "Install Dependencies"
              run: npm install

            - name: "Run Test"
              run: npm run test
              env:
                TB_KEY: ${{ secrets.TB_KEY }}
                TB_SECRET: ${{ secrets.TB_SECRET }}
                TB_BUILD: github-${{ github.run_id }}-${{ github.run_attempt }}

Here we're including the testingbot/testingbot-tunnel-action TestingBot GitHub action with a key, secret and tunnelIdentifier.

The tunnelIdentifier is used to indicate in your tests that a connection should be made to this specific tunnel.

We also expose a TB_BUILD environment variable, built from the GitHub run context (github.run_id and github.run_attempt). By passing this as the build capability in your tests (see below), every browser session started in this workflow run is grouped into a single TestingBot build. This lets you report one combined pass/fail status back to the commit or Pull Request.

Example Test

For example, let's say your npm run test starts a local http-server and then runs a Selenium WebDriver test:

http-server ./test -p 8080 > /dev/null 2>&1 &

#!/usr/bin/env ruby

require 'rubygems'
require 'selenium-webdriver'

options = Selenium::WebDriver::Chrome::Options.new
options.add_option('platformName', 'WIN11')
options.add_option('browserVersion', 'latest')
options.add_option('tb:options', {
  'key' => ENV['TB_KEY'],
  'secret' => ENV['TB_SECRET'],
  'name' => 'GitHub Action Test',
  'build' => ENV['TB_BUILD'],
  'tunnel-identifier' => 'github-action-tunnel'
})

driver = Selenium::WebDriver.for(
  :remote,
  url: "https://hub.testingbot.com/wd/hub",
  options: options)
driver.navigate.to "http://localhost:8080"
puts driver.title
driver.quit

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.URL;
import java.util.HashMap;
import java.util.Map;

public class AccessibilityTest {

  public static final String KEY = System.getenv("TB_KEY");
  public static final String SECRET = System.getenv("TB_SECRET");
  public static final String URL = "https://hub.testingbot.com/wd/hub";

  public static void main(String[] args) throws Exception {
    ChromeOptions options = new ChromeOptions();
    options.setCapability("browserVersion", "latest");
    options.setCapability("platformName", "WIN11");

    Map<String, Object> tbOptions = new HashMap<>();
    tbOptions.put("key", KEY);
    tbOptions.put("secret", SECRET);
    tbOptions.put("name", "GitHub Action Test");
    tbOptions.put("build", System.getenv("TB_BUILD"));
    tbOptions.put("tunnel-identifier", "github-action-tunnel");
    options.setCapability("tb:options", tbOptions);

    WebDriver driver = new RemoteWebDriver(new URL(URL), options);
    driver.get("http://localhost:8080");

    System.out.println(driver.getTitle());

    driver.quit();
  }
}

<?php

require_once('vendor/autoload.php');
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\Remote\DesiredCapabilities;

$tbKey = getenv('TB_KEY');
$tbSecret = getenv('TB_SECRET');

$capabilities = DesiredCapabilities::chrome();
$capabilities->setCapability('platformName', 'WIN11');
$capabilities->setCapability('browserVersion', 'latest');
$capabilities->setCapability('tb:options', [
  'key' => $tbKey,
  'secret' => $tbSecret,
  'name' => 'GitHub Action Test',
  'build' => getenv('TB_BUILD'),
  'tunnel-identifier' => 'github-action-tunnel'
]);

$web_driver = RemoteWebDriver::create(
  "https://hub.testingbot.com/wd/hub",
  $capabilities,
  120000
);
$web_driver->get("http://localhost:8080");
echo $web_driver->getTitle();
$web_driver->quit();
?>

import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.set_capability('platformName', 'WIN11')
options.set_capability('browserVersion', 'latest')
options.set_capability('tb:options', {
  'key': os.environ['TB_KEY'],
  'secret': os.environ['TB_SECRET'],
  'name': 'GitHub Action Test',
  'build': os.environ.get('TB_BUILD'),
  'tunnel-identifier': 'github-action-tunnel'
})

driver = webdriver.Remote(
    command_executor='https://hub.testingbot.com/wd/hub',
    options=options
)
driver.get("http://localhost:8080")
print(driver.title)
driver.quit()

const webdriver = require('selenium-webdriver');
const chrome = require('selenium-webdriver/chrome');

const testingbotKey = process.env.TB_KEY;
const testingbotSecret = process.env.TB_SECRET;

async function runGitHubActionTest () {
  let options = new chrome.Options();
  options.set('platformName', 'WIN11');
  options.set('browserVersion', 'latest');
  options.set('tb:options', {
    'key': testingbotKey,
    'secret': testingbotSecret,
    'name': 'GitHub Action Test',
    'build': process.env.TB_BUILD,
    'tunnel-identifier': 'github-action-tunnel'
  });

  let driver = new webdriver.Builder()
    .usingServer('https://hub.testingbot.com/wd/hub')
    .withCapabilities(options)
    .build();
  await driver.get("http://localhost:8080");
  const title = await driver.getTitle();
  console.log(title);
  await driver.quit();
}
runGitHubActionTest();

using System;
using System.Collections.Generic;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Remote;

namespace SeleniumTest {
  class Program {
    static void Main(string[] args) {
      var options = new ChromeOptions();
      options.PlatformName = "WIN11";
      options.BrowserVersion = "latest";
      options.AddAdditionalOption("tb:options", new Dictionary<string, object>
      {
        { "key", Environment.GetEnvironmentVariable("TB_KEY") },
        { "secret", Environment.GetEnvironmentVariable("TB_SECRET") },
        { "name", "GitHub Action Test" },
        { "build", Environment.GetEnvironmentVariable("TB_BUILD") },
        { "tunnel-identifier", "github-action-tunnel" }
      });

      IWebDriver driver = new RemoteWebDriver(
        new Uri("https://hub.testingbot.com/wd/hub"), options
      );
      driver.Navigate().GoToUrl("http://localhost:8080");
      Console.WriteLine(driver.Title);
      driver.Quit();
    }
  }
}

Your test will connect to the local http-server via the TestingBot GitHub Action and print the title of the page.

PR Status Checks with the GitHub App

The TestingBot GitHub App posts a pass or fail TestingBot / tests status check on each pull request, with a browser and device result matrix and an optional merge gate. It works with the workflow above: register the run's build against the pull request and TestingBot posts the check.

Build Status on a Pull Request

Because every session in the workflow run shares the same build capability (the TB_BUILD value from the example above), TestingBot groups them into a single build. When your test runner reports the pass/fail state of each test back to TestingBot, we automatically calculate the overall status of that build:

• If every test in the build passed, the build is marked as Passing.
• If any test failed, the build is marked as Failed.
• If we don't receive a state for every test, the build status is Unknown.

See Marking a build as passed/failed for how to send the success state of each test using the TestingBot REST API.

Show a status badge

You can embed a TestingBot status badge in your repository's README to show the result of your most recent build:

[![TestingBot Test Status](https://testingbot.com/buildstatus/YOUR_TESTINGBOT_KEY?auth={authentication})](https://testingbot.com/builds/YOUR_TESTINGBOT_KEY?auth={authentication})

Query a build's status from the API

You can also look up the aggregate status of a specific build (for example, to add your own custom check step to a workflow). The REST API returns whether the build passed, together with the number of total, passed and failed tests:

curl -u "$TB_KEY:$TB_SECRET" \
  https://api.testingbot.com/v1/builds/github-${GITHUB_RUN_ID}-${GITHUB_RUN_ATTEMPT}

A failing build (one or more failed tests) lets you fail the workflow step, which surfaces as a failed check on the commit and Pull Request.

Require Tests to Pass before Merging

A popular workflow is to block a Pull Request from being merged until all tests pass. Make the check mandatory using branch protection rules:

1. In your GitHub repository, go to Settings → Branches (or Rules → Rulesets).
2. Add a branch protection rule for your default branch (for example main).
3. Enable Require status checks to pass before merging.
4. Search for and select TestingBot / tests (the GitHub App check) as a required check. If you are not using the app, select your TestingBot workflow job instead (for example Action Test).

Once enabled, the Merge button stays disabled until the TestingBot build passes, so no Pull Request is merged with failing tests. An incomplete or never-finished build reports a failing check rather than a passing one, so a broken run cannot slip through.

Inputs

The TestingBot GitHub Action accepts the following inputs:

Artifacts

The TestingBot GitHub Action will, by default, upload the logfile generated by the TestingBot tunnel.
This allows you to view what happened with the TestingBot Tunnel for each of your Actions.
The logfile is saved as an artifact and can be downloaded.

If you don't want this, you can disable it by setting the input uploadLogFile: "false" in your workflow file.

More Information

More information about this is available on the GitHub Documentation pages and our TestingBot GitHub Action repository.