Simple start with Acceptance Testing for WordPress

8 min read
717

Acceptance (UI, E2E, GUI) tests are the highest testing layer and describe a customer’s behavior in browsers. Think of them as good old “user stories”. They are all about what a user can do and see. I’m sure that even one such kind of test can multiply improve your code quality. If you don’t have a strong opinion about test needs, I recommend starting with general testing information that I’ve written a bit earlier.

The current test layer gives a working capacity guaranty of your awesome application. The tests deal with your website’s front-end and back-end parts and are intuitively clear during writing or reading. Sounds good? But as usual, all great things in development have a fly in the ointment and, in our case, is a testing environment. I spent a lot of time enrolling testing environment and want to share my experience with you, save your time, and warn you about rabbit holes because there are too many.

Install the Codeception Testing Framework

First of all, you need to install the Codeception testing framework, which allows dealing with any testing layers and helps to avoid a lot of headaches. For WordPress setup, we need to install the excellent Codeception module for WordPress — WP Browser. Also, the Codeception frameworks have many different modules, and I installed the Asserts, DB, and WebDriver modules. So, update your composer.json file and then run composer install:

// ...
"require-dev": {
	"codeception/codeception": "^4.1",    
	"codeception/module-asserts": "^1.3.1",
	"codeception/module-db": "^1.1",
	"codeception/module-webdriver": "^1.2",
	"lucatume/wp-browser": "^2.6"
}
// ...

Setup configuration files for the Codeception

Secondly, you should create the main configuration file for the Codeception framework into the root codeception.yml:

paths:
	tests: tests/php
	output: codeception/_output
	data: codeception/_data
	support: codeception/_support
	envs: codeception/_envs
actor_suffix: Tester
extensions:
	enabled:
		- Codeception\Extension\RunFailed
params:
	- codeception/_config/params.php
settings:
	backup_globals: false
	colors: true

Thirdly, let’s create the file with basic constants in the codeception/_config/params.php file:

<?php

return [
	'WP_URL'            => 'http://your-site.local',
	'WP_ADMIN_USERNAME' => 'admin',
	'WP_ADMIN_PASSWORD' => 'pass',
	'WP_ADMIN_PATH'     => '/wp-admin',
	'DB_HOST'           => 'localhost',
	'DB_NAME'           => 'acceptance_db',
	'DB_USER'           => 'root',
	'DB_PASSWORD'       => 'root',
	'DB_TABLE_PREFIX'   => 'wp_',
];

Fourthly, let’s set up the configuration file for acceptance tests tests/acceptance.suite.yml:

actor: AcceptanceTester
extensions:
	enabled:
		-   Codeception\Extension\RunProcess:
			0: chromedriver --url-base=/wd/hub
		- Codeception\Extension\RunFailed
	commands:
		- Codeception\Command\GenerateWPUnit
		- Codeception\Command\GenerateWPRestApi
		- Codeception\Command\GenerateWPRestController
		- Codeception\Command\GenerateWPRestPostTypeController
		- Codeception\Command\GenerateWPAjax
		- Codeception\Command\GenerateWPCanonical
		- Codeception\Command\GenerateWPXMLRPC
modules:
	enabled:
		- WPDb
		- WPWebDriver
	config:
		WPDb:
			dsn: 'mysql:host=%DB_HOST%;dbname=%DB_NAME%'
			user: '%DB_USER%'
			password: '%DB_PASSWORD%'
			dump: 'codeception/_data/dump.sql'
			populate: true
			cleanup: true
			waitlock: 10
			url: '%WP_URL%'
			urlReplacement: true
		WPWebDriver:
			url: '%WP_URL%'
			port: 9515
	window_size: maximize
            browser: chrome
            host: localhost
            adminUsername: '%WP_ADMIN_USERNAME%'
            adminPassword: '%WP_ADMIN_PASSWORD%'
            adminPath: '%WP_ADMIN_PATH%'
            capabilities:
                "goog:chromeOptions":
                    args: ["--user-agent=wp-browser",  "--ignore-certificate-errors"]

I know it looks scary, but overall these are intuitive settings. If you wish, you can figure out the settings yourself.

The fifth point, to avoid coupling between tests. We should run each test separately from the default state. In our case, the default state is a default state for a database. So, let’s create a separate acceptance_db database, activate the tested WordPress plugin, install the tested plugin/theme, and export the database to the codeception/_data/dump.sql. It can be done manually or to show that you are a cool enough developer using WP CLI:

wp config create --dbname="acceptance_db" --dbusroot" --dbpass="root" --dbhost="localhost" --dbprefix="wp_"
wp core install --url="http://your-site.local" --title="Test" --admin_user="admin" --admin_password="pass" --admin_email="i@wp-punk.com" --skip-email
wp rewrite structure '/%postname%/' --hard
wp plugin activate --all

mysqldump --host="localhost" --user="root" --password="root" acceptance_db > codeception/_data/dump.sql

The sixth item, we need to create a tester class. The class will be automatically generated due to all your installed modules. In other words, the class will have all methods which Codeception and Codeception modules add to a tester. The codeception/_support/AcceptanceTester.php file:

<?php

use Codeception\Actor;

class AcceptanceTester extends Actor {
	use _generated\AcceptanceTesterActions;
}

Next, let’s update the wp_config.php file to divide the database for the simple run and acceptance tests. I want to remind you that our testing framework will always update the acceptance_db database from the dump.sql we prepared earlier.

if ( 
    isset( $_SERVER['HTTP_X_TESTING'] )
    || ( isset( $_SERVER['HTTP_USER_AGENT'] ) && $_SERVER['HTTP_USER_AGENT'] === 'wp-browser' )
    || getenv( 'WPBROWSER_HOST_REQUEST' )
) {
    define( 'DB_NAME', 'codeception_db' );
} else {
    define( 'DB_NAME', 'local' );
}

The good news, we did the hardest part. The last, you need to install ChromeDriver or update it:

  • MacOS: brew cask install chromedriver.
  • Windows: choco install chromedriver

The First Acceptance Test

Finally, let’s get down to the most interesting part – writing the first test. JFYI, all acceptance tests by default should have the Cest suffix. Let’s create the first test-cest – tests/acceptance/FirstCest.php:

<?php

class FirstCest {

	public function visitSettingsPage( AcceptanceTester $I ) {
		$I->loginAsAdmin();
		$I->amOnAdminPage( 'admin.php?page=plugin-name' );
		$I->see( 'Plugin Name Settings' );
	}
}

Probably, you already understand what the test is doing. The user ($I) enters for some page inside the admin area and sees there the page includes the Plugin Name Settings text. Looks really simple and easy to read.

Acceptance Tests Run

vendor/bin/codecept run acceptance

When you run the command, you should see it opening a new Chrome window and then quickly doing the test steps. Then, you can see the result inside your console:

How to use UI, E2E, GUI tests to verify your changes inside your WordPress plugin or theme? How to setup the Codeception and GH Actions for acceptance tests? - WP Punk

As you can see, the most sophisticated thing is setting up the environment. The tests themselves are elementary and straightforward without any special skills.

Acceptance Tests inside GitHub Actions

The tests won’t give you benefits if they have run rarely. We have a bunch of different continuous integration services in our time. Still, I suggest using GH Actions as an example to run tests every time we have a push to a remote repository.

Firstly, we should prepare a server for GH Actions. Let’s use Apache and create a config for it inside the .github/workflows/plugin-name.conf file:

<VirtualHost *:80>
  DocumentRoot /home/runner/work/WPPlugin/WPPlugin/wordpress
  ServerName plugin-name.test

  ErrorLog /home/runner/work/WPPlugin/WPPlugin/logs/error.log
  CustomLog /home/runner/work/WPPlugin/WPPlugin/logs/access.log combined

  DirectoryIndex index.php index.html /index.php

  <Directory /home/runner/work/WPPlugin/WPPlugin/wordpress>
    Options FollowSymLinks
  	DirectoryIndex index.php index.html /index.php
	AllowOverride All
	Require all granted
  </Directory>
</VirtualHost>

<VirtualHost *:443>
  DocumentRoot /home/runner/work/WPPlugin/WPPlugin/wordpress
  ServerName plugin-name.test

  ErrorLog /home/runner/work/WPPlugin/WPPlugin/logs/error.log
  CustomLog /home/runner/work/WPPlugin/WPPlugin/logs/access.log combined

  DirectoryIndex index.php index.html /index.php

  <Directory /home/runner/work/WPPlugin/WPPlugin/wordpress>
    Options FollowSymLinks
    DirectoryIndex index.php index.html /index.php
    AllowOverride All
    Require all granted
  </Directory>

  SSLEngine on
  SSLCertificateFile /etc/apache2/ssl/plugin-name.test/plugin-name.test.pem
  SSLCertificateKeyFile /etc/apache2/ssl/plugin-name.test/plugin-name.test-key.pem
</VirtualHost>

Do you remember as we created params with a list of environment constants? Such as we have a local environment and GH Action environment, we should modify and divide it. Therefore, we change the code in the codeception/_config/params.php file:

<?php

if ( in_array( 'github-actions', $argv, true ) && file_exists( $config ) ) {
	
	// Config for GH Actions.
	return [
		'WP_URL'            => getenv( 'WP_URL' ),
		'WP_ADMIN_USERNAME' => getenv( 'WP_ADMIN_USERNAME' ),
		'WP_ADMIN_PASSWORD' => getenv( 'WP_ADMIN_PASSWORD' ),
		'WP_ADMIN_PATH'     => getenv( 'WP_ADMIN_PATH' ),
		'DB_HOST'           => getenv( 'DB_HOST' ),
		'DB_NAME'           => getenv( 'DB_NAME' ),
		'DB_USER'           => getenv( 'DB_USER' ),
		'DB_PASSWORD'       => getenv( 'DB_PASSWORD' ),
		'DB_TABLE_PREFIX'   => getenv( 'DB_TABLE_PREFIX' ),
	];
}

return [
	'WP_URL'            => 'http://your-site.local',
	'WP_ADMIN_USERNAME' => 'admin',
	'WP_ADMIN_PASSWORD' => 'pass',
	'WP_ADMIN_PATH'     => '/wp-admin',
	'DB_HOST'           => 'localhost',
	'DB_NAME'           => 'acceptance_db',
	'DB_USER'           => 'root',
	'DB_PASSWORD'       => 'root',
	'DB_TABLE_PREFIX'   => 'wp_',
];

And the last here is GH Actions is the .github/workflows/plugin-name.yml configuration file:

name: PluginName GitHub Actions

on: [push]

jobs:
    build:

        strategy:
            matrix:
                php-versions: [7.4]

        runs-on: ubuntu-latest

        env:
            php-ext-cache-key: cache-v1 # can be any string, change to clear the extension cache.
            php-extensions: mysql
            php-ini-values: post_max_size=256M
            wp-directory: wordpress
            wp-plugins-directory: wordpress/wp-content/plugins/plugin-name/
            DB_HOST: 127.0.0.1
            DB_TABLE_PREFIX: wp_
            DB_NAME: test_db
            DB_USER: user
            DB_PASSWORD: passw0rd
            WP_URL: https://plugin-name.test
            WP_ADMIN_PATH: /wp-admin
            WP_DOMAIN: plugin-name.test
            WP_ADMIN_USERNAME: admin
            WP_ADMIN_PASSWORD: admin
            WP_ADMIN_EMAIL: admin@plugin-name.test

        services:
            mysql:
                image: mysql:5.6
                env:
                    MYSQL_USER: ${{ env.DB_USER }}
                    MYSQL_PASSWORD: ${{ env.DB_PASSWORD }}
                    MYSQL_DATABASE: ${{ env.DB_NAME }}
                    MYSQL_ALLOW_EMPTY_PASSWORD: yes
                ports:
                    - 3306:3306
                options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3

        steps:
            -   name: Download Plugin
                uses: actions/checkout@v2
                with:
                    path: ${{ env.wp-plugins-directory }}

            -   name: Setup cache environment
                id: cache-env
                uses: shivammathur/cache-extensions@v1
                with:
                    php-version: ${{ matrix.php-versions }}
                    extensions: ${{ env.php-extensions }}
                    key: ${{ env.php-ext-cache-key }}
                env:
                    update: true

            -   name: Cache extensions
                uses: actions/cache@v1
                with:
                    path: ${{ steps.cache-env.outputs.dir }}
                    key: ${{ steps.cache-env.outputs.key }}
                    restore-keys: ${{ steps.cache-env.outputs.key }}

            -   name: Setup PHP
                uses: shivammathur/setup-php@v2
                with:
                    tools: pecl
                    php-version: ${{ matrix.php-versions }}
                    extensions: ${{ env.php-extensions }}
                    ini-values: ${{ env.php-ini-values }}
                env:
                    update: true

            -   name: Install dependencies
                working-directory: ${{ env.wp-plugins-directory }}
                run: composer install

            -   name: Install WP CLI
                run: |
                    curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
                    chmod +x wp-cli.phar
                    mkdir -p wp-cli
                    sudo mv wp-cli.phar wp-cli/wp
                    echo "$GITHUB_WORKSPACE/wp-cli" >> $GITHUB_PATH
                    echo -n "apache_modules:\n  - mod_rewrite" > "${{ env.wp-directory }}/wp-cli.yml"

            -   name: Install WP
                working-directory: ${{ env.wp-directory }}
                run: |
                    wp core download --version=5.5
                    wp config create --dbname="${{ env.DB_NAME }}" --dbuser="${{ env.DB_USER }}" --dbpass="${{ env.DB_PASSWORD }}" --dbhost="${{ env.DB_HOST }}" --dbprefix="${{ env.DB_TABLE_PREFIX }}"
                    wp core install --url="${{ env.WP_URL }}" --title="Test" --admin_user="${{ env.WP_ADMIN_USERNAME }}" --admin_password="${{ env.WP_ADMIN_PASSWORD }}" --admin_email="${{ env.WP_ADMIN_EMAIL }}" --skip-email
                    wp rewrite structure '/%postname%/' --hard
                    wp plugin activate --all

            -   name: Make a DB dump for Codeception
                working-directory: ${{ env.wp-plugins-directory }}
                run: mysqldump --column-statistics=0 --host="${{ env.DB_HOST }}" --user="${{ env.DB_USER }}" --password="${{ env.DB_PASSWORD }}" ${{ env.DB_NAME }} > .codeception/_data/dump.sql

            -   name: Setup hosts
                run: |
                    echo ${{ env.DB_HOST }} ${{ env.WP_DOMAIN }} | sudo tee -a /etc/hosts
                    cat /etc/hosts

            -   name: Force SSL
                run: |
                    sudo apt install libnss3-tools -y
                    wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.3/mkcert-v1.4.3-linux-amd64 -O mkcert
                    chmod +x mkcert
                    sudo mv mkcert /usr/local/bin/
                    mkcert -install
                    localcaroot=$(mkcert -CAROOT)
                    sudo mkdir -p /etc/apache2/ssl/${{ env.WP_DOMAIN }} && cd $_
                    sudo CAROOT=$localcaroot mkcert ${{ env.WP_DOMAIN }}

            -   name: Install & configure Apache
                run: |
                    sudo add-apt-repository ppa:ondrej/php
                    sudo apt-get update
                    sudo apt-get install apache2 libapache2-mod-php7.4
                    mkdir -p logs
                    sudo cp ${{ env.wp-plugins-directory }}/.github/workflows/plugin-name.conf /etc/apache2/sites-available/plugin-name.conf
                    sudo a2enmod headers
                    sudo a2enmod rewrite
                    sudo a2enmod ssl
                    sudo a2ensite plugin-name
                    sudo apachectl configtest
                    sudo service apache2 restart

            -   name: Set .htaccess
                working-directory: ${{ env.wp-directory }}
                run: |
                    echo '# BEGIN WordPress
                    RewriteEngine On
                    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
                    RewriteBase /
                    RewriteRule ^index\.php$ - [L]
                    RewriteCond %{REQUEST_FILENAME} !-f
                    RewriteCond %{REQUEST_FILENAME} !-d
                    RewriteRule . /index.php [L]
                    # END WordPress' > .htaccess

            -   name: Setup Chromedriver
                uses: nanasess/setup-chromedriver@master

            -   name: Run Chromedriver
                run: |
                    export DISPLAY=:99
                    chromedriver --url-base=/wd/hub &
                    sudo Xvfb -ac :99 -screen 0 1280x1024x24 > /dev/null 2>&1 & # optional

            -   name: Run Acceptance Tests
                working-directory: ${{ env.wp-plugins-directory }}
                run: composer acceptance -- --env github-actions

            -   name: Archive Codeception output
                uses: actions/upload-artifact@v1
                if: failure()
                with:
                    name: codeception-output
                    path: ${{ env.wp-plugins-directory }}/.codeception/_output

            -   name: Archive Apache Logs
                uses: actions/upload-artifact@v1
                if: failure()
                with:
                    name: apache-logs
                    path: logs

So, if all works as well, then acceptance tests will check your changes on every push. If someone breaks the page settings, the pipeline will fail, and the developer can see the problem early.

Conclusion

The hardest part of acceptance testing is their environment, but you need to do it once. Tests look really, super simple, and you can teach your QA engineers to write them. In the beginning, it will be really slow your development process than manual testing, but with growth expertise and your tests base, you will save a lot of human resources and prevent huge bugs amount. When quality is a key target for your product, then you should start with acceptance tests today.

The article looks really massive, so I decided to make the next part with many technical details, focusing on writing tests, working with the Codeception testing framework like a guru, etc. The next part if you want to dive into the topic deeply inside the Acceptance Tests.

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to news. I promise not to spam :)
Follow me, don't be shy