Acceptance (UI, E2E, GUI) tests are the highest testing layer and describe a customer’s behavior in browsers. Think of them as good old “user stories”. They are all about what a user can do and see. I’m sure that even one such kind of test can multiply improve your code quality. If you don’t have a strong opinion about test needs, I recommend starting with general testing information that I’ve written a bit earlier.
The current test layer gives a working capacity guaranty of your awesome application. The tests deal with your website’s front-end and back-end parts and are intuitively clear during writing or reading. Sounds good? But as usual, all great things in development have a fly in the ointment and, in our case, is a testing environment. I spent a lot of time enrolling testing environment and want to share my experience with you, save your time, and warn you about rabbit holes because there are too many.
Install the Codeception Testing Framework
First of all, you need to install the Codeception testing framework, which allows for dealing with any testing layers and helps to avoid a lot of headaches. For WordPress setup, we need to install the excellent Codeception module for WordPress — WP Browser. Also, the Codeception frameworks have many different modules, and I installed the Asserts, DB, and WebDriver modules. So, update your composer.json file and then run composer install:
// ...
"require-dev": {
"codeception/codeception": "^4.1",
"codeception/module-asserts": "^1.3.1",
"codeception/module-db": "^1.1",
"codeception/module-webdriver": "^1.2",
"lucatume/wp-browser": "^2.6"
}
// ...Setup configuration files for the Codeception
Secondly, you should create the main configuration file for the Codeception framework into the root codeception.yml:
paths:
tests: tests/php
output: codeception/_output
data: codeception/_data
support: codeception/_support
envs: codeception/_envs
actor_suffix: Tester
extensions:
enabled:
- CodeceptionExtensionRunFailed
params:
- codeception/_config/params.php
settings:
backup_globals: false
colors: trueThirdly, let’s create the file with basic constants in the codeception/_config/params.php file:
<?php
return [
'WP_URL' => 'http://your-site.local',
'WP_ADMIN_USERNAME' => 'admin',
'WP_ADMIN_PASSWORD' => 'pass',
'WP_ADMIN_PATH' => '/wp-admin',
'DB_HOST' => 'localhost',
'DB_NAME' => 'acceptance_db',
'DB_USER' => 'root',
'DB_PASSWORD' => 'root',
'DB_TABLE_PREFIX' => 'wp_',
];Fourthly, let’s set up the configuration file for acceptance tests tests/acceptance.suite.yml:
actor: AcceptanceTester
extensions:
enabled:
- CodeceptionExtensionRunProcess:
0: chromedriver --url-base=/wd/hub
- CodeceptionExtensionRunFailed
commands:
- CodeceptionCommandGenerateWPUnit
- CodeceptionCommandGenerateWPRestApi
- CodeceptionCommandGenerateWPRestController
- CodeceptionCommandGenerateWPRestPostTypeController
- CodeceptionCommandGenerateWPAjax
- CodeceptionCommandGenerateWPCanonical
- CodeceptionCommandGenerateWPXMLRPC
modules:
enabled:
- WPDb
- WPWebDriver
config:
WPDb:
dsn: 'mysql:host=%DB_HOST%;dbname=%DB_NAME%'
user: '%DB_USER%'
password: '%DB_PASSWORD%'
dump: 'codeception/_data/dump.sql'
populate: true
cleanup: true
waitlock: 10
url: '%WP_URL%'
urlReplacement: true
WPWebDriver:
url: '%WP_URL%'
port: 9515
window_size: maximize
browser: chrome
host: localhost
adminUsername: '%WP_ADMIN_USERNAME%'
adminPassword: '%WP_ADMIN_PASSWORD%'
adminPath: '%WP_ADMIN_PATH%'
capabilities:
"goog:chromeOptions":
args: ["--user-agent=wp-browser", "--ignore-certificate-errors"]I know it looks scary, but overall these are intuitive settings. If you wish, you can figure out the settings yourself.
The fifth point, to avoid coupling between tests. We should run each test separately from the default state. In our case, the default state is a default state for a database. So, let’s create a separate acceptance_db database, activate the tested WordPress plugin, install the tested plugin/theme, and export the database to the codeception/_data/dump.sql. It can be done manually or to show that you are a cool enough developer using WP CLI:
wp config create --dbname="acceptance_db" --dbusroot" --dbpass="root" --dbhost="localhost" --dbprefix="wp_"
wp core install --url="http://your-site.local" --title="Test" --admin_user="admin" --admin_password="pass" --admin_email="[email protected]" --skip-email
wp rewrite structure '/%postname%/' --hard
wp plugin activate --all
mysqldump --host="localhost" --user="root" --password="root" acceptance_db > codeception/_data/dump.sqlThe sixth item, we need to create a tester class. The class will be automatically generated due to all your installed modules. In other words, the class will have all methods which Codeception and Codeception modules add to a tester. The codeception/_support/AcceptanceTester.php file:
<?php
use CodeceptionActor;
class AcceptanceTester extends Actor {
use _generatedAcceptanceTesterActions;
}Next, let’s update the wp_config.php file to divide the database for the simple run and acceptance tests. I want to remind you that our testing framework will always update the acceptance_db database from the dump.sql we prepared earlier.
if (
isset( $_SERVER['HTTP_X_TESTING'] )
|| ( isset( $_SERVER['HTTP_USER_AGENT'] ) && $_SERVER['HTTP_USER_AGENT'] === 'wp-browser' )
|| getenv( 'WPBROWSER_HOST_REQUEST' )
) {
define( 'DB_NAME', 'codeception_db' );
} else {
define( 'DB_NAME', 'local' );
}The good news, we did the hardest part. The last, you need to install ChromeDriver or update it:
- MacOS:
brew cask install chromedriver. - Windows:
choco install chromedriver
The First Acceptance Test
Finally, let’s get down to the most interesting part – writing the first test. JFYI, all acceptance tests by default should have the Cest suffix. Let’s create the first test-cest – tests/acceptance/FirstCest.php:
<?php
class FirstCest {
public function visitSettingsPage( AcceptanceTester $I ) {
$I->loginAsAdmin();
$I->amOnAdminPage( 'admin.php?page=plugin-name' );
$I->see( 'Plugin Name Settings' );
}
}Probably, you already understand what the test is doing. The user ($I) enters for some page inside the admin area and sees there the page includes the Plugin Name Settings text. Looks really simple and easy to read.
Acceptance Tests Run
vendor/bin/codecept run acceptanceWhen you run the command, you should see it opening a new Chrome window and then quickly doing the test steps. Then, you can see the result inside your console:
As you can see, the most sophisticated thing is setting up the environment. The tests themselves are elementary and straightforward without any special skills.
Acceptance Tests inside GitHub Actions
The tests won’t give you benefits if they have run rarely. We have a bunch of different continuous integration services in our time. Still, I suggest using GH Actions as an example to run tests every time we have a push to a remote repository.
Firstly, we should prepare a server for GH Actions. Let’s use Apache and create a config for it inside the .github/workflows/plugin-name.conf file:
<VirtualHost *:80>
DocumentRoot /home/runner/work/WPPlugin/WPPlugin/wordpress
ServerName plugin-name.test
ErrorLog /home/runner/work/WPPlugin/WPPlugin/logs/error.log
CustomLog /home/runner/work/WPPlugin/WPPlugin/logs/access.log combined
DirectoryIndex index.php index.html /index.php
<Directory /home/runner/work/WPPlugin/WPPlugin/wordpress>
Options FollowSymLinks
DirectoryIndex index.php index.html /index.php
AllowOverride All
Require all granted
</Directory>
</VirtualHost>
<VirtualHost *:443>
DocumentRoot /home/runner/work/WPPlugin/WPPlugin/wordpress
ServerName plugin-name.test
ErrorLog /home/runner/work/WPPlugin/WPPlugin/logs/error.log
CustomLog /home/runner/work/WPPlugin/WPPlugin/logs/access.log combined
DirectoryIndex index.php index.html /index.php
<Directory /home/runner/work/WPPlugin/WPPlugin/wordpress>
Options FollowSymLinks
DirectoryIndex index.php index.html /index.php
AllowOverride All
Require all granted
</Directory>
SSLEngine on
SSLCertificateFile /etc/apache2/ssl/plugin-name.test/plugin-name.test.pem
SSLCertificateKeyFile /etc/apache2/ssl/plugin-name.test/plugin-name.test-key.pem
</VirtualHost>
Do you remember as we created params with a list of environment constants? Such as we have a local environment and GH Action environment, we should modify and divide it. Therefore, we change the code in the codeception/_config/params.php file:
<?php
if ( in_array( 'github-actions', $argv, true ) && file_exists( $config ) ) {
// Config for GH Actions.
return [
'WP_URL' => getenv( 'WP_URL' ),
'WP_ADMIN_USERNAME' => getenv( 'WP_ADMIN_USERNAME' ),
'WP_ADMIN_PASSWORD' => getenv( 'WP_ADMIN_PASSWORD' ),
'WP_ADMIN_PATH' => getenv( 'WP_ADMIN_PATH' ),
'DB_HOST' => getenv( 'DB_HOST' ),
'DB_NAME' => getenv( 'DB_NAME' ),
'DB_USER' => getenv( 'DB_USER' ),
'DB_PASSWORD' => getenv( 'DB_PASSWORD' ),
'DB_TABLE_PREFIX' => getenv( 'DB_TABLE_PREFIX' ),
];
}
return [
'WP_URL' => 'http://your-site.local',
'WP_ADMIN_USERNAME' => 'admin',
'WP_ADMIN_PASSWORD' => 'pass',
'WP_ADMIN_PATH' => '/wp-admin',
'DB_HOST' => 'localhost',
'DB_NAME' => 'acceptance_db',
'DB_USER' => 'root',
'DB_PASSWORD' => 'root',
'DB_TABLE_PREFIX' => 'wp_',
];
And the last here is GH Actions is the .github/workflows/plugin-name.yml configuration file:
name: PluginName GitHub Actions
on: [push]
jobs:
build:
strategy:
matrix:
php-versions: [7.4]
runs-on: ubuntu-latest
env:
php-ext-cache-key: cache-v1 # can be any string, change to clear the extension cache.
php-extensions: mysql
php-ini-values: post_max_size=256M
wp-directory: wordpress
wp-plugins-directory: wordpress/wp-content/plugins/plugin-name/
DB_HOST: 127.0.0.1
DB_TABLE_PREFIX: wp_
DB_NAME: test_db
DB_USER: user
DB_PASSWORD: passw0rd
WP_URL: https://plugin-name.test
WP_ADMIN_PATH: /wp-admin
WP_DOMAIN: plugin-name.test
WP_ADMIN_USERNAME: admin
WP_ADMIN_PASSWORD: admin
WP_ADMIN_EMAIL: [email protected]
services:
mysql:
image: mysql:5.6
env:
MYSQL_USER: ${{ env.DB_USER }}
MYSQL_PASSWORD: ${{ env.DB_PASSWORD }}
MYSQL_DATABASE: ${{ env.DB_NAME }}
MYSQL_ALLOW_EMPTY_PASSWORD: yes
ports:
- 3306:3306
options: --health-cmd="mysqladmin ping" --health-interval=10s --health-timeout=5s --health-retries=3
steps:
- name: Download Plugin
uses: actions/checkout@v2
with:
path: ${{ env.wp-plugins-directory }}
- name: Setup cache environment
id: cache-env
uses: shivammathur/cache-extensions@v1
with:
php-version: ${{ matrix.php-versions }}
extensions: ${{ env.php-extensions }}
key: ${{ env.php-ext-cache-key }}
env:
update: true
- name: Cache extensions
uses: actions/cache@v1
with:
path: ${{ steps.cache-env.outputs.dir }}
key: ${{ steps.cache-env.outputs.key }}
restore-keys: ${{ steps.cache-env.outputs.key }}
- name: Setup PHP
uses: shivammathur/setup-php@v2
with:
tools: pecl
php-version: ${{ matrix.php-versions }}
extensions: ${{ env.php-extensions }}
ini-values: ${{ env.php-ini-values }}
env:
update: true
- name: Install dependencies
working-directory: ${{ env.wp-plugins-directory }}
run: composer install
- name: Install WP CLI
run: |
curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
chmod +x wp-cli.phar
mkdir -p wp-cli
sudo mv wp-cli.phar wp-cli/wp
echo "$GITHUB_WORKSPACE/wp-cli" >> $GITHUB_PATH
echo -n "apache_modules:n - mod_rewrite" > "${{ env.wp-directory }}/wp-cli.yml"
- name: Install WP
working-directory: ${{ env.wp-directory }}
run: |
wp core download --version=5.5
wp config create --dbname="${{ env.DB_NAME }}" --dbuser="${{ env.DB_USER }}" --dbpass="${{ env.DB_PASSWORD }}" --dbhost="${{ env.DB_HOST }}" --dbprefix="${{ env.DB_TABLE_PREFIX }}"
wp core install --url="${{ env.WP_URL }}" --title="Test" --admin_user="${{ env.WP_ADMIN_USERNAME }}" --admin_password="${{ env.WP_ADMIN_PASSWORD }}" --admin_email="${{ env.WP_ADMIN_EMAIL }}" --skip-email
wp rewrite structure '/%postname%/' --hard
wp plugin activate --all
- name: Make a DB dump for Codeception
working-directory: ${{ env.wp-plugins-directory }}
run: mysqldump --column-statistics=0 --host="${{ env.DB_HOST }}" --user="${{ env.DB_USER }}" --password="${{ env.DB_PASSWORD }}" ${{ env.DB_NAME }} > .codeception/_data/dump.sql
- name: Setup hosts
run: |
echo ${{ env.DB_HOST }} ${{ env.WP_DOMAIN }} | sudo tee -a /etc/hosts
cat /etc/hosts
- name: Force SSL
run: |
sudo apt install libnss3-tools -y
wget https://github.com/FiloSottile/mkcert/releases/download/v1.4.3/mkcert-v1.4.3-linux-amd64 -O mkcert
chmod +x mkcert
sudo mv mkcert /usr/local/bin/
mkcert -install
localcaroot=$(mkcert -CAROOT)
sudo mkdir -p /etc/apache2/ssl/${{ env.WP_DOMAIN }} && cd $_
sudo CAROOT=$localcaroot mkcert ${{ env.WP_DOMAIN }}
- name: Install & configure Apache
run: |
sudo add-apt-repository ppa:ondrej/php
sudo apt-get update
sudo apt-get install apache2 libapache2-mod-php7.4
mkdir -p logs
sudo cp ${{ env.wp-plugins-directory }}/.github/workflows/plugin-name.conf /etc/apache2/sites-available/plugin-name.conf
sudo a2enmod headers
sudo a2enmod rewrite
sudo a2enmod ssl
sudo a2ensite plugin-name
sudo apachectl configtest
sudo service apache2 restart
- name: Set .htaccess
working-directory: ${{ env.wp-directory }}
run: |
echo '# BEGIN WordPress
RewriteEngine On
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteBase /
RewriteRule ^index.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
# END WordPress' > .htaccess
- name: Setup Chromedriver
uses: nanasess/setup-chromedriver@master
- name: Run Chromedriver
run: |
export DISPLAY=:99
chromedriver --url-base=/wd/hub &
sudo Xvfb -ac :99 -screen 0 1280x1024x24 > /dev/null 2>&1 & # optional
- name: Run Acceptance Tests
working-directory: ${{ env.wp-plugins-directory }}
run: composer acceptance -- --env github-actions
- name: Archive Codeception output
uses: actions/upload-artifact@v1
if: failure()
with:
name: codeception-output
path: ${{ env.wp-plugins-directory }}/.codeception/_output
- name: Archive Apache Logs
uses: actions/upload-artifact@v1
if: failure()
with:
name: apache-logs
path: logs
So, if all works as well, then acceptance tests will check your changes on every push. If someone breaks the page settings, the pipeline will fail, and the developer can see the problem early.
Conclusion
The hardest part of acceptance testing is the environment, but you need to do it once. Tests look really, super simple, and you can teach your QA engineers to write them. In the beginning, it will be really slow your development process than manual testing, but with growth expertise and your tests base, you will save a lot of human resources and prevent huge bugs amount. When quality is a key target for your product, then you should start with acceptance tests today.
The article looks really massive, so I decided to make the next part with many technical details, focusing on writing tests, working with the Codeception testing framework like a guru, etc. The next part if you want to dive into the topic deeply inside the Acceptance Tests.
