Advanced usage of Acceptance Tests for WordPress

The day is here! I’m writing about tests again. Those who missed the first part about acceptance tests feel free to start with it. There was described environment setup and wrote your first acceptance test. The current article is for teams that already tried and wrote a couple of dozen tests and focused on long-term investing in the test base. I’m going to describe the common things that allow you to speed up and simplify your test writing.

Which is better XPath or CSS?

During writing for acceptance tests, we will often deal with DOM elements. There are only two methods available to us to work with DOM elements: CSS selectors or XPath. In a nutshell, CSS selectors are better to use when dealing with tag names, IDs, and classes. They are shorter and easier to read. On the other hand, the XPath has many functions and allows you to create super-duper complex queries. Let’s compare CSS3 and XPath:

Actors for Codeception Tests

Actors are the main concepts for the Codeception framework to separate different tests layers such as unit, functional, and acceptance. Please pay attention that it has nothing to do with use cases or use diagrams. One suite – one tester, it’s all.

So, let’s check or create your acceptance tester inside the codeception/_support/AcceptanceTester file:

<?php

use CodeceptionActor;

class AcceptanceTester extends Actor {
	use _generatedAcceptanceTesterActions;
}

The most crucial part is the _generatedAcceptanceTesterActions trait, which is used magic for enabling modules. If you are adding or removing any module, then you should regenerate the trait:

php vendor/bin/codecept build

What can you add to the AcceptanceTester? Inside the tester class you can add simple actions, e.g., see, check, scroll, click, assert, etc. Let’s add a method that allows scrolling to a shortcode:

<?php

use CodeceptionActor;

class AcceptanceTester extends Actor {
	use _generatedAcceptanceTesterActions;

	/**
	 * Scroll to the plugin shortcode.
	 *
	 * @param string $url Expected URL.
	 */
	public function scrollToShortcode( $url ) {

		$this->scrollTo( '//div[contains(@class, "shortcode-wrapper")]', 0, -100 );
	}
}

Modules & Helpers for Codeception Tests

The Codeception framework has a lot of awesome modules, starting with helpful Asserts and ending the WPCLI module. You can find a lot of ready for usage modules on the official Codeception website and GitHub.
To enable the module you just need to add a module to the project and enable it inside your suite YML.

Modules for Codeception Tests

Imagine that your plugin has a WP-CLI command that you’re going to test. To deal with it, you should just install the WPCLI module. Such as the WPCLI is a part of the WP Browser package; we don’t need to add any new dependencies to the project, only enable it inside the tests/acceptance.suite.yml file:

# ...
modules:
    enabled:
        - WPCLI
# ...

And as you remember, we should regenerate our tester:

php vendor/bin/codecept build

Wow, now our tester has new methods, e.g.: cli, dontSeeInShellOutput, etc.

Helpers for Codeception Tests

The next point here is a Helpers. I don’t know the real reason for the naming, but it’s absolutely the same as modules. Maybe only one difference is the Helpers located inside your codeception/_support directory. Let’s add a new module inside the codeception/_support/Module.php file:

<?php

namespace YourPlugin;

class Module extends CodeceptionModule {
}

Finally, we should enable our new module inside the tests/acceptance.suite.yml file:

# ...
modules:
    enabled:
        - YourPluginModule
# ...

The Modules and Testers are approximately the same. When might you need your custom module? It would help if you had your custom module to extend or change behavior for another module. The WPWebDriver module doesn’t have the method which gets the URL for the current page, so let’s add it:

<?php

namespace YourPlugin;

class Module extends CodeceptionModule {
	public function getCurrentUrl() {
		return $this->getModule( 'WPWebDriver' )->webDriver->getCurrentURL();
	}
}

Last, but not least, we should rebuild again our tester:

php vendor/bin/codecept build

Page Objects for Acceptance Tests

We’ve reached my favorite part. Page objects are a rocket feature for reusing your code and creating quick tests without diving into details. What the heck is page objects?

The page object represents a web page as a class and the DOM elements on that page as its properties and some basic interactions as its methods.

Codeception documentation

Try to divide your project into logical pages, such as page settings, a post page with a shortcode, mailbox, a CPT list, a single CPT page, etc… Then, create constants with selectors for key elements on these pages and add some most used methods for the page. For example, you can add methods on the CPT list page: createNewCPTItem, viewCPTItem, deleteCPTItem, visitCPTPage, etc. Let’s add the BooksOverview page object inside the codeception/_support/PageObjects/BooksOverview.php file:

<?php

namespace PageObjects;

use AcceptanceTester;

class BooksOverview extends PageObject {

	private $i;
	private $isLogged = false;

	const PAGE_URL = 'admin.php?page=wpforms-overview';
	const PAGE_TITLE = 'Books Overview';
	const ADD_BUTTON_SELECTOR = '//*[contains(@class, "add-new-h2")]';
	const BOOK_TITLE_SELECTOR_PATTERN = '//input[@name="form_id[]"][@value=%d]/../../td/a/strong';
	const BOOK_PREVIEW_SELECTOR_PATTERN = '//input[@name="form_id[]"][@value=%d]/../..//div[@class="row-actions"]/span[@class="preview"]/a["Preview"]';

	public function __construct( AcceptanceTester $i ) {
		$this->i = $i;
	}

	public function visitBooksOverviewPage() {
		if ( ! $this->isLogged ) {
			$this->i->loginAsAdmin();
			$this->isLogged = true;
		}

		$this->i->amOnAdminPage( self::PAGE_URL );
	}

	public function addNewBook() {
		$this->i->click( self::ADD_BUTTON_SELECTOR );
	}

	public function editBook( $bookId ) {
		$bookTitleSelector       = sprintf( self::BOOK_TITLE_SELECTOR_PATTERN, $bookId );
		$bookPreviewLinkSelector = sprintf( self::BOOK_PREVIEW_SELECTOR_PATTERN, $bookId );

		$this->i->moveMouseOver( $bookTitleSelector );
		$this->i->waitForElementClickable( $bookPreviewLinkSelector );
		$this->i->click( $bookPreviewLinkSelector );
		$this->i->switchToNextTab();
	}
}

The $i property is the AcceptanceTester. Constants PAGE_URL, PAGE_TITLE, ADD_BUTTON_SELECTOR, BOOK_TITLE_SELECTOR_PATTERN, BOOK_PREVIEW_SELECTOR_PATTERN are XPath selectors or patterns for selectors. The visitBooksOverviewPage, addNewBook, editBook are common actions on the page. To summarize, the page objects are the main way to reuse a code.

The heart of the Acceptance tests – Cests

Cests are acceptance test classes, and here you describe your test cases and all this quality assurance stuff. A cest should have the suffix Cest at the end of the class name. Every public method (except those starting with _) inside a cest will be run as a test. Cests should be located in the directory with the same name as their suite name. If your suite is tests/acceptance.suite.yml, you should create the tests/acceptance/ directory nearby your suite and put all your Cests there.

Before we dive into the testing, let’s quickly summarize the main components: ​actors, ​modules & helpers, page objects. Modules & helpers are automatically converted to actor’s methods, but how do actors and page objects inside the cests?

You can don’t believe me, but really easy-peasy. The Codeception framework uses the best pattern ever is dependency injection. You can put all your additional stuff inside the method arguments with type-hinting, and you will get them. Let’s create the tests/acceptance/BooksArchiveCest.php file:

<?php

use PageObjectsBooksOverview;

class BooksArchiveCest {
	public function createNewBook( AcceptanceTester $i, BooksOverview $booksOverview ) {

	}
}

Let’s write some logic to our createNewBook test:

<?php

use PageObjectsBooksOverview;

class BooksArchiveCest {
	public function createNewBook( AcceptanceTester $i, BooksOverview $booksOverview ) {
		$bookName   = 'ABC Book';
		$archiveUrl = '/books/';

		$booksOverview->visitBooksOverviewPage();
		$booksOverview->addNewBook();
		$i->see( 'Edit Book' );
		$i->fillField( '/*[contains(@class,"editor-post-title__input")]', $bookName );
		$i->click( '/*[contains(@class,"editor-post-publish-button")]' );
		$i->amOnPage( $archiveUrl );
		$i->see( $bookName );
	}
}

The code has good candidates for page objects: EditBookPage and BooksArchivePage, but I’ve punted it.

One more suggestion is to create one more abstraction layer for your Cests is a Cest class that you can use to modify all your Cests. For example, inside the GH Actions, your browser’s window will be 800×600, which is too small for comfortable testing. Let’s change the screen size to 1980×1050 and create the codeception/Cest.php file:

<?php

abstract class Cest {

	public function _before( AcceptanceTester $i ) {

		global $argv;

		if ( in_array( 'github-actions', $argv, true ) ) {
			$i->resizeWindow( 1980, 1050 );
		}
	}
}

Don’t forget to extends the BooksArchiveCest from our own Cest class.

Note, _before and _after methods, you can use common setups and teardowns for the tests in the class.

Launch a test a few times with different data

Sometimes, you can run your test a few times with different data. It’s beneficial when your test cases depend on special data, e.g., a form validation, API requests, etc. You can use the @example PHPDoc tag and pass a list of needed data when every @example tag runs the test case. The data will be passed to the argument with the CodeceptionExample type:

<?php

use CodeceptionExample;
use PageObjectsBooksOverview;

class BooksArchiveCest extends Cest {
	/**
	 * @example [ 'ABC Book' ]
	 * @example [ 'Our Class is a Family' ]
	 */
	public function createNewBook( AcceptanceTester $i, BooksOverview $booksOverview, Example $data ) {
		$archiveUrl = '/books/';

		$booksOverview->visitBooksOverviewPage();
		$booksOverview->addNewBook();
		$i->see( 'Edit Book' );
		$i->fillField( '.editor-post-title__input', $data[0] );
		$i->click( '.editor-post-publish-button' );
		$i->amOnPage( $archiveUrl );
		$i->see( $data[0] );
	}
}

If you want, you can describe it as an object:

<?php

use CodeceptionExample;
use PageObjectsBooksOverview;

class BooksArchiveCest extends Cest {
	/**
	 * @example { "bookName": "ABC Book" }
	 * @example { "bookName": "Our Class is a Family" }
	 */
	public function createNewBook( AcceptanceTester $i, BooksOverview $booksOverview, Example $data ) {
		$archiveUrl = '/books/';

		$booksOverview->visitBooksOverviewPage();
		$booksOverview->addNewBook();
		$i->see( 'Edit Book' );
		$i->fillField( '.editor-post-title__input', $data['bookName'] );
		$i->click( '.editor-post-publish-button' );
		$i->amOnPage( $archiveUrl );
		$i->see( $data['bookName'] );
	}
}

Or create a special method and then, via the @dataProvider tag, leave the link for your method.

<?php

use CodeceptionExample;
use PageObjectsBooksOverview;

class BooksArchiveCest extend Cest {
	/**
	 * @dataProvider booksProvider
	 */
	public function createNewBook( AcceptanceTester $i, BooksOverview $booksOverview, Example $data ) {
		$archiveUrl = '/books/';

		$booksOverview->visitBooksOverviewPage();
		$booksOverview->addNewBook();
		$i->see( 'Edit Book' );
		$i->fillField( '.editor-post-title__input', $data['bookName'] );
		$i->click( '.editor-post-publish-button' );
		$i->amOnPage( $archiveUrl );
		$i->see( $data['bookName'] );
	}

	private function booksProvider() {
		return [
			[ 'title' => 'ABC Book' ],
			[ 'title' => 'Our Class is a Family' ],
		];
	}
}

Launch order for Acceptance Tests

To achieve higher quality for your tests, the great thing is to run them in random order. To run your tests randomly, you need to update your codeception.yml file:

settings:
    shuffle: true

Right now, before every launch your tests will be shuffled randomly:

$ codecept run 
Codeception PHP Testing Framework v4.1.21
Powered by PHPUnit 9.5.6 by Sebastian Bergmann and contributors.
[Seed] 2145298665

Anything that can go wrong will, and when your test fails in a specific order, you can copy the seed into the --seed option to run it exactly in the same order:

codecept run --seed 2145298665

Your test case can depend on another test case or include the same steps. Here you can help with the @before and @after annotation:

<?php


class BooksArchiveCest extends Cest {

	/**
	 * @before login
	 * @before cleanup
	 * @after  logout
	 * @after  close
	 */
	public function addUser( AcceptanceTester $I ) {

		$I->amOnPage( '/users/charlie-parker' );
		$I->see( 'Ban', '.button' );
		$I->click( 'Ban' );
	}
}

How to debug Acceptance tests in GitHub Actions

The configuration for GH Actions was described inside the 1st part of Acceptance Tests, but I want to emphasize and explain the debug process.

Firstly you can enable debug using the --debug option:

-   name: Run Acceptance Tests
    working-directory: ${{ env.wp-plugins-directory }}
    run: composer acceptance -- --env github-actions --debug

The Codeception framework generates screenshots and snapshots for failed tests inside the codeception/_output/ directory. To get them from GH Actions, you should create an artifact.

-   name: Archive Codeception output
    uses: actions/upload-artifact@v1
    if: failure()
    with:
        name: codeception-output
        path: ${{ env.wp-plugins-directory }}/.codeception/_output

Last but not least, you can make screenshots or snapshots inside your test via tester methods. Then these screenshots and snapshots will be available inside the output directory:

<?php

$i->makeElementScreenshot( '.foo' );
$i->makeHtmlSnapshot();
$i->makeScreenshot();

Coding Standards for Acceptance Tests

I’m susceptible to the clean code of the code and always aim at long-term work. The coding standards are the crucial tool to achieve the goal. All stuff around the Codeception framework is a bit in conflict with WordPress coding standards. I suggest to:

• use camelCase anywhere inside your tests due to Codeception modules
• Allow using methods that start with an underscore to allow using the _before, _after, _passed, _failed, etc. methods.

<?xml version="1.0"?>
<ruleset name="CS">
	<description>Custom coding standards.</description>

	<!-- Coding standards for tests -->
	<rule ref="PSR2.Methods.MethodDeclaration.Underscore">
		<exclude-pattern>codeception/_support/*</exclude-pattern>
		<exclude-pattern>*/tests/*</exclude-pattern>
	</rule>
	<rule ref="WordPress.NamingConventions.ValidFunctionName.MethodNameInvalid">
		<exclude-pattern>codeception/_support/*</exclude-pattern>
		<exclude-pattern>*/tests/*</exclude-pattern>
	</rule>
	<rule ref="WordPress.NamingConventions.ValidVariableName.UsedPropertyNotSnakeCase">
		<exclude-pattern>codeception/_support/*</exclude-pattern>
		<exclude-pattern>*/tests/*</exclude-pattern>
	</rule>
	<rule ref="WordPress.NamingConventions.ValidVariableName.VariableNotSnakeCase">
		<exclude-pattern>codeception/_support/*</exclude-pattern>
		<exclude-pattern>*/tests/*</exclude-pattern>
	</rule>
	<rule ref="WordPress.NamingConventions.ValidVariableName.PropertyNotSnakeCase">
		<exclude-pattern>codeception/_support/*</exclude-pattern>
		<exclude-pattern>*/tests/*</exclude-pattern>
	</rule>
	<rule ref="Squiz.NamingConventions.ValidVariableName">
		<exclude name="Squiz.NamingConventions.ValidVariableName.PrivateNoUnderscore"/>
		<include-pattern>codeception/_support/*</include-pattern>
		<include-pattern>*/tests/*</include-pattern>
	</rule>
</ruleset>

Hopefully, I described all the great stuff for the Codeception framework, and the article was helpful for you all. Feel free to ask any questions in the comments, and don’t forget to follow me on the website and social media.