Automated testing is a way to improve product quality

Testing –  checking the correspondence between the real and expected program behavior.

Some smart person

Adding new functionality, you need to check how these changes affected the entire application. You should check how the new feature works and whether the previously written features work the same way. The larger your application, the more time it takes to test it. I’ve participated in teams where each developer had two or three quality assurance engineers.

I hate the monkey job and love product development. In general big products just must have automation testing. Automation testing is about a transformation from monkey job to the quality knowledge base of your application and makes your business more flexible and more independent.

The advantages and disadvantages of automated testing

The advantages:

• fixing more bugs in the development stage;
• reducing the time for double-checking changes;
• quality improvement of your application;
• saving time for testing as longtime value;
• multiple usages.

The disadvantages:

• a higher level of staff is needed;
• time for implementation of a test environment and test culture;
• more time for first testing;
• decrease in productivity at the initial stage.

Myths of automated testing

After the implementation of automated testing, there will be no more bugs.

An endless number of bugs can be found in any code. Tests help you to find them during development and find more of them than you would manually find.

We can implement the automation tests for a few days/weeks/months.

Unfortunately, no. You can implement and write the first tests quickly enough, but maintaining the test environment and controlling the quality of tests is a continuous process. Tests also need to be validated, you can even test tests.

Test Coverage must be 100%

Of course, you can strive for this. But I would prefer 60%. For every next 20%, I would double the time limit. For example, for 60% coverage, it takes 100 working hours, and for 100% – 400 hours. It’s very simple, for tests you need to prepare the code and additional time will be spent on refactoring.

Automated testing layers

The main idea is to create as large a test base as possible and minimize manual testing. The testing base allows us to reuse the tests by request or inside task for continuous integration.

Manual Testing

It’s a kind of developer’s art to foresee all possible options and not overstep the line of unnecessary engineering.

We all have at least some idea of manual testing. But I would recommend reading the best manual testing practices for developers because it will help them write better code. A good trick while testing your application is to change your goal from “make sure it works” to “make sure it breaks.”

• Environment: none
• Code: none
• Author: a developer or QA engineer
• Speed: very slow
• Tests quality: not bad

Acceptance, End-to-End, GUI, Walk-Through tests.

The simplest type of testing. Using a test thread, you need to start the environment in which your application is running and execute the actual user behavior. One user’s story is called a use case. Use cases are very easy to write, very often they are written by QA engineers.

The tricky part is deploying a real environment to run the tests. For example, for the web, you need to install a web driver and a test framework. You will work with the real database, with the front-end and back-end part of the application, everything that testers do. This is the only test level that validates the overall application work.

• Environment: hard
• Code: very easy
• Author: QA engineer or developer;
• Speed: slow
• Tests quality: maximum

Example of test case where customer logins to the admin area and visit some page:

$I->loginAsAdmin();
$I->amOnPage( '/wp-admin/admin.php?page=plugin-name' );
$I->see( 'Plugin Name Settings' );

Very simple code and intuitive to understand.

Integration, Backend, Contract, API driven tests

This test level is a stripped-down version of the previous level. In general, you just need to remove the front-end part, which will speed up the test speed several times. This is a transitional level between the unit and acceptance test layers. Most of the use cases are written by developers. From the name you can understand that we need to check the work of the backend, your API, or an external API, all cases where the frontend part not important for us.

For example, we called some function that launched several classes and these classes somehow interact with each other. How they do it is not important to us, it is only important that passing some data to the input, we will receive the expected data at the output.

• Environment: medium
• Code: medium
• Author: a developer
• Speed: medium
• Tests quality: medium

Example test for request in some API:

$client   = new Client();

$response = $client->do_request();

$this->assertSame( 200, $response->get_code() );
$this->assertSame( '{"success": true}', $response->get_body() );

In general, it doesn’t matter how the Client will be implemented, it’s important for us that the do_request method returns a response to us as the object that has methods get_code and get_body, and these methods return the data that we expect. If the answer is changed in the API, then we will find out about it, because our tests will fall.

Unit, Block, Component tests

I’m sure all developers have heard of unit tests. In my practice, I’ve noticed that the most difficult thing is to understand the difference between this level and others. We need to test functions and public methods separately from the whole system. There is only a class/function and its internal world that needs to be checked, for all dependencies with the external world we use stubs or mocks.

These tests are written by developers for developers. The level of code quality changes dramatically after the introduction of a unit testing culture. If the unit tests are broken, you can definitely see a problem spot in the code.

Unit tests are a code quality tool that allows you to modify painlessly how legacy code works. Of course, the development time increases, but it also allows you to find problems in the code during the development. Also, these tests are incredibly fast, because they only require the CLI version of the language to work.

• Environment: medium
• Code: easy
• Author: a developer
• Speed: rocket
• Tests quality: easy (because it coverage for only code level)

Example of test for enqueue styles inside WordPress:

$settings = new Settings();
expect( 'wp_enqueue_style' )
	->once()
	->with(
		'plugin-name-settings',
		PLUGIN_NAME_URL . 'assets/css/build/settings.css',
		[],
		Plugin::VERSION,
		'all'
	);

$settings->enqueue_styles();

The main idea of unit testing is testing outside the other dependencies. For example, we use the mock (the expect function) instead of WordPress function wp_enqueque_script which describes the arguments and amount of executing.

Conclusion

The base of the pyramid is the test knowledge base that can be reused. This will allow us to achieve a result in which adding new functionality doesn’t break the old features in any way. And if it does, we will see it

Do you have any ideas, questions, suggestions? You can get in touch with me.