thePHP.cc Logo Deutsch Contact
Migrating to PHPUnit 9

Migrating to PHPUnit 9

In the area of conflict between progress and backward compatibility, a good developer experience must not be neglected.

February: that time of year when I write articles like "Help! My tests stopped working." or " The Death Star Version Constraint ". In this year's installment I provide guidance on how to migrate from PHPUnit 8 to PHPUnit 9 .

Do not fear major versions

Through the concatenation of nouns, the German language enables the formation of practically endlessly long words like "Hauptversionsnummererhöhungsangst". With this word, one describes the fear of increasing the major version number, for example from 8.5 to 9.0, that many software projects seem to have.

For example, there were eleven years between the release of PHP 5.0 and the release of PHP 7.0 . Mozilla has shown that there is another way with the release process for the Firefox browser, of which a new major version is released every six to eight weeks. Starting this year, Mozilla even plans to release a new major version of Firefox every four weeks. Against this backdrop, the annual increase in PHPUnit's major version numbers is tame.

PHPUnit 9, a new major version of the standard test framework of the PHP world, was released today. A new major version number like "9" is usually associated with a large amount of new features or other performance improvements. However, the PHPUnit project handles this differently and, for example, PHPUnit 9 .0 does not bring any significant new features. These will only come with PHPUnit 9 .1 et cetera later this year.

A new version of PHPUnit is released every two months, on the first Friday in February, April, June, August, October and December. The new version in February is a new major version, the other versions are new minor versions. Following the principles of Semantic Versioning, this means that the February version is not backwards compatible with the previous versions and, for example, removes features. The other versions, on the other hand, are compatible with their predecessors and, for example, introduce new features. But why are features removed from PHPUnit every February or changed so that it is necessary to adapt your own tests?

Why does PHPUnit remove functionality?

PHPUnit is almost twenty years old. On the one hand, this means that features that were necessary five, ten, fifteen, or even twenty years ago no longer have any use today. In many cases, such as supporting direct testing of private object properties, old functionality not only no longer has a positive benefit, it actually has a negative effect: developers think that just because there is a specific functionality in PHPUnit it is a good idea to use it. In the worst case, this can lead to software being developed improperly. On the other hand, software like PHPUnit has to be constantly adapted to changes in the ecosystem: new versions of PHP must be supported as well as new distribution channels. The migration of a code base like that of PHPUnit cannot be migrated overnight from PHP 5 to PHP 7 . At the time, the replacement of PEAR packages with the support for Composer and PHP Archives (PHARs) also took a lot of time and effort. In addition, the complexity of the code increases due to workarounds for bugs in certain PHP versions and support for different PHP versions in the same line of PHPUnit versions as well as the addition of new functionality.

For a long time I was, more or less, the only developer of PHPUnit. Because only I was able – and willing – to work with this code base. That was my fault, because twenty years ago I did not know what I know about architecture and programming today and I could not foresee the consequences of design decisions.

Alberto Brandolini describes such a situation as follows:

The dark secret of the Dungeon Master is that he knows every trap in the existing legacy software, because he was the one to leave the traps around. [...] Knowledge, in the form of accidental complexity starts accumulating in the head of the Dungeon Master, and silently grows.

– Alberto Brandolini: " The rise and fall of the Dungeon Master "

It comforts me a little that I am not alone. Erich Gamma and Kent Beck, the original authors of JUnit, have similar problems:

The problem is in software design often the consequences of your decisions don't become apparent for years. One of the advantages of having to live with JUnit for 8 years, is now we can look back and see which decisions we made worked nicely and which we would have done differently.

– Kent Beck: " We thought we were just programming on an airplane "

In recent years, the situation of PHPUnit's code base has improved significantly. On the one hand, the annual major versions, which both remove or change functionality from PHPUnit in such a way that implementation and use become easier as well as the removal of support for old PHP versions contribute to the fact that PHPUnit's code base is now easier to understand and maintain. On the other hand, regular code sprints have lead to new developers joining the PHPUnit project. These new developers now support me in my work.

Marco Pivetta summarizes this as follows:

[PHPUnit] has been very stale for ages, and is now moving much faster thanks to people now willing to invest time in it (thanks to constant rework and cleaning)

– Marco Pivetta on Twitter

PHPUnit 9

After the "why" of the annual changes has hopefully become clearer, it is time for us to take a closer look at the changes in PHPUnit 9 .

PHP 7.3 Requirement

PHPUnit 9 requires PHP 7.3 (or newer). The active support of PHP 7.2 by the PHP project ended on November 30, 2019. As of today, only PHP 7.3 and PHP 7.4 are officially and actively supported. PHPUnit 9 is supported on PHP 7.3 and PHP 7.4. When PHP 8 is released at the end of the year, this version will very likely also be supported by PHPUnit 9 .

If you are still using PHP 7.2, you should now start migrating to a current PHP version, ideally PHP 7.4. The PHP project no longer offers bug fixes for PHP 7.2 and security-critical bugs will only be fixed by November 30, 2020. A long-term goal should be to update the PHP version used as a regular task - and not as a special project that is only tackled every jubilee year. The corresponding update process should be based on the active support of the PHP project for current PHP versions .

If you cannot or do not want to use PHP 7.3, you cannot use PHPUnit 9 , of course. This is not yet a major problem, since PHPUnit 8.5, which still works with PHP 7.2, will be supported with bug fixes until February 2021 . However, you will of course miss any improvements in newer versions of PHP and PHPUnit.

assertEquals() (and related assertion methods)

assertEquals() is the most commonly used assertion method. Unfortunately, because assertSame() is almost always the better alternative. Many optional parameters have been added to assertEquals() over the years. Some of these could not be used together, which has repeatedly led to edge case errors in the underlying implementation. The main problem with a long list of optional parameters is that you have to specify the first four if you want to use the fifth, for example. And who can remember what all these parameters are good for? At least not me.

To remedy this, specialized alternatives were introduced in PHPUnit 7.5 and the following optional parameters of assertEquals() were deprecated in PHPUnit 8 :

These parameters, of which $maxDepth did not have an effect for a long time, were now removed in PHPUnit 9 .

The optional parameter $delta could be used to compare two numerical values with each other in such a way that they are also considered to be the same if they – apart from a delta – are only almost the same. For example,

$this->assertEquals(1.0, 1.1, '', 0.1);

asserts that the actual value 1.1 corresponds to the expected value 1.0 with a delta of 0.1 . This can (and must) now be expressed easier and more readable with

$this->assertEqualsWithDelta(1.0, 1.1, 0.1);

The optional parameter $canonicalize could be used to bring both the expected and the actual value into a canonical form before their comparison. For example, sorting is performed for values of the array type. Since $canonicalize was the sixth parameter (as well as the fourth optional parameter after the two non-optional parameters $expected and $actual ), you had to do this in the past with

$this->assertEquals($expected, $actual, ''', 0.0, 10, true);

This can (and must) now be expressed easier and more readable with

$this->assertEqualsCanonicalizing($expected, $actual);

The optional parameter $ignoreCase could be used to ignore upper and lower case for the comparison of expected and actual value. In the past, such an assertion could only be expressed with

$this->assertEquals($expected, $actual, ''', 0.0, 10, false, true);

This can (and must) now be expressed easier and more readable with

$this->assertEqualsIgnoringCase($expected, $actual);

For the sake of completeness, it should be mentioned that everything that has been explained above also applies to assertNotEquals() , the inverse of assertEquals() .

Analogous to removing the optional parameters $canonicalize and $ignoreCase from the API of assertEquals() , the corresponding optional parameters were also removed from the API of assertFileEquals() and assertStringEqualsFile() . In their place, there are now specialized assertion methods:

assertContains()

Over time, optional parameters had also been added to the assertContains() assertion method. In addition, their range of application from arrays and objects that implement the Iterator interface has been expanded to strings. Again and again there were problems that also resulted in the implementation of different, sometimes conflicting, use cases in the same code unit.

To remedy these problems, PHPUnit 7.5 introduced specialized alternatives that only operate on strings. The optional parameters $checkForObjectIdentity , $checkForNonObjectIdentity , and $ignoreCase were deprecated in PHPUnit 8 . In addition, the use of assertContains() with string haystacks was also deprecated. The parameters mentioned and the possibility to use assertContains() with string haystacks have now been removed in PHPUnit 9 .

assertContains() has been changed so that a type-safe comparison (with the === operator) is always performed. If a comparison is desired that is not type-safe (with the == operator), the new assertContainsEquals() method should be used. Here are some examples:

For the sake of completeness, it should be mentioned that everything that was stated above also applies to assertNotContains() , the inverse of assertContains() .

assertInternalType()

The assertInternalType() assertion method could be used to ensure that a variable contains a value of a certain type that is not user-defined. Here is an example that ensures that a variable contains an array:

$this->assertInternalType('array', $variable);

An important aspect of the assertion shown above is hidden in a parameter, 'array' . This is problematic. For example, an IDE cannot offer auto-completion for 'array' . Furthermore, there is no protection against typing errors in the 'array' string. After all, it is just a string.

To fix this problem, specialized alternatives were introduced in PHPUnit 7.5 . These new assertion methods no longer hide essential information in a string parameter, but make their responsibility explicit in their method name:

$this->assertIsArray($variable);

Of course, a more explicit API means that there are more methods. However, each of these methods is simpler because it only implements exactly one use case. This leads to code that is easier to understand. It is also easier to write because the IDE can now offer auto-completion.

For the sake of completeness, it should be mentioned that everything that has been explained above also applies to assertNotInternalType() , the inverse of assertInternalType() .

assertArraySubset()

The assertArraySubset() method was a constant source of confusion and frustration, which can be found in many tickets in the PHPUnit issue tracker. This situation arose because I had not reviewed the original pull request thoroughly enough. I myself had no use for the proposed assertion method. At first glance, the implementation looked like it would meet the use case of the developer who proposed the new functionality. Over the years I have repeatedly accepted pull requests that I thought would only fix bugs in the assertArraySubset() implementation. However, some of these pull requests added new functionality that partially conflicted with other existing and supported use cases of assertArraySubset() . This should not have happened. However, this additional functionality could not be removed without breaking backwards compatibility.

While working on PHPUnit 8, I came to the realization that the problems of assertArraySubset() cannot be solved without breaking the backwards compatibility. So I decided to deprecate assertArraySubset() and remove it in PHPUnit 9 . This results in a backward compatibility break that is explicit and obvious. Any change in the behavior of assertArraySubset() would also be a break in backward compatibility, but this would have been implicit and not obvious.

If you need the functionality of assertArraySubset() , you can restore it by installing an extension for PHPUnit by Rafael Dohms.

Assertions and Non-Public Properties

Too large objects with problematic dependencies are common in legacy code. Such objects often have to be tested indirectly by considering their non-public state. In the past, assertions such as assertAttributeEquals() were introduced that work on non-public properties. These assertions were only intended for testing legacy code. Their use was never recommended as best practice, and certainly not for new code. Unfortunately, these assertions were used too often to test new code that neglected testability. It is a mistake to try to test the content of non-public properties. Instead of wanting to test the condition of an object, the behavior of an object should be tested.

Over the years, it has been shown that providing assertion methods such as assertAttributeEquals() has resulted in poor testing practices. For PHPUnit 8 I therefore decided to deprecate these assertions and in PHPUnit 9 they have now been removed.

Non-public methods should not be tested directly by bypassing their visibility using the Reflection API. The private state of an object must also not be considered in a test. Both of these practices result in a tight coupling of test code to the code being tested, and therefore in testing implementation details instead of the API. As soon as the implementation changes, the test breaks because it relies on private implementation details.

Expecting Exceptions

For a long time, it was best practice to test exceptions by documenting the expected exception using the @expectedException annotation on the test. This approach has two fundamental problems:

I now believe that using annotations for expecting exceptions does more harm than good. Therefore, I deprecated the use of annotations such as @expectedException in PHPUnit 8 and the corresponding functionality has now been removed in PHPUnit 9 .

/**
 * @expectedException \vendor\project\MyException
 */
public   function   testSomething ( ) :   void
{
     // ...
}

The example shown above needs to be adapted like so:

public   function   testSomething ( ) :   void
{
     $this -> expectException ( MyException :: class ) ;
 
     // ...
}

While we are on the topic of testing exceptions: the expectExceptionMessageRegExp() method no longer exists in PHPUnit. It was already deprecated in PHPUnit 8 and expectExceptionMessageMatches() should be used instead.

PHP Errors

The classes Deprecated , Error , Notice , Warning in the PHPUnit\Framework\Error namespace are internal implementation details of PHPUnit, with which corresponding PHP error situations are represented. Nevertheless, you had to use these private details of PHPUnit in your tests so far, for example, to be able to test that a certain PHP error occurs during a test:

<?php  declare ( strict_types = 1 ) ;
use   PHPUnit\Framework\TestCase ;
use   PHPUnit\Framework\Error\Notice ;
 
final   class   Test   extends   TestCase
{
     public   function   testOne ( )
     {
         $this -> expectException ( Notice :: class ) ;
 
         $a   =   $b ;
     }
}

Specialized methods exist now that can be used to document the expected occurrence of a PHP error:

The use of expectException() with Deprecated , Error , Notice , Warning was deprecated in PHPUnit 9 and will no longer be possible with PHPUnit 10 .

The example shown above must now be formulated as follows:

<?php  declare ( strict_types = 1 ) ;
use   PHPUnit\Framework\TestCase ;
 
final   class   Test   extends   TestCase
{
     public   function   testOne ( )
     {
         $this -> expectNotice ( ) ;
 
         $a   =   $b ;
     }
}

Test Stubs and Mock Objects

The methods getMockBuilder() , createMock() , createConfiguredMock() , and createPartialMock() can be used to create objects that can be used instead of a real dependency or instead of a real collaborating object in a test. To do this, it is sufficient to specify the type of the object, either in the form of an interface name or a class name: $this->createMock(Service::class) creates an object that looks like (has the same type as) a Service object but does not execute the real object's code.

Over the years, the code generator behind PHPUnit's scenes that makes the API shown above work for creating mock objects has been expanded to include more than one type for creating a mock object. Since it is almost always a sign of bad software design when this functionality is required, and because this functionality (unnecessarily) complicates the implementation of the code generator for mock objects, the specification of more than one type for the methods listed above is deprecated in PHPUnit 9 . This functionality will no longer be available in PHPUnit 10 .

CLI Test Runner

The ability to call phpunit ServiceTest.php and alternatively phpunit ServiceTest on the command line has resulted in unnecessarily complicated code that was difficult to understand and very susceptible to errors. The phpunit ServiceTest variant was therefore already deprecated in PHPUnit 8 and has now been removed in PHPUnit 9 .

Let's start from scratch!

Every developer knows the wish "Let's throw it all away and start over on the green field!". This wish is also not foreign to me and so I have thought several times over the years to develop PHPUnit as a whole or in parts (for example "only" the test runner) from scratch. I last thought about this two years ago and shared my thoughts and ideas in a presentation .

So far, I have always decided against such a revolutionary approach. Firstly, because it would require a lot of work that would have to be done in one piece. Above all, however, because with such a fresh start it would be almost inevitable that all tests using PHPUnit would have to be adjusted at least minimally – but probably significantly.

For me, therefore, only an evolutionary approach makes sense, in which parts of PHPUnit are exchanged or improved step by step. During a code sprint last year, for example, Marco Pivetta removed very old and dirty code that takes care of parsing the annotations in PHPUnit tests. The new code has fewer errors and is much easier to understand and adapt. This work has already been incorporated into PHPUnit 8 . For PHPUnit 9 the code which takes care of loading the XML configuration file has now been rewritten. For 2020, the development of an event-based extension system for the test runner is on the agenda. Since the last code sprint, this topic has been worked on by Ewout Pieter den Ouden, Andreas Möller, and Arne Blankerts. This work will likely be incorporated into PHPUnit 9.1 or PHPUnit 9.2 .