Questioning PHPUnit Best Practices

The Original Best Practice

A long time ago, when PHP 5.0 had just been released and PHPUnit 2 was the latest and greatest, the approach shown in the example below was the best practice for testing exceptions:

Like any other test, the test shown above has three distinct phases: in the arrange phase the object under test is prepared, in the act phase the action to be tested is performed, and in the assert phase the outcome of the action is verified. The best practice for testing exceptions back then was to wrap the code of the act phase into a try block. The return statement in the respective catch block would signal a success to PHPUnit. The call to the fail() would signal a test failure to PHPUnit.

The Current Best Practice

The approach shown in the example above for testing exceptions was inconvenient and often times resulted in hard to read test code. Together with the fact that DocBlock-based annotations were getting popular in the PHP world this lead me to believe that the following would be a better approach:

In the example above, the @expectedException annotation tells PHPUnit that this test expects the code under test to raise an exception of a specified type. If that exception is raised then the test is considered a success. If the exception is not raised the test will be considered a failure.

The implementation of the @expectedException annotation uses the setExpectedException() method to set up the expectation. This method can also be used directly, for instance if you prefer expressing the assert phase explicitly in code rather than in a comment. Due to the nature of exceptions the assert phase and act phase are swapped in the test code shown below:

Over time the @expectedExceptionCode , @expectedExceptionMessage , and @expectedExceptionMessageRegExp annotations were added. These allow the configuration of expectations for exception codes and messages. Unfortunately, though, they were not implemented in a clean way, which made the setExpectedException() method less and less convenient to use as an alternative to the annotations.

Ever since I added the @expectedException annotation to PHPUnit I considered using it a best practice. This was reflected in PHPUnit's documentation as well as in my conference presentations and trainings, for instance.

A New Best Practice

One morning not too long ago I got a phone call from Stefan Priebsch . He had just discussed the topic of testing exceptions with the students of his master class at the University of Rosenheim. One of his students had used the setExpectedException() method in his homework. When Stefan told him that he should have used the @expectedException annotation, the student challenged that best practice. After discussing the topic over the phone I had to admit that the student was right. While the @expectedException annotation is convenient to use it is also problematic. Let's look at the problems the student pointed out.

Back when the annotation was added to PHPUnit there was no support for namespaces in PHP. These days, though, namespaces are commonly used in PHP code. And since an annotation such as @expectedException is technically only a comment and not part of the code you have to use a fully-qualified class name such as vendor\project\Example when you use it. In a comment you cannot use an unqualified class name, Example for instance, that you would be able to use in code when that class is in or imported into the current namespace.

In the example above, we use the expectException() method that was introduced in PHPUnit 5.2 to tell PHPUnit that the test expects an exception of a specified type to be raised. Thanks to the class constant that holds the fully-qualified name of a class we do not need to write vendor\project\ExpectedException in the test code, we can write ExpectedException::class instead. This improves the readability of the test and makes automated refactorings in modern IDEs such as PhpStorm reliable.

Another advantage of setting up the expectation in the test code has to do with the three phases of a test we discussed earlier. When we use the @expectedException annotation then PHPUnit will consider the test successful if the exception is raised at any point in time during the execution of the test method. This may not always be what you want:

PHPUnit will consider the test shown in the example above then and only then a success when the expected exception is raised after the call to the expectException() method. If the constructor of the Example class raises an exception of the same type then this will be considered an error by PHPUnit.

In addition to the expectException() method, PHPUnit 5.2 also introduces the expectExceptionCode() , expectExceptionMessage , and expectExceptionMessageRegExp() methods for programmatically setting expectations for exceptions. There is nothing that you can do with these new methods that you could not have done with the old setExpectedException() method. However, because setExpectedException() can do everything these new methods can its API was convoluted and inconvenient. Separation of concerns, anyone? The setExpectedException() method has been deprecated and will be removed in PHPUnit 6.

For the sake of completeness, I should also mention the following idea that has been brought up :

In the example above, the hypothetical assertException() method would execute the code of the act phase and then perform the assertion. The argument that is made in favor of this approach is that it allows checks for multiple exceptions within a single test. In my opinion, though, this is an argument against it. A unit test should be fine-grained and only test a single aspect of one object. Why would you need to check for different exceptions in a single test?

Hindsight is easier than foresight. I am not sure anymore that adding the @expectedException did more good than harm. I do not currently plan to remove it from PHPUnit, though. But going forward I will recommend using expectException() etc. for testing exceptions. I would like to invite you all to question best practices like this. Nothing can be set in stone if we want to evolve PHP and its ecosystem of tools, frameworks, and libraries.

Update (February 6, 2016): Added discussion of using closures for testing exceptions based on feedback from Andrea Faulds.