Performance Tests and Assertions

Speed matters. Application Performance is key to success as a slow app means less user engagement and possibly less revenue.

Over time, performance of most web applications inexorably decline: the size of the response, the time it takes to render the response content, the number of assets, and more.

But how do you test the performance of your web applications? Load-testing is one possible answer, but it’s tedious to setup, and that’s not going to tell you why your application is slow.

At Blackfire, we think that writing performance tests should be as easy as writing unit tests or functional tests. Meet Blackfire assertions. Blackfire assertions are safe-guards that continuously verify that the performance of your application stays in safe boundaries (in terms of CPU, memory, and much more as you will see below).

To get started, first upgrade to the latest version of the Probe, Agent, and Companion. Then, create a .blackfire.yml file under your project root directory, and start adding some tests. The most obvious test is checking that any HTTP request does not take too long to be handled:

tests:
    "All pages should be fast":
        # run the assertions for all HTTP requests
        path: "/.*"
        assertions:
            # wall clock time must be less than 100ms
            - "main.wall_time < 100ms"

An assertion is an expression that returns a Boolean. Now, each time you profile a page, Blackfire is also going to run this test and report the result in the interface. That simple!

Of course, assertions have access to all data gathered by Blackfire. Let’s see how you can leverage this information in your assertions.

Want to be sure that you don’t have any SQL statements executed on the homepage? Restrict the path to / , and add the following assertion:

metrics.sql.queries.count == 0

Want to check that memory consumption is not too high? Done:

main.peak_memory < 5mb

What about the size of the response?

metrics.output.network_out < 100kb

You can even be more precise depending on the libraries your are using. If you are using the PHP PDO abstraction for instance, you might want to limit the number of connections to the database:

metrics.io.db.connection.count <= 2

Using Symfony? Check that the number of sub-requests is sane:

metrics.symfony.subrequests.count < 5

Doctrine fan? Be sure that your code does not create too many entities:

metrics.doctrine.orm.entities.created.count < 50

Or that it does not consume too much memory:

metrics.doctrine.orm.entities.created.peak_memory <= 50mb

The same goes for a lot of other libraries and projects like Redis:

metrics.predis.database.calls.count < 50

As you can see, Blackfire gives you all the information you need to create your very own performance referential.

Detailed results are available on each profile:

Each assertion has been evaluated, with the actual value of variables in grey to let easily understand how far away your code is from the target.

Of course, there is more than that. Performance is not just something you check once. You want to ensure that performance does not degrade over time. To help you out, you can also write assertions that are executed when comparing a profile with a reference one.

Two quick examples:

You probably don’t want the time it takes for your application to render a page to decrease too dramatically from one version of the code to the next one:

percent(main.wall_time) < 10%

And you probably don’t want to let the number of SQL statements executed to generate an HTTP response to increase too badly:

diff(metrics.sql.queries.count) <= 2

Blackfire assertions are very powerful. The whole Blackfire team is very excited about this new feature and we hope that you’re going to love it. Note: Assertions are available to Premium and Enterprise users only.

As always, read the dedicated documentation page to learn more about the assertion syntax, available metrics, and more.

Happy profiling!