Automated Testing 101

Setting up your first tests and automation can be achieved by a single person in less than 15 minutes.

Don’t feel ashamed

I’m not a developer. I’ve never written any test in my life, and the last bit of code I wrote myself was during my engineering studies. When I started working at cutting-edge web development company SensioLabs, on Blackfire, it came to me that every developer in the world was writing tests. At least that was what I saw around me every day. Then I started selling Blackfire to the “real world”.

You, reading those words, are not writing tests? Don’t feel ashamed. I can’t count anymore how many times I’ve had that embarrassing moment when I asked customers “OK, I guess you’re already writing unit tests and integration tests?”. And that the customer would hesitate for some seconds and say “Y… yes, we… just started!”.

Automated Testing for the Real World

As a 101 course, let’s first make sure we ask the right questions. What tools to use? How to setup a Jenkins/Travis CI? Which infrastructure do you need? How do you write a test? What is TDD, really? And above all, what are you trying to achieve?

That last question should definitely help you find a guideline. It should support you in gaining experience and confidence in what to do and how. Step by step. Don’t believe you’ll get to bug-free code every time you deploy, overnight.

Blackfire at the rescue

Blackfire’s underlying technology is a profiler. Even though it is first intended to help you find performance bottlenecks in your code and fix it, it does more that that.

As a profiler, it “reads” all of the code which is executed in a request, at runtime. All of the good code, and all of the bad code. And “bad” can mean in terms of performance, but also quality, and security. It tells you the bare truth: what is happening in your code at runtime.

So, how do you write tests?

With Blackfire, at first, you just don’t. We’ve got you covered with “recommendations“. Those are tests written by experts in various frameworks (Symfony, Drupal, Typo3, eZPlatform, Magento) and plain PHP. And they cover performance, quality and security. Each of those tests is fully documented, so that if ever one of them fails on your code, you’ll know:

• Why did we write such a test
• How to fix the issue
• How exactly we wrote this test

That last point is key. Writing a test in Blackfire is pretty much like writing tests with any other tool. It is about writing a list of expectations for your code’s behavior, storing and changing that list together with your code. The good part is that with the Blackfire recommendations, the tests are already written, and that you can copy/paste them to your .blackfire.ymland adjust them to your liking. They’re indeed just “recommendations”. They may, or may not, be relevant to your code. So you can always discard them or make them more relevant!

This gives a first answer to “How do you write a test”. Hopefully it can help you bootstrap.

Now you definitely want to dig further, and understand why you might not want to write a test based on time, even if your objective is to improve your code’s performance.

How do you setup a test infrastructure?

With Blackfire, at first, you don’t. Blackfire is built to be used in production, test, staging, and development. First of all, all of the tests you’ll use from the previous paragraph can be run directly on you local machine.

Now if you’re looking at automated testing, there’s a simple solution. Even if we’d recommend testing before pushing to production, it obviously makes sense to at least test there. And you’re about 5 minutes away from there. Two things.

1. Remember that .blackfire.yml file? Go ahead and list the most business critical URLs in your app, and use them to write scenarios. It probably includes your home page, some checkout or product page. Basically any bit of code that has a direct impact on your app’s mission.
2. Create an “environment” in Blackfire (you’ll need a Premium or Enterprise subscription, or give the demo a try) and simply configure periodic builds and email notifications. All in a few clicks.

Even if you didn’t write any custom tests, Blackfire will start profiling automatically all of the requests you listed. If ever it detects your code doesn’t comply with one of our recommendations, you’ll get an email to warn you about it.

Of course, you need to have a look at Jenkins/Travis CI and other solutions. But Blackfire is a very low hanging fruit to get started!

In a next article, we’ll figure out which next easy step you can make, so you become a pro at testing.

Happy testing!