Crawling a page with Blackfire Player

By Grégoire Pineau, on Mar 06, 2017

In January, we released the new version of the Blackfire Player. Blackfire Player is a powerful Web Crawling, Web Testing, and Web Scraper application. It provides a nice DSL to crawl HTTP services, assert responses, and extract data from HTML/XML/JSON responses. This blog series gives concrete examples of how the Player features can be used.

It integrates with Blackfire in order to automate performance testing, but can also cover a lot more use cases, without necessarily involving any sort of performance management.

Let’s have a look at a use case on the crawling capacities of Blackfire Player. With the simple DSL below, we can crawl each Symfony version tagged on GitHub, and return the results in a JSON file.

This:

Executed by this:

Gives this:

 

Grégoire Pineau

Grégoire is a (Symfony-certified!) developer at Blackfire.io. He's also involved in Symfony development and several other Open Source projects, and is the Swiss Army Knife of DevOps. It would only be him, most of the projects he worked on would be named after savanna animals.