The Power of Distributed Profiling

When meeting people in conferences (I mean, IRL like conferences used to be 🤔), we often get asked about the ability of Blackfire to profile micro-services. The answer is YES, thanks to Distributed Profiling. Distributed Profiling is indeed a feature that makes it possible to profile any service your application interacts with, the most common being other HTTP applications such as micro-services.

But it doesn’t stop there, especially because Blackfire supports many languages such as PHP, Go, and Python! There is no limit in the number of sub-profiles that can be generated, either from HTTP or from CLI commands. You can generate a cascade of profiles and sub-profiles from all the components your application interacts with.

For example, a profile of a CLI application written in Go can be propagated to an HTTP application written in PHP, and to another one written in Python:

Go app (CLI)
├── PHP app (HTTP)
│   └── Go app 2 (CLI)
└── Python app (HTTP)

How does it work?

Another recurring question from developers is: “How does it work? Do all the apps need to go through the same agent?” (OK that makes 2 questions 😅).

Profiles and sub-profiles are linked together with parent/child identifiers. Data from all the profiles is collected by the different probes, and processed by the agent(s). In the end, like for any profile, the agent(s) send the aggregated data to Blackfire.io servers. The parent/child chain of profiles is then being reconstructed by Blackfire.io servers, giving you the ability to consistently visualize the cascade of profiles.

The parent/child hierarchy of profiles being treated by Blackfire.io, you might understand that Distributed Profiling is not limited to only one agent! Even better, all the agents involved don’t have to be located within the same network; as long as the requirements are fulfilled, the feature will run as expected.

Last but not least, agents may even use different credentials. In this case, the requiment is that the user triggering the main profile has access to the environments tied with the different agents.

I want to use it, please tell me how to

Distributed Profiling is available for all Premium and Enterprise plans.

You don’t have anything specific to configure in the probe or in the agent, as propagation is activated by default. You can still disable it from blackfire CLI command with --no-propagation switch, or by checking Disable distributed profiling in Blackfire browser extension.

Depending on the architecture of your application, you may need to adapt your code a bit. In PHP, if you use curl, file_get_contents() , or fopen() functions (which are used by most PHP HTTP clients, including Guzzle), you don’t have to do anything as the probe will instrument these calls to add the specific HTTP headers.
However, if you want to sub-profile a CLI command, or if you use Python and/or Go, you will need to adapt your code a bit by generating a sub-profile query with the help of the available SDKs.

See the examples below.

PHP

// Spawning to sub.php.
// Works with any language supported by Blackfire.

$processEnv = $_ENV;
if (\BlackfireProbe::isEnabled()) {
    $probe = \BlackfireProbe::getMainInstance();
    $processEnv['BLACKFIRE_QUERY'] = $probe->createSubProfileQuery();
}

$proc = proc_open(
    'php sub.php',
    [STDIN, STDOUT, STDOUT],
    $pipes, null,
    $processEnv
);
proc_close($proc);

Read more information on generating sub-queries in PHP.

Python

import requests
from blackfire import probe

with probe.run():
    url = 'https://mycustomapi.com'
    headers = {'X-Blackfire-Query:': probe.generate_subprofile_query()}
    r = requests.get(url, headers=headers)

Read more information on generating sub-queries in Python.

Go

import "github.com/blackfireio/go-blackfire"

if id, err := blackfire.GenerateSubProfileQuery(); err == nil {
    w.Header().Set("X-Blackfire-Query", id)
}

Read more information on generating sub-queries in Go.

Happy Distributed Profiling!