Understanding continuous profiling:  part 1

Blackfire recently announced the release of its continuous profiler for PHP, Python, Node.js, and Golang. This new addition to the Blackfire observability solution breaks new ground with a feature combining profiling and monitoring with minimal overhead.

With its unique, lightweight, and holistic nature, our new continuous profiler is designed to enrich your observability strategy. By collecting in-depth information on every activity for the supported runtimes, the continuous profiler is a hugely valuable asset.

If the success of an observability initiative lies in the quality of the information obtained from the collected data and in how smoothly it has been integrated into the development and deployment workflows, it is important to understand the fundamental differences between how we collect data and turn it into actionable insights.

Understanding the nature of continuous profiling

Fully understanding the nature of continuous profiling requires distinguishing between deterministic and probabilistic observability, which are not self-revealing terms. This series of blog posts will guide you through the different data types Blackfire’s features collect to understand their nature, potential, and overhead costs fully.

We embark on a journey through Blackfire components that will walk you backstage and highlight the inherent nature of the different data collection methods. By the end of this series, you should clearly understand the different layers of observability data and tools available to you so you can combine them in a way that best suits your applications and teams.

To begin with, let’s start with the basic and the deterministic profiler. Blackfire has been historically built around a deterministic stack. Let’s trigger a profile and explore the information provided in a way that shines a light on the inherent nature of deterministic profiling.

The deterministic profiler

The application at escape.book.b7e.io can be profiled by anyone. You need a Blackfire account, but not necessarily a Blackfire subscription.  This simple app is a discovery game often used during events to have attendees discover Blackfire Profiler in about 10 minutes. If you’re not yet familiar with Blackfire, or interested in solving geeky puzzles, head to escape.book.b7e.io/landing to get started.

One of the simplest ways to trigger profiles is through the browser extension. Let’s click on the big red button to get started. Behind the scenes, the browser extension is calling that URL with a specific header that informs Blackfire Probe that it needs to start collecting data.

Then, let’s click on the View Timeline button. Here is a public profile of the Blackfire Escape game homepage:
https://app.blackfire.io/profiles/e99d2386-c3c0-45d3-9c77-33135e726eac/graph

Exploring the profile dashboard

The timeline view represents all the functions called during the script’s execution. Each box represents a unique function call. The larger the box, the more time spent on that function. From top to bottom, we can witness to caller/callees relationship. 

To express it in a way that will ease later comparisons, the deterministic profiler only collects data when explicitly instructed to. Therefore, the profile we are exploring only shows the reality at the time of measurement, excluding previous samples of the same endpoint and other requests being processed at the exact same time.

We can also see a blue wave in the background. This represents the evolution of the peak memory consumption over time. While letting us visualize eventual memory leaks in our code, it also informs us of another key aspect of Blackfire profiling.

Multidimensional observability

Deterministic observability is inherently multidimensional. We collect data on the consumption of multiple dimensions, then process it and display the output information in the most actionable way possible.Let’s now look at a pane on the left side of the timeline view. This section lists all the metrics related to the profiled script. The deterministic profiler has this particularity to collect data from the very first function being called to the very last.

Consequently, we can derive metrics from those observations by grouping individual function calls into categories of resource consumption families. The fact that every function is being scrutinized makes those metrics 100% reliable.

Compare deterministic measurements

We will then be able to make comparisons based on those metric values. We will later see that we can make comparisons between measurements (or profiles), and also compare a profile with the expectations we would have previously defined.Our exploration of the profiles continues with the call graphs which are tree-like representations of sequences of function and service calls made when handling this request. Again, one box usually represents one function call, even if some less impactful ones can be grouped together to make the overall easier to apprehend at a glance.

Some of those boxes have a colored border. Together, they compose the critical path or the sequence of calls consuming more resources than all the other sequences of calls. The nodes with a colored background consume significantly more resources than the other nodes. In both cases, the stronger the color, the more resource-intensive the element is.

The need for actionable representations of the collected data

This highlights another important aspect of an observability initiative. The amount of data collected may remain worthless unless processed and visualized in an actionable way. Call graphs allow us to spot the most resource-consuming elements for an application quickly, zoom in on them, and climb up the tree until we identify the responsible function calls on our codebase.

Since deterministic observability is multidimensional, there is also one call graph for each available dimension: wall-time, I/0, CPU, peak memory, and network. Not only are we observing each and every function call, but, by evaluating their contribution to all dimensions, we can display specific representations of their dynamics.

The profile also has SQL, HTTP, and Cache tabs, which list an exhaustive list of SQL queries, HTTP calls, and the exact configuration and usage of cache.  That information is inherently accurate because all function calls are observed and scrutinized.

Control performance with automatically evaluated tests

Finally, the possibility of relying on metrics allows performance tests and recommendations. Recommendations are actionable insights prompted when specific combinations of metrics are observed.

As we will see in the next installment of that series, the performance tests rely on user-defined assertions. Every time a profile is made, all the matching assertions will be evaluated against the observed metrics to determine whether the performance requirements are met.

The nature of deterministic profiling

Let’s wrap up by summarizing what we gathered from the deterministic profiling so far. Deterministic profiling is:

• Multi-dimensional
• Observe all functions of the designated requests/scripts
• Two samples can be compared
• A sample always contains reliable pieces of information
• Metrics can be derived from samples
• Strong overhead

In the next post, we will explore how far we can push deterministic observability’s logic. This is also the story of how Blackfire built such a compelling observability solution from its unique profiler. Meanwhile, let’s continue the conversation on Dev.to,  Slack, Reddit, or our new community portal. 

To better observability and beyond!