Profiling 101 for Python Developers: Profiles Visualizations 5/6

By Sümer Cip, on Feb 10, 2020

Deterministic Profilers Output Format

A deterministic profiler, such as Yappi or cProfile usually outputs formats that can be visualized by other tools. cProfile outputs something called pstats and can be visualized by snakeviz. Yappi can both output callgrind and pstats. Callgrind format can be visualized by KCacheGrind.

Deterministic profilers generally outputs Call Graphs. Call Graphs are Directed Acyclic Graphs in which the edges represent the metrics and the nodes represent the function calls. The nodes are usually stored as caller->callee pairs. Such a visualization enables to have a full overview of the code’s behavior, including interactions between the business logic, vendors and frameworks.

Flame graphs are generated by collecting and aggregating sampled call stacks. The x-axis shows the time spent and the y-axis shows the stack depth. Such a visualization enables to find very quickly what are the most cost consuming stack frames. Lots of performance visualization tools have been written to work with full call stack information as they are the natural output for Statistical profilers.

Statistical Profilers Output Format

A statistical profiler captures multiple call stack data at intervals. There are many ways to visualize this kind of data. The most used one seems to be flamegraphs. It is the pretty much the standard way of visualizing profiler outputs nowadays.

There is also another format called speedscope which is also worth mentioning. Some recent tools also started using this format, too.

Blackfire visualizations

Blackfire offers two complementary types of visualizations: call-graphs, and “timeline”.

The timeline graph is somewhat close to flame graphs. This representation sorts stack frames according to their execution time, from the start of the profiled code to its end. It gives a better understanding of when, in time, a specific event happens or function call is executed.

Sümer Cip

Sümer Cip is currently a Senior Software Engineer at Blackfire. He has been coding for nearly 20 years and contributed to many open source projects in his career. He has a deep love for Python and obsession with code performance and low-level stuff.