Profiling 101 for Python Developers: Profiles Visualizations 5/6
This article is the fifth of a series of six on Profilers in the Python world, and how Blackfire is the best-of-breed tool to introspect code behavior and optimize its performance.
Deterministic Profilers Output Format
A deterministic profiler, such as
cProfile usually outputs formats that can be visualized by other tools.
cProfile outputs something called pstats and can be visualized by snakeviz. Yappi can both output callgrind and
pstats. Callgrind format can be visualized by KCacheGrind.
Deterministic profilers generally outputs Call Graphs. Call Graphs are Directed Acyclic Graphs in which the edges represent the metrics and the nodes represent the function calls. The nodes are usually stored as caller->callee pairs. Such a visualization enables to have a full overview of the code’s behavior, including interactions between the business logic, vendors and frameworks.
Flame graphs are generated by collecting and aggregating sampled call stacks. The x-axis shows the time spent and the y-axis shows the stack depth. Such a visualization enables to find very quickly what are the most cost consuming stack frames. Lots of performance visualization tools have been written to work with full call stack information as they are the natural output for Statistical profilers.
A statistical profiler captures multiple call stack data at intervals. There are many ways to visualize this kind of data. The most used one seems to be flamegraphs. It is the pretty much the standard way of visualizing profiler outputs nowadays.
There is also another format called speedscope which is also worth mentioning. Some recent tools also started using this format, too.
Blackfire offers two complementary types of visualizations: call-graphs, and “timeline”.
The timeline graph is somewhat close to flame graphs. This representation sorts stack frames according to their execution time, from the start of the profiled code to its end. It gives a better understanding of when, in time, a specific event happens or function call is executed.