Skip to content
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
dd27e5e
Extend RemoteUnwinder to capture precise bytecode locations
pablogsal Dec 3, 2025
70f2ae0
Add opcode utilities and --opcodes CLI flag
pablogsal Dec 3, 2025
aedc000
Track opcode sample counts in flamegraph collector
pablogsal Dec 3, 2025
19ff11b
Emit opcode interval markers in Gecko collector
pablogsal Dec 3, 2025
af27d23
Add bytecode panel to heatmap visualization
pablogsal Dec 3, 2025
7ffe4cb
Add opcode panel to live profiler TUI
pablogsal Dec 3, 2025
8b423df
Update tests for location tuple and opcode field
pablogsal Dec 3, 2025
8129e3d
Merge remote-tracking branch 'upstream/main' into tachyon-opcodes
pablogsal Dec 7, 2025
965f521
Better test
pablogsal Dec 7, 2025
dac78a5
Merge remote-tracking branch 'upstream/main' into tachyon-opcodes
pablogsal Dec 7, 2025
12c02f6
Add news entry
pablogsal Dec 7, 2025
c10628a
CI fixes
pablogsal Dec 7, 2025
04563f0
CI fixes
pablogsal Dec 8, 2025
f368890
Fix C-API calls
pablogsal Dec 8, 2025
93f7abd
CSS fixes for classes and dark mode
savannahostrowski Dec 9, 2025
dc127bb
Merge pull request #110 from savannahostrowski/tachyon-opcodes-savannah
pablogsal Dec 9, 2025
43a298b
address review
pablogsal Dec 9, 2025
a4685fd
Merge branch 'main' into tachyon-opcodes
pablogsal Dec 9, 2025
b13b6f0
Docs
pablogsal Dec 9, 2025
50f63d0
Make bytecode spacer
pablogsal Dec 9, 2025
6eed927
Update Lib/profiling/sampling/_heatmap_assets/heatmap.js
pablogsal Dec 9, 2025
1c630ce
Update heatmap.css
pablogsal Dec 9, 2025
56e68c1
Tachyon Heatmap responsive styles
savannahostrowski Dec 10, 2025
e010870
Merge pull request #113 from savannahostrowski/heatmap-responsive
pablogsal Dec 10, 2025
ede0f79
Update Doc/library/profiling.sampling.rst
pablogsal Dec 10, 2025
f838c5d
Fix shift when selecting
StanFromIreland Dec 10, 2025
13ecd61
Merge pull request #114 from StanFromIreland/tachyon-opcodes-jump
pablogsal Dec 10, 2025
66610ff
Use any for stdlib line numbers
pablogsal Dec 10, 2025
aab1f3c
Update profiling.sampling.rst
pablogsal Dec 11, 2025
947f555
Docs
pablogsal Dec 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions Doc/library/profiling.sampling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,10 @@ Generate a line-by-line heatmap::

python -m profiling.sampling run --heatmap script.py

Enable opcode-level profiling to see which bytecode instructions are executing::

python -m profiling.sampling run --opcodes --flamegraph script.py


Commands
========
Expand Down Expand Up @@ -379,6 +383,44 @@ see substantial time in ``<GC>`` frames, consider investigating object
allocation rates or using object pooling.


Opcode-aware profiling
----------------------

The ``--opcodes`` option enables instruction-level profiling that captures
which Python bytecode instructions are executing at each sample::

python -m profiling.sampling run --opcodes --flamegraph script.py

This feature provides visibility into Python's bytecode execution, including
adaptive specialization optimizations. When a generic instruction like
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if it's worth linking back to https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-pep659, in case people want to learn more about the specializing adaptive interpreter?

``LOAD_ATTR`` is specialized at runtime into a more efficient variant like
``LOAD_ATTR_INSTANCE_VALUE``, the profiler shows both the specialized name
and the base instruction.

Opcode information appears in several output formats:

- **Live mode**: An opcode panel shows instruction-level statistics for the
selected function, accessible via keyboard navigation
- **Flame graphs**: Nodes display opcode information when available, helping
identify which instructions consume the most time
- **Heatmap**: Expandable bytecode panels per source line show instruction
breakdown with specialization percentages
- **Gecko format**: Opcode transitions are emitted as interval markers in the
Firefox Profiler timeline

This level of detail is particularly useful for:

- Understanding the performance impact of Python's adaptive specialization
- Identifying hot bytecode instructions that might benefit from optimization
- Analyzing the effectiveness of different code patterns at the instruction level
- Debugging performance issues that occur at the bytecode level

The ``--opcodes`` option is compatible with ``--live``, ``--flamegraph``,
``--heatmap``, and ``--gecko`` formats. It requires additional memory to store
opcode information and may slightly reduce sampling performance, but provides
unprecedented visibility into Python's execution model.


Real-time statistics
--------------------

Expand Down Expand Up @@ -760,6 +802,11 @@ and thread status statistics (GIL held percentage, CPU usage, GC time). The
main table shows function statistics with the currently sorted column indicated
by an arrow (▼).

When ``--opcodes`` is enabled, an additional opcode panel appears below the
main table, showing instruction-level statistics for the currently selected
function. This panel displays which bytecode instructions are executing most
frequently, including specialized variants and their base opcodes.


Keyboard commands
-----------------
Expand Down Expand Up @@ -813,6 +860,11 @@ Within live mode, keyboard commands control the display:
:kbd:`h` or :kbd:`?`
Show the help screen with all available commands.

:kbd:`j` / :kbd:`k`
Navigate through opcode entries in the opcode panel (when ``--opcodes`` is
enabled). These keys scroll through the instruction-level statistics for the
currently selected function.

When profiling finishes (duration expires or target process exits), the display
shows a "PROFILING COMPLETE" banner and freezes the final results. You can
still navigate, sort, and filter the results before pressing :kbd:`q` to exit.
Expand Down Expand Up @@ -939,6 +991,13 @@ Sampling options

Enable async-aware profiling for asyncio programs.

.. option:: --opcodes

Gather bytecode opcode information for instruction-level profiling. Shows
which bytecode instructions are executing, including specializations.
Compatible with ``--live``, ``--flamegraph``, ``--heatmap``, and ``--gecko``
formats only.


Mode options
------------
Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_global_objects_fini_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Include/internal/pycore_global_strings.h
Original file line number Diff line number Diff line change
Expand Up @@ -660,6 +660,7 @@ struct _Py_global_strings {
STRUCT_FOR_ID(only_keys)
STRUCT_FOR_ID(oparg)
STRUCT_FOR_ID(opcode)
STRUCT_FOR_ID(opcodes)
STRUCT_FOR_ID(open)
STRUCT_FOR_ID(opener)
STRUCT_FOR_ID(operation)
Expand Down
1 change: 1 addition & 0 deletions Include/internal/pycore_runtime_init_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Include/internal/pycore_unicodeobject_generated.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

78 changes: 78 additions & 0 deletions Lib/profiling/sampling/_flamegraph_assets/flamegraph.css
Original file line number Diff line number Diff line change
Expand Up @@ -862,6 +862,84 @@ body.resizing-sidebar {
text-align: center;
}

/* --------------------------------------------------------------------------
Tooltip Bytecode/Opcode Section
-------------------------------------------------------------------------- */

.tooltip-opcodes {
margin-top: 16px;
padding-top: 12px;
border-top: 1px solid var(--border);
}

.tooltip-opcodes-title {
color: var(--accent);
font-size: 13px;
margin-bottom: 8px;
font-weight: 600;
}

.tooltip-opcodes-list {
background: var(--bg-tertiary);
border: 1px solid var(--border);
border-radius: 6px;
padding: 10px;
}

.tooltip-opcode-row {
display: grid;
grid-template-columns: 1fr 60px 60px;
gap: 8px;
align-items: center;
padding: 3px 0;
}

.tooltip-opcode-name {
font-family: var(--font-mono);
font-size: 11px;
color: var(--text-primary);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}

.tooltip-opcode-name.specialized {
color: var(--spec-high-text);
}

.tooltip-opcode-base-hint {
color: var(--text-muted);
font-size: 11px;
margin-left: 4px;
}

.tooltip-opcode-badge {
background: var(--spec-high);
color: white;
font-size: 9px;
padding: 1px 4px;
border-radius: 3px;
margin-left: 4px;
}

.tooltip-opcode-count {
text-align: right;
font-size: 11px;
color: var(--text-secondary);
}

.tooltip-opcode-bar {
background: var(--bg-secondary);
border-radius: 2px;
height: 8px;
overflow: hidden;
}

.tooltip-opcode-bar-fill {
background: linear-gradient(90deg, var(--python-blue), var(--python-blue-light));
height: 100%;
}

/* --------------------------------------------------------------------------
Responsive (Flamegraph-specific)
-------------------------------------------------------------------------- */
Expand Down
77 changes: 77 additions & 0 deletions Lib/profiling/sampling/_flamegraph_assets/flamegraph.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,32 @@ let currentThreadFilter = 'all';
// Heat colors are now defined in CSS variables (--heat-1 through --heat-8)
// and automatically switch with theme changes - no JS color arrays needed!

// Opcode mappings - loaded from embedded data (generated by Python)
let OPCODE_NAMES = {};
let DEOPT_MAP = {};

// Initialize opcode mappings from embedded data
function initOpcodeMapping(data) {
if (data && data.opcode_mapping) {
OPCODE_NAMES = data.opcode_mapping.names || {};
DEOPT_MAP = data.opcode_mapping.deopt || {};
}
}

// Get opcode info from opcode number
function getOpcodeInfo(opcode) {
const opname = OPCODE_NAMES[opcode] || `<${opcode}>`;
const baseOpcode = DEOPT_MAP[opcode];
const isSpecialized = baseOpcode !== undefined;
const baseOpname = isSpecialized ? (OPCODE_NAMES[baseOpcode] || `<${baseOpcode}>`) : opname;

return {
opname: opname,
baseOpname: baseOpname,
isSpecialized: isSpecialized
};
}

// ============================================================================
// String Resolution
// ============================================================================
Expand Down Expand Up @@ -249,6 +275,53 @@ function createPythonTooltip(data) {
</div>`;
}

// Create bytecode/opcode section if available
let opcodeSection = "";
const opcodes = d.data.opcodes;
if (opcodes && typeof opcodes === 'object' && Object.keys(opcodes).length > 0) {
// Sort opcodes by sample count (descending)
const sortedOpcodes = Object.entries(opcodes)
.sort((a, b) => b[1] - a[1])
.slice(0, 8); // Limit to top 8

const totalOpcodeSamples = sortedOpcodes.reduce((sum, [, count]) => sum + count, 0);
const maxCount = sortedOpcodes[0][1] || 1;

const opcodeLines = sortedOpcodes.map(([opcode, count]) => {
const opcodeInfo = getOpcodeInfo(parseInt(opcode, 10));
const pct = ((count / totalOpcodeSamples) * 100).toFixed(1);
const barWidth = (count / maxCount) * 100;
const specializedBadge = opcodeInfo.isSpecialized
? '<span class="tooltip-opcode-badge">SPECIALIZED</span>'
: '';
const baseOpHint = opcodeInfo.isSpecialized
? `<span class="tooltip-opcode-base-hint">(${opcodeInfo.baseOpname})</span>`
: '';
const nameClass = opcodeInfo.isSpecialized
? 'tooltip-opcode-name specialized'
: 'tooltip-opcode-name';

return `
<div class="tooltip-opcode-row">
<div class="${nameClass}">
${opcodeInfo.opname}${baseOpHint}${specializedBadge}
</div>
<div class="tooltip-opcode-count">${count.toLocaleString()} (${pct}%)</div>
<div class="tooltip-opcode-bar">
<div class="tooltip-opcode-bar-fill" style="width: ${barWidth}%;"></div>
</div>
</div>`;
}).join('');

opcodeSection = `
<div class="tooltip-opcodes">
<div class="tooltip-opcodes-title">Bytecode Instructions:</div>
<div class="tooltip-opcodes-list">
${opcodeLines}
</div>
</div>`;
}

const fileLocationHTML = isSpecialFrame ? "" : `
<div class="tooltip-location">${filename}${d.data.lineno ? ":" + d.data.lineno : ""}</div>`;

Expand All @@ -275,6 +348,7 @@ function createPythonTooltip(data) {
` : ''}
</div>
${sourceSection}
${opcodeSection}
<div class="tooltip-hint">
${childCount > 0 ? "Click to zoom into this function" : "Leaf function - no children"}
</div>
Expand Down Expand Up @@ -994,6 +1068,9 @@ function initFlamegraph() {
processedData = resolveStringIndices(EMBEDDED_DATA);
}

// Initialize opcode mapping from embedded data
initOpcodeMapping(EMBEDDED_DATA);

originalData = processedData;
initThreadFilter(processedData);

Expand Down
Loading
Loading