-
-
Notifications
You must be signed in to change notification settings - Fork 33.7k
gh-138122: Make the tachyon profiler opcode-aware #142394
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
dd27e5e
Extend RemoteUnwinder to capture precise bytecode locations
pablogsal 70f2ae0
Add opcode utilities and --opcodes CLI flag
pablogsal aedc000
Track opcode sample counts in flamegraph collector
pablogsal 19ff11b
Emit opcode interval markers in Gecko collector
pablogsal af27d23
Add bytecode panel to heatmap visualization
pablogsal 7ffe4cb
Add opcode panel to live profiler TUI
pablogsal 8b423df
Update tests for location tuple and opcode field
pablogsal 8129e3d
Merge remote-tracking branch 'upstream/main' into tachyon-opcodes
pablogsal 965f521
Better test
pablogsal dac78a5
Merge remote-tracking branch 'upstream/main' into tachyon-opcodes
pablogsal 12c02f6
Add news entry
pablogsal c10628a
CI fixes
pablogsal 04563f0
CI fixes
pablogsal f368890
Fix C-API calls
pablogsal 93f7abd
CSS fixes for classes and dark mode
savannahostrowski dc127bb
Merge pull request #110 from savannahostrowski/tachyon-opcodes-savannah
pablogsal 43a298b
address review
pablogsal a4685fd
Merge branch 'main' into tachyon-opcodes
pablogsal b13b6f0
Docs
pablogsal 50f63d0
Make bytecode spacer
pablogsal 6eed927
Update Lib/profiling/sampling/_heatmap_assets/heatmap.js
pablogsal 1c630ce
Update heatmap.css
pablogsal 56e68c1
Tachyon Heatmap responsive styles
savannahostrowski e010870
Merge pull request #113 from savannahostrowski/heatmap-responsive
pablogsal ede0f79
Update Doc/library/profiling.sampling.rst
pablogsal f838c5d
Fix shift when selecting
StanFromIreland 13ecd61
Merge pull request #114 from StanFromIreland/tachyon-opcodes-jump
pablogsal 66610ff
Use any for stdlib line numbers
pablogsal aab1f3c
Update profiling.sampling.rst
pablogsal 947f555
Docs
pablogsal File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Docs
- Loading branch information
commit b13b6f09c115d8157d62ccb692746ae22abbd775
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -146,6 +146,10 @@ Generate a line-by-line heatmap:: | |
|
|
||
| python -m profiling.sampling run --heatmap script.py | ||
|
|
||
| Enable opcode-level profiling to see which bytecode instructions are executing:: | ||
|
|
||
| python -m profiling.sampling run --opcodes --flamegraph script.py | ||
|
|
||
|
|
||
| Commands | ||
| ======== | ||
|
|
@@ -379,6 +383,44 @@ see substantial time in ``<GC>`` frames, consider investigating object | |
| allocation rates or using object pooling. | ||
|
|
||
|
|
||
| Opcode-aware profiling | ||
| ---------------------- | ||
|
|
||
| The ``--opcodes`` option enables instruction-level profiling that captures | ||
| which Python bytecode instructions are executing at each sample:: | ||
|
|
||
| python -m profiling.sampling run --opcodes --flamegraph script.py | ||
|
|
||
| This feature provides visibility into Python's bytecode execution, including | ||
| adaptive specialization optimizations. When a generic instruction like | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Wondering if it's worth linking back to https://docs.python.org/3/whatsnew/3.11.html#whatsnew311-pep659, in case people want to learn more about the specializing adaptive interpreter? |
||
| ``LOAD_ATTR`` is specialized at runtime into a more efficient variant like | ||
| ``LOAD_ATTR_INSTANCE_VALUE``, the profiler shows both the specialized name | ||
| and the base instruction. | ||
|
|
||
| Opcode information appears in several output formats: | ||
|
|
||
| - **Live mode**: An opcode panel shows instruction-level statistics for the | ||
| selected function, accessible via keyboard navigation | ||
| - **Flame graphs**: Nodes display opcode information when available, helping | ||
| identify which instructions consume the most time | ||
| - **Heatmap**: Expandable bytecode panels per source line show instruction | ||
| breakdown with specialization percentages | ||
| - **Gecko format**: Opcode transitions are emitted as interval markers in the | ||
| Firefox Profiler timeline | ||
|
|
||
| This level of detail is particularly useful for: | ||
|
|
||
| - Understanding the performance impact of Python's adaptive specialization | ||
| - Identifying hot bytecode instructions that might benefit from optimization | ||
| - Analyzing the effectiveness of different code patterns at the instruction level | ||
| - Debugging performance issues that occur at the bytecode level | ||
|
|
||
| The ``--opcodes`` option is compatible with ``--live``, ``--flamegraph``, | ||
| ``--heatmap``, and ``--gecko`` formats. It requires additional memory to store | ||
| opcode information and may slightly reduce sampling performance, but provides | ||
| unprecedented visibility into Python's execution model. | ||
|
|
||
|
|
||
| Real-time statistics | ||
| -------------------- | ||
|
|
||
|
|
@@ -760,6 +802,11 @@ and thread status statistics (GIL held percentage, CPU usage, GC time). The | |
| main table shows function statistics with the currently sorted column indicated | ||
| by an arrow (▼). | ||
|
|
||
| When ``--opcodes`` is enabled, an additional opcode panel appears below the | ||
| main table, showing instruction-level statistics for the currently selected | ||
| function. This panel displays which bytecode instructions are executing most | ||
| frequently, including specialized variants and their base opcodes. | ||
|
|
||
|
|
||
| Keyboard commands | ||
| ----------------- | ||
|
|
@@ -813,6 +860,11 @@ Within live mode, keyboard commands control the display: | |
| :kbd:`h` or :kbd:`?` | ||
| Show the help screen with all available commands. | ||
|
|
||
| :kbd:`j` / :kbd:`k` | ||
pablogsal marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| Navigate through opcode entries in the opcode panel (when ``--opcodes`` is | ||
| enabled). These keys scroll through the instruction-level statistics for the | ||
| currently selected function. | ||
|
|
||
| When profiling finishes (duration expires or target process exits), the display | ||
| shows a "PROFILING COMPLETE" banner and freezes the final results. You can | ||
| still navigate, sort, and filter the results before pressing :kbd:`q` to exit. | ||
|
|
@@ -939,6 +991,13 @@ Sampling options | |
|
|
||
| Enable async-aware profiling for asyncio programs. | ||
|
|
||
| .. option:: --opcodes | ||
|
|
||
| Gather bytecode opcode information for instruction-level profiling. Shows | ||
| which bytecode instructions are executing, including specializations. | ||
| Compatible with ``--live``, ``--flamegraph``, ``--heatmap``, and ``--gecko`` | ||
| formats only. | ||
|
|
||
|
|
||
| Mode options | ||
| ------------ | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.