Skip to content

gh-138122: Don't sample partial frame chains #141912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pablogsal merged 10 commits into from
Dec 7, 2025
Merged

gh-138122: Don't sample partial frame chains #141912

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
7209306
gh-138122: Add incomplete sample detection and fix generator frame un…
pablogsal Nov 24, 2025
111e70c
Implement mark's idea
pablogsal Nov 25, 2025
664d439
Add note to the docs
pablogsal Nov 25, 2025
29a04b7
Fix frame termination
pablogsal Nov 25, 2025
3d9c294
Fix assignment
pablogsal Nov 25, 2025
e14dc8e
ReuseFRAME_OWNED_BY_INTERPRETER
pablogsal Nov 27, 2025
9986d88
Merge upstream/main into gh-138122-fix
pablogsal Dec 6, 2025
13bd7e4
Fix thread cache test
pablogsal Dec 7, 2025
a01bd24
Merge upstream/main into gh-138122-fix
pablogsal Dec 7, 2025
c9ab431
Fix warnings
pablogsal Dec 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add note to the docs
  • Loading branch information
@pablogsal
pablogsal committed Nov 25, 2025
commit 664d43943d80fb8b74baaee99bbcbac4ad2a78bf
29 changes: 29 additions & 0 deletions InternalDocs/frames.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,35 @@ The shim frame points to a special code object containing the `INTERPRETER_EXIT`
instruction which cleans up the shim frame and returns.


### Base frame

Each thread state contains an embedded `_PyInterpreterFrame` called the "base frame"
that serves as a sentinel at the bottom of the frame stack. This frame is allocated
in `_PyThreadStateImpl` (the internal extension of `PyThreadState`) and initialized
when the thread state is created. The `owner` field is set to `FRAME_OWNED_BY_THREAD_STATE`.

External profilers and sampling tools can validate that they have successfully unwound
the complete call stack by checking that the frame chain terminates at the base frame.
The `PyThreadState.base_frame` pointer provides the expected address to compare against.
If a stack walk doesn't reach this frame, the sample is incomplete (possibly due to a
race condition) and should be discarded.

The base frame is embedded in `_PyThreadStateImpl` rather than `PyThreadState` because
`_PyInterpreterFrame` is defined in internal headers that cannot be exposed in the
public API. A pointer (`PyThreadState.base_frame`) is provided for profilers to access
the address without needing internal headers.

See the initialization in `new_threadstate()` in [Python/pystate.c](../Python/pystate.c).

#### How profilers should use the base frame

External profilers should read `tstate->base_frame` before walking the stack, then
walk from `tstate->current_frame` following `frame->previous` pointers until reaching
a frame with `owner == FRAME_OWNED_BY_THREAD_STATE`. After the walk, verify that the
last frame address matches `base_frame`. If not, discard the sample as incomplete
since the frame chain may have been in an inconsistent state due to concurrent updates.


### The Instruction Pointer

`_PyInterpreterFrame` has two fields which are used to maintain the instruction
Expand Down
Loading