Update test_sys from v3.14.3 and impl more sys module#7075
Update test_sys from v3.14.3 and impl more sys module#7075youknowone wants to merge 12 commits intoRustPython:mainfrom
Conversation
📝 WalkthroughWalkthroughAdds a runtime monitoring subsystem (sys.monitoring) with per-code-object events and tool callbacks; refactors frame storage to use non-owning FramePtr and per-thread ThreadSlot with atomic exception tracking; enhances codegen/IR line-marker handling; adds codec fallbacks and various VM/frame lifecycle and API adjustments. Changes
Sequence Diagram(s)sequenceDiagram
participant VM as VirtualMachine
participant Frame as Frame Execution
participant Monitor as sys.monitoring
participant Callback as Tool Callback
VM->>Frame: execute instruction / change line
Frame->>Monitor: fire_line / fire_instruction / fire_call / fire_return
Monitor->>Monitor: check global & local masks, per-code disable
Note right of Monitor: collect active tool callbacks
Monitor->>Callback: invoke callbacks with (code, offset, info)
Callback-->>Monitor: callback result
Monitor-->>VM: continue execution
sequenceDiagram
participant Thread as ThreadSlot
participant VM as VirtualMachine
participant FramePtr as FramePtr
VM->>Thread: init_thread_slot_if_needed
Thread->>FramePtr: push(frame)
VM->>Thread: update_thread_exception
VM->>FramePtr: current_frame() -> FramePtr.as_ref()
FramePtr-->>VM: &Py<Frame>
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts (beta)
No actionable comments were generated in the recent review. 🎉 Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Code has been automatically formatted The code in this PR has been formatted using:
git pull origin sys |
71af7b0 to
33b66f9
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/vm/src/frame.rs (1)
788-829:⚠️ Potential issue | 🟠 MajorHandle monitoring callback errors in
gen_throwconsistently with other code paths.In the
runfunction, monitoring callback errors fromfire_raiseandfire_py_unwindreplace the original exception. However, ingen_throw, errors fromfire_py_throw,fire_raise, andfire_py_unwindare silently ignored withlet _, creating an inconsistency. Apply the same error-handling pattern to all monitoring callbacks here.
🤖 Fix all issues with AI agents
In `@crates/vm/src/builtins/frame.rs`:
- Around line 197-235: Extract the duplicated pointer-comparison logic in f_back
into a small helper (e.g., find_frame_by_ptr or frame_matches_ptr) that takes an
iterator/collection of &PyRef<Frame> (or a closure to produce such an iterator)
and the previous pointer from previous_frame(), performs the pointer equality
using the correct deref chain (&***f) and returns Option<PyRef<Frame>>; then
call this helper for vm.frames.borrow().iter() and for each
slot.frames.lock().iter() inside the threading block, replacing the duplicated
closures (do not use &****f).
In `@crates/vm/src/stdlib/io.rs`:
- Around line 2619-2679: The tuple validation in
StatelessIncrementalEncoder::encode and StatelessIncrementalDecoder::decode
relies on try_into_value(), which produces different TypeError messages than the
codec helpers; update both methods to perform the same downcast + length check
used in crate::codecs::{encode, decode} (or call those helpers) so non-tuple
inputs and wrong-length tuples produce the identical CPython-compatible error
text. Concretely, replace res.try_into_value(vm)? with
res.downcast::<PyTupleRef>(vm).map_err(|_| vm.new_type_error("encoder must
return a tuple (object, integer)")) (and the corresponding decoder message for
decode) and then check tuple.len() exactly as in the codec helpers to keep
messages and behavior consistent.
In `@crates/vm/src/stdlib/sys/monitoring.rs`:
- Around line 371-378: The is_disable function currently checks only the class
name and .is_none(), which treats both DISABLE and MISSING (both
MonitoringSentinel/_Sentinel) as disable; change it to perform an identity
comparison against the actual DISABLE singleton instead: obtain the DISABLE
object (either by fetching the module-level DISABLE singleton or by caching it
on MonitoringState like missing) and return obj.is(disable_obj) (or
pointer/identity equality) rather than comparing class name; update is_disable
to use that cached/obtained DISABLE symbol so MISSING is not misclassified.
In `@crates/vm/src/vm/python_run.rs`:
- Around line 47-62: cache_source_for_traceback currently builds the `lines`
list via split_inclusive('\n') which can leave the final line without a trailing
newline; update the logic in cache_source_for_traceback to post-process the
collected `lines` (before converting to Py objects) and ensure the last element
ends with '\n' (append one if missing) so the resulting tuple `entry` matches
CPython's linecache invariant `(size, mtime=None, lines, path)` with every line
ending in newline; keep the existing creation of `entry` and call to
cache.set_item as-is.
🧹 Nitpick comments (8)
crates/derive-impl/src/pymodule.rs (1)
233-243: Generated submodule init code panics on failure with no diagnostic context.The four
.unwrap()calls in the generated block will panic at runtime without any indication of which submodule failed or why. While the rest of the file also uses.unwrap()for__module_set_attr, those are single-attribute operations; here you have an entire module lifecycle (create_module→__init_methods→module_exec→set_attr) that could fail at multiple stages.Consider using
.expect(...)with the submodule name to improve debuggability:Suggested improvement
submodule_inits.push(quote! { #(`#cfgs`)* { let child_def = `#mod_ident`::module_def(ctx); - let child = child_def.create_module(vm).unwrap(); - child.__init_methods(vm).unwrap(); - `#mod_ident`::module_exec(vm, &child).unwrap(); + let child = child_def.create_module(vm) + .expect(concat!("failed to create submodule ", `#py_name`)); + child.__init_methods(vm) + .expect(concat!("failed to init methods for submodule ", `#py_name`)); + `#mod_ident`::module_exec(vm, &child) + .expect(concat!("failed to exec submodule ", `#py_name`)); let child: ::rustpython_vm::PyObjectRef = child.into(); vm.__module_set_attr(module, ctx.intern_str(`#py_name`), child).unwrap(); } });crates/codegen/src/ir.rs (1)
247-273: Second NOP-removal pass looks correct; consider inlining auseforHashSet.The two-pass approach makes sense:
remove_nops()(line 195) handles pre-existing NOPs before optimization passes, while this pass handles NOPs freshly created byconvert_pseudo_opsandlabel_exception_targets. The set-based deduplication logic is sound — a NOP is kept only when no real instruction covers its line and no other NOP for that line was already retained.Minor nit:
std::collections::HashSetis spelled out three times. A localusewould reduce noise.♻️ Optional: local import for readability
+ use std::collections::HashSet; // Collect lines that have non-NOP instructions in this block - let non_nop_lines: std::collections::HashSet<_> = block + let non_nop_lines: HashSet<_> = block .instructions .iter() .filter(|ins| !matches!(ins.instr.real(), Some(Instruction::Nop))) .map(|ins| ins.location.line) .collect(); - let mut kept_nop_lines: std::collections::HashSet<OneIndexed> = - std::collections::HashSet::new(); + let mut kept_nop_lines: HashSet<OneIndexed> = HashSet::new();crates/vm/src/stdlib/thread.rs (1)
890-895: Avoid holding the registry lock while locking per-thread frames.
thread_frames.lock()is held whileslot.frames.lock()is acquired for each entry. This creates lock contention and prevents thread registration/cleanup during the snapshot. The faulthandler.rs timeout handler demonstrates the correct pattern: clone slot references under the registry lock, drop the registry lock, then lock individual slot frames.♻️ Suggested refactor (drop registry lock before per-slot locks)
- let registry = vm.state.thread_frames.lock(); - registry - .iter() - .filter_map(|(id, slot)| slot.frames.lock().last().cloned().map(|f| (*id, f))) - .collect() + let registry = vm.state.thread_frames.lock(); + let slots: Vec<_> = registry.iter().map(|(id, slot)| (*id, slot.clone())).collect(); + drop(registry); + slots + .into_iter() + .filter_map(|(id, slot)| slot.frames.lock().last().cloned().map(|f| (id, f))) + .collect()crates/vm/src/frame.rs (1)
466-503: Sharedprev_linebetween tracing and monitoring may suppress first Line event after mid-framesettrace()if still on the same line.The code unconditionally updates
prev_lineat line 502 outside themonitoring_maskcheck. While this is intentional (per the comment on line 484 to prevent spurious LINE events when monitoring is enabled mid-function), it means ifsys.settrace()is called mid-frame while still on the same line, the firstLineevent will not fire until the line changes. This design prioritizes shared state efficiency over system independence. Clarify whether this matches CPython 3.12+ behavior; if stricter separation is needed, trackprev_trace_lineindependently for tracing while retainingprev_linefor monitoring's mid-function safety.crates/vm/src/stdlib/sys/monitoring.rs (4)
430-451:FIRINGguard is not panic-safe — use a RAII drop guard instead.If a Rust panic occurs inside the callback loop (lines 432–448), line 450 is never reached and
FIRINGstaystruefor the thread permanently, silently suppressing all future monitoring events on that thread.Proposed fix using a drop guard
+struct FiringGuard; +impl FiringGuard { + fn new() -> Self { + FIRING.with(|f| f.set(true)); + Self + } +} +impl Drop for FiringGuard { + fn drop(&mut self) { + FIRING.with(|f| f.set(false)); + } +} + // ... inside fire_event_inner: - FIRING.with(|f| f.set(true)); - let result = (|| { + let _guard = FiringGuard::new(); + let result = (|| { for (tool, cb) in callbacks { // ... } Ok(()) })(); - FIRING.with(|f| f.set(false)); result
265-275: Replace magic event-ID literals with derived constants.
18,8,9correspond to the bit positions ofBRANCH,BRANCH_LEFT, andBRANCH_RIGHTrespectively, but bare literals are fragile and will silently break if bit assignments change.Proposed fix
You could define constants at the module level:
const BRANCH_EVENT_ID: usize = MonitoringEvents::BRANCH.bits().trailing_zeros() as usize; const BRANCH_LEFT_EVENT_ID: usize = MonitoringEvents::BRANCH_LEFT.bits().trailing_zeros() as usize; const BRANCH_RIGHT_EVENT_ID: usize = MonitoringEvents::BRANCH_RIGHT.bits().trailing_zeros() as usize;Then replace all magic numbers in
register_callbackand thefire_*functions accordingly.
356-356: Remove decorative section separator.As per coding guidelines, "Do not add decorative section separators (e.g.
// -----------,// ===,/* *** */). Use///doc-comments or short//comments only when they add value".
457-726: Consider a macro to reduce boilerplate acrossfire_*functions.All 17
fire_*functions follow the same pattern: extractcode_id, buildFuncArgs, callfire_event_inner. The differences are the event ID, event bit constant, and the argument list. A declarative macro could eliminate the repetition while keeping each variant's distinct arguments clear.This is a readability/maintainability suggestion — no functional concern.
| pub fn f_back(&self, vm: &VirtualMachine) -> Option<PyRef<Frame>> { | ||
| // TODO: actually store f_back inside Frame struct | ||
| let previous = self.previous_frame(); | ||
| if previous.is_null() { | ||
| return None; | ||
| } | ||
|
|
||
| // get the frame in the frame stack that appears before this one. | ||
| // won't work if this frame isn't in the frame stack, hence the todo above | ||
| vm.frames | ||
| if let Some(frame) = vm | ||
| .frames | ||
| .borrow() | ||
| .iter() | ||
| .rev() | ||
| .skip_while(|p| !p.is(self.as_object())) | ||
| .nth(1) | ||
| .find(|f| { | ||
| let ptr: *const Frame = &****f; | ||
| core::ptr::eq(ptr, previous) | ||
| }) | ||
| .cloned() | ||
| { | ||
| return Some(frame); | ||
| } | ||
|
|
||
| #[cfg(feature = "threading")] | ||
| { | ||
| let registry = vm.state.thread_frames.lock(); | ||
| for slot in registry.values() { | ||
| if let Some(frame) = slot | ||
| .frames | ||
| .lock() | ||
| .iter() | ||
| .find(|f| { | ||
| let ptr: *const Frame = &****f; | ||
| core::ptr::eq(ptr, previous) | ||
| }) | ||
| .cloned() | ||
| { | ||
| return Some(frame); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| None |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check previous_frame() return type
echo "=== previous_frame() signature ==="
rg -n 'fn previous_frame' crates/vm/src/builtins/frame.rs -C 5
# Check Frame struct definition and Deref impls
echo "=== Frame struct definition ==="
rg -n 'pub struct Frame' crates/vm/src/builtins/frame.rs -C 3
# Check for Deref impl on Frame
echo "=== Deref implementations for Frame ==="
rg -n 'impl.*Deref.*Frame' crates/vm/src/builtins/frame.rs -C 3
# Check vm.frames type
echo "=== vm.frames field definition ==="
rg -n 'frames:' crates/vm/src/vm.rs -C 2
# Check the exact types involved - get PyRef definition
echo "=== Check FrameRef type alias ==="
rg -n 'type FrameRef' crates/vm/src/builtins/frame.rs -C 1Repository: RustPython/RustPython
Length of output: 317
🏁 Script executed:
#!/bin/bash
# List the top-level structure
echo "=== Repository structure ==="
git ls-files | head -30
# Find files named frame.rs or vm.rs
echo "=== Finding frame.rs ==="
find . -name "frame.rs" -type f 2>/dev/null
echo "=== Finding vm.rs ==="
find . -name "vm.rs" -type f 2>/dev/null
# Check crates directory structure
echo "=== Crates directory ==="
ls -la crates/ 2>/dev/null | head -20Repository: RustPython/RustPython
Length of output: 2216
🏁 Script executed:
#!/bin/bash
# Examine the frame.rs file around the relevant lines
echo "=== previous_frame() and surrounding code in frame.rs ==="
sed -n '1,50p' crates/vm/src/builtins/frame.rs | head -40
echo "=== Frame struct definition ==="
rg -n 'pub struct Frame' crates/vm/src/builtins/frame.rs -A 15
echo "=== FrameRef type alias ==="
rg -n 'FrameRef\|type.*Frame' crates/vm/src/builtins/frame.rs | head -10
echo "=== previous_frame method ==="
rg -n 'fn previous_frame' crates/vm/src/builtins/frame.rs -A 3Repository: RustPython/RustPython
Length of output: 1197
🏁 Script executed:
#!/bin/bash
# Check size of frame.rs
wc -l crates/vm/src/frame.rs
# Search for previous_frame in frame.rs
echo "=== previous_frame in crates/vm/src/frame.rs ==="
rg -n 'fn previous_frame' crates/vm/src/frame.rs -A 5
# Search for Frame struct definition
echo "=== Frame struct in crates/vm/src/frame.rs ==="
rg -n 'pub struct Frame' crates/vm/src/frame.rs -A 20Repository: RustPython/RustPython
Length of output: 1200
🏁 Script executed:
#!/bin/bash
# Search for FrameRef type definition
echo "=== FrameRef definition in frame.rs ==="
rg -n 'type FrameRef\|pub type FrameRef' crates/vm/src/frame.rs
# Check if Frame implements Deref
echo "=== Deref impl for Frame ==="
rg -n 'impl.*Deref.*for Frame' crates/vm/src/frame.rs -A 5
# Find vm.frames definition (search in lib.rs or vm.rs equivalent)
echo "=== Looking for VirtualMachine struct and frames field ==="
fd -t f -n 'lib.rs' crates/vm/src/ | head -5
# Check if there's a VirtualMachine definition
rg -n 'pub struct VirtualMachine' crates/vm/src/ -A 30 | head -50Repository: RustPython/RustPython
Length of output: 2689
🏁 Script executed:
#!/bin/bash
# Search for FrameRef definition
echo "=== FrameRef type definition ==="
rg -n 'FrameRef' crates/vm/src/frame.rs | grep -E 'type|pub' | head -5
# More specific search
echo "=== Searching in frame.rs for FrameRef ==="
grep -n 'FrameRef' crates/vm/src/frame.rs | head -10
# Search across entire vm crate
echo "=== FrameRef definition anywhere in vm ==="
rg -n '^pub type FrameRef' crates/vm/src/Repository: RustPython/RustPython
Length of output: 487
🏁 Script executed:
#!/bin/bash
# Now that we know FrameRef = PyRef<Frame>, let's verify the deref chain
# Check if Frame implements Deref
echo "=== All impl blocks for Frame ==="
rg -n 'impl.*Frame' crates/vm/src/frame.rs | head -20
# Get the full deref chain visualization - read around the f_back method
echo "=== The f_back method implementation ==="
sed -n '197,240p' crates/vm/src/builtins/frame.rsRepository: RustPython/RustPython
Length of output: 1574
Extract the frame-matching logic to a helper to avoid duplication, and fix the deref depth.
The pointer-comparison closure is duplicated between the vm.frames search (lines 208–210) and the thread-frames search (lines 225–227). Per coding guidelines, extract the differing iterator source and reuse the comparison logic.
Additionally, the deref chain &****f appears incorrect. Given f: &PyRef<Frame> from the iterator:
*f→&PyRef<Frame>**f→&Py<Frame>(via PyRef::Deref)***f→&Frame(via Py::Deref)
The correct pointer should be &***f, not &****f. Unless Frame implements Deref, the fourth dereference is invalid.
🤖 Prompt for AI Agents
In `@crates/vm/src/builtins/frame.rs` around lines 197 - 235, Extract the
duplicated pointer-comparison logic in f_back into a small helper (e.g.,
find_frame_by_ptr or frame_matches_ptr) that takes an iterator/collection of
&PyRef<Frame> (or a closure to produce such an iterator) and the previous
pointer from previous_frame(), performs the pointer equality using the correct
deref chain (&***f) and returns Option<PyRef<Frame>>; then call this helper for
vm.frames.borrow().iter() and for each slot.frames.lock().iter() inside the
threading block, replacing the duplicated closures (do not use &****f).
| fn encode( | ||
| &self, | ||
| input: PyObjectRef, | ||
| _final: OptionalArg<bool>, | ||
| vm: &VirtualMachine, | ||
| ) -> PyResult { | ||
| let mut args: Vec<PyObjectRef> = vec![input]; | ||
| if let Some(errors) = &self.errors { | ||
| args.push(errors.to_owned().into()); | ||
| } | ||
| let res = self.encode.call(args, vm)?; | ||
| let tuple: PyTupleRef = res.try_into_value(vm)?; | ||
| if tuple.len() != 2 { | ||
| return Err(vm.new_type_error("encoder must return a tuple (object, integer)")); | ||
| } | ||
| Ok(tuple[0].clone()) | ||
| } | ||
|
|
||
| #[pymethod] | ||
| fn reset(&self) {} | ||
|
|
||
| #[pymethod] | ||
| fn setstate(&self, _state: PyObjectRef) {} | ||
|
|
||
| #[pymethod] | ||
| fn getstate(&self, vm: &VirtualMachine) -> PyObjectRef { | ||
| vm.ctx.new_int(0).into() | ||
| } | ||
|
|
||
| #[pygetset] | ||
| fn name(&self) -> Option<PyStrRef> { | ||
| self.name.clone() | ||
| } | ||
| } | ||
|
|
||
| #[pyclass(module = "_io", name, no_attr)] | ||
| #[derive(Debug, PyPayload)] | ||
| struct StatelessIncrementalDecoder { | ||
| decode: PyObjectRef, | ||
| errors: Option<PyStrRef>, | ||
| } | ||
|
|
||
| #[pyclass] | ||
| impl StatelessIncrementalDecoder { | ||
| #[pymethod] | ||
| fn decode( | ||
| &self, | ||
| input: PyObjectRef, | ||
| _final: OptionalArg<bool>, | ||
| vm: &VirtualMachine, | ||
| ) -> PyResult { | ||
| let mut args: Vec<PyObjectRef> = vec![input]; | ||
| if let Some(errors) = &self.errors { | ||
| args.push(errors.to_owned().into()); | ||
| } | ||
| let res = self.decode.call(args, vm)?; | ||
| let tuple: PyTupleRef = res.try_into_value(vm)?; | ||
| if tuple.len() != 2 { | ||
| return Err(vm.new_type_error("decoder must return a tuple (object, integer)")); | ||
| } | ||
| Ok(tuple[0].clone()) |
There was a problem hiding this comment.
Align stateless encode/decode tuple validation with codecs.rs.
try_into_value() yields different TypeError messages for non-tuples, and the decoder message currently differs from the codec helper. To preserve CPython-compatible errors and keep behavior consistent with crate::codecs::{encode, decode}, mirror the same downcast/len check (or reuse those helpers).
🔧 Suggested fix (consistent tuple validation + error messages)
- let tuple: PyTupleRef = res.try_into_value(vm)?;
- if tuple.len() != 2 {
- return Err(vm.new_type_error("encoder must return a tuple (object, integer)"));
- }
- Ok(tuple[0].clone())
+ let tuple = res
+ .downcast::<PyTuple>()
+ .ok()
+ .filter(|t| t.len() == 2)
+ .ok_or_else(|| vm.new_type_error("encoder must return a tuple (object, integer)"))?;
+ Ok(tuple[0].clone())- let tuple: PyTupleRef = res.try_into_value(vm)?;
- if tuple.len() != 2 {
- return Err(vm.new_type_error("decoder must return a tuple (object, integer)"));
- }
- Ok(tuple[0].clone())
+ let tuple = res
+ .downcast::<PyTuple>()
+ .ok()
+ .filter(|t| t.len() == 2)
+ .ok_or_else(|| vm.new_type_error("decoder must return a tuple (object,integer)"))?;
+ Ok(tuple[0].clone())📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| fn encode( | |
| &self, | |
| input: PyObjectRef, | |
| _final: OptionalArg<bool>, | |
| vm: &VirtualMachine, | |
| ) -> PyResult { | |
| let mut args: Vec<PyObjectRef> = vec![input]; | |
| if let Some(errors) = &self.errors { | |
| args.push(errors.to_owned().into()); | |
| } | |
| let res = self.encode.call(args, vm)?; | |
| let tuple: PyTupleRef = res.try_into_value(vm)?; | |
| if tuple.len() != 2 { | |
| return Err(vm.new_type_error("encoder must return a tuple (object, integer)")); | |
| } | |
| Ok(tuple[0].clone()) | |
| } | |
| #[pymethod] | |
| fn reset(&self) {} | |
| #[pymethod] | |
| fn setstate(&self, _state: PyObjectRef) {} | |
| #[pymethod] | |
| fn getstate(&self, vm: &VirtualMachine) -> PyObjectRef { | |
| vm.ctx.new_int(0).into() | |
| } | |
| #[pygetset] | |
| fn name(&self) -> Option<PyStrRef> { | |
| self.name.clone() | |
| } | |
| } | |
| #[pyclass(module = "_io", name, no_attr)] | |
| #[derive(Debug, PyPayload)] | |
| struct StatelessIncrementalDecoder { | |
| decode: PyObjectRef, | |
| errors: Option<PyStrRef>, | |
| } | |
| #[pyclass] | |
| impl StatelessIncrementalDecoder { | |
| #[pymethod] | |
| fn decode( | |
| &self, | |
| input: PyObjectRef, | |
| _final: OptionalArg<bool>, | |
| vm: &VirtualMachine, | |
| ) -> PyResult { | |
| let mut args: Vec<PyObjectRef> = vec![input]; | |
| if let Some(errors) = &self.errors { | |
| args.push(errors.to_owned().into()); | |
| } | |
| let res = self.decode.call(args, vm)?; | |
| let tuple: PyTupleRef = res.try_into_value(vm)?; | |
| if tuple.len() != 2 { | |
| return Err(vm.new_type_error("decoder must return a tuple (object, integer)")); | |
| } | |
| Ok(tuple[0].clone()) | |
| fn encode( | |
| &self, | |
| input: PyObjectRef, | |
| _final: OptionalArg<bool>, | |
| vm: &VirtualMachine, | |
| ) -> PyResult { | |
| let mut args: Vec<PyObjectRef> = vec![input]; | |
| if let Some(errors) = &self.errors { | |
| args.push(errors.to_owned().into()); | |
| } | |
| let res = self.encode.call(args, vm)?; | |
| let tuple = res | |
| .downcast::<PyTuple>() | |
| .ok() | |
| .filter(|t| t.len() == 2) | |
| .ok_or_else(|| vm.new_type_error("encoder must return a tuple (object, integer)"))?; | |
| Ok(tuple[0].clone()) | |
| } | |
| #[pymethod] | |
| fn reset(&self) {} | |
| #[pymethod] | |
| fn setstate(&self, _state: PyObjectRef) {} | |
| #[pymethod] | |
| fn getstate(&self, vm: &VirtualMachine) -> PyObjectRef { | |
| vm.ctx.new_int(0).into() | |
| } | |
| #[pygetset] | |
| fn name(&self) -> Option<PyStrRef> { | |
| self.name.clone() | |
| } | |
| } | |
| #[pyclass(module = "_io", name, no_attr)] | |
| #[derive(Debug, PyPayload)] | |
| struct StatelessIncrementalDecoder { | |
| decode: PyObjectRef, | |
| errors: Option<PyStrRef>, | |
| } | |
| #[pyclass] | |
| impl StatelessIncrementalDecoder { | |
| #[pymethod] | |
| fn decode( | |
| &self, | |
| input: PyObjectRef, | |
| _final: OptionalArg<bool>, | |
| vm: &VirtualMachine, | |
| ) -> PyResult { | |
| let mut args: Vec<PyObjectRef> = vec![input]; | |
| if let Some(errors) = &self.errors { | |
| args.push(errors.to_owned().into()); | |
| } | |
| let res = self.decode.call(args, vm)?; | |
| let tuple = res | |
| .downcast::<PyTuple>() | |
| .ok() | |
| .filter(|t| t.len() == 2) | |
| .ok_or_else(|| vm.new_type_error("decoder must return a tuple (object, integer)"))?; | |
| Ok(tuple[0].clone()) |
🤖 Prompt for AI Agents
In `@crates/vm/src/stdlib/io.rs` around lines 2619 - 2679, The tuple validation in
StatelessIncrementalEncoder::encode and StatelessIncrementalDecoder::decode
relies on try_into_value(), which produces different TypeError messages than the
codec helpers; update both methods to perform the same downcast + length check
used in crate::codecs::{encode, decode} (or call those helpers) so non-tuple
inputs and wrong-length tuples produce the identical CPython-compatible error
text. Concretely, replace res.try_into_value(vm)? with
res.downcast::<PyTupleRef>(vm).map_err(|_| vm.new_type_error("encoder must
return a tuple (object, integer)")) (and the corresponding decoder message for
decode) and then check tuple.len() exactly as in the codec helpers to keep
messages and behavior consistent.
| fn set_events(tool_id: i32, event_set: i32, vm: &VirtualMachine) -> PyResult<()> { | ||
| let tool = check_valid_tool(tool_id, vm)?; | ||
| check_tool_in_use(tool, vm)?; | ||
| let normalized = normalize_event_set(event_set, false, vm)?; | ||
| let mut state = vm.state.monitoring.lock(); | ||
| state.global_events[tool] = normalized; | ||
| update_events_mask(vm, &state); | ||
| Ok(()) | ||
| } |
There was a problem hiding this comment.
TOCTOU race: check_tool_in_use releases the lock before set_events re-acquires it.
check_tool_in_use (line 288) locks, verifies the tool is registered, and unlocks. Then line 290 locks again to mutate. Between the two lock acquisitions another thread can call free_tool_id, removing the tool. The same pattern exists in set_local_events (lines 319–323) and free_tool_id (lines 236–237, where clear_tool_id locks/unlocks before the second lock).
Fold the check into the critical section so validation and mutation happen under a single lock hold.
Sketch for set_events
fn set_events(tool_id: i32, event_set: i32, vm: &VirtualMachine) -> PyResult<()> {
let tool = check_valid_tool(tool_id, vm)?;
- check_tool_in_use(tool, vm)?;
let normalized = normalize_event_set(event_set, false, vm)?;
let mut state = vm.state.monitoring.lock();
+ if state.tool_names[tool].is_none() {
+ return Err(vm.new_value_error(format!("tool {tool} is not in use")));
+ }
state.global_events[tool] = normalized;
update_events_mask(vm, &state);
Ok(())
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| fn set_events(tool_id: i32, event_set: i32, vm: &VirtualMachine) -> PyResult<()> { | |
| let tool = check_valid_tool(tool_id, vm)?; | |
| check_tool_in_use(tool, vm)?; | |
| let normalized = normalize_event_set(event_set, false, vm)?; | |
| let mut state = vm.state.monitoring.lock(); | |
| state.global_events[tool] = normalized; | |
| update_events_mask(vm, &state); | |
| Ok(()) | |
| } | |
| fn set_events(tool_id: i32, event_set: i32, vm: &VirtualMachine) -> PyResult<()> { | |
| let tool = check_valid_tool(tool_id, vm)?; | |
| let normalized = normalize_event_set(event_set, false, vm)?; | |
| let mut state = vm.state.monitoring.lock(); | |
| if state.tool_names[tool].is_none() { | |
| return Err(vm.new_value_error(format!("tool {tool} is not in use"))); | |
| } | |
| state.global_events[tool] = normalized; | |
| update_events_mask(vm, &state); | |
| Ok(()) | |
| } |
| /// Check if DISABLE sentinel was returned by a callback. | ||
| fn is_disable(obj: &PyObjectRef, vm: &VirtualMachine) -> bool { | ||
| // DISABLE is the _Sentinel singleton stored on the monitoring module. | ||
| // We check its type name to avoid needing a reference to the exact object. | ||
| let name = obj.class().name(); | ||
| let name_str: &str = &name; | ||
| name_str == "_Sentinel" && !vm.is_none(obj) | ||
| } |
There was a problem hiding this comment.
is_disable cannot distinguish DISABLE from MISSING — both are _Sentinel instances.
Both DISABLE (line 749) and MISSING (line 754) are instances of MonitoringSentinel with class name "_Sentinel", and neither is None. This function returns true for both sentinels, so a callback that returns MISSING will be misinterpreted as DISABLE, silently disabling monitoring for that code location.
CPython uses identity comparison against the specific DISABLE singleton. You should do the same here — compare by object identity against the known DISABLE object rather than by type name.
Proposed fix
-/// Check if DISABLE sentinel was returned by a callback.
-fn is_disable(obj: &PyObjectRef, vm: &VirtualMachine) -> bool {
- // DISABLE is the _Sentinel singleton stored on the monitoring module.
- // We check its type name to avoid needing a reference to the exact object.
- let name = obj.class().name();
- let name_str: &str = &name;
- name_str == "_Sentinel" && !vm.is_none(obj)
+/// Check if DISABLE sentinel was returned by a callback.
+fn is_disable(obj: &PyObjectRef, vm: &VirtualMachine) -> bool {
+ // Compare by identity against the actual DISABLE singleton.
+ if let Ok(monitoring_mod) = vm.import("sys.monitoring", 0) {
+ if let Ok(disable_obj) = monitoring_mod.get_attr("DISABLE", vm) {
+ return obj.is(&disable_obj);
+ }
+ }
+ false
}Alternatively, cache the DISABLE object in MonitoringState (similar to how missing is cached) for a cheaper check:
pub struct MonitoringState {
pub tool_names: [Option<String>; TOOL_LIMIT],
pub global_events: [u32; TOOL_LIMIT],
pub local_events: HashMap<(usize, usize), u32>,
pub callbacks: HashMap<(usize, usize), PyObjectRef>,
pub disabled: HashSet<(usize, usize, usize)>,
pub missing: Option<PyObjectRef>,
+ pub disable: Option<PyObjectRef>,
}Then compare obj.is(disable_obj) in is_disable.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /// Check if DISABLE sentinel was returned by a callback. | |
| fn is_disable(obj: &PyObjectRef, vm: &VirtualMachine) -> bool { | |
| // DISABLE is the _Sentinel singleton stored on the monitoring module. | |
| // We check its type name to avoid needing a reference to the exact object. | |
| let name = obj.class().name(); | |
| let name_str: &str = &name; | |
| name_str == "_Sentinel" && !vm.is_none(obj) | |
| } | |
| /// Check if DISABLE sentinel was returned by a callback. | |
| fn is_disable(obj: &PyObjectRef, vm: &VirtualMachine) -> bool { | |
| // Compare by identity against the actual DISABLE singleton. | |
| if let Ok(monitoring_mod) = vm.import("sys.monitoring", 0) { | |
| if let Ok(disable_obj) = monitoring_mod.get_attr("DISABLE", vm) { | |
| return obj.is(&disable_obj); | |
| } | |
| } | |
| false | |
| } |
🤖 Prompt for AI Agents
In `@crates/vm/src/stdlib/sys/monitoring.rs` around lines 371 - 378, The
is_disable function currently checks only the class name and .is_none(), which
treats both DISABLE and MISSING (both MonitoringSentinel/_Sentinel) as disable;
change it to perform an identity comparison against the actual DISABLE singleton
instead: obtain the DISABLE object (either by fetching the module-level DISABLE
singleton or by caching it on MonitoringState like missing) and return
obj.is(disable_obj) (or pointer/identity equality) rather than comparing class
name; update is_disable to use that cached/obtained DISABLE symbol so MISSING is
not misclassified.
| fn cache_source_for_traceback(&self, source_path: &str, source: &str) -> PyResult<()> { | ||
| let linecache = self.import("linecache", 0)?; | ||
| let cache = linecache.get_attr("cache", self)?; | ||
| let lines: Vec<PyObjectRef> = source | ||
| .split_inclusive('\n') | ||
| .map(|line| self.ctx.new_str(line).into()) | ||
| .collect(); | ||
| let entry = self.ctx.new_tuple(vec![ | ||
| self.ctx.new_int(source.len()).into(), | ||
| self.ctx.none(), | ||
| self.ctx.new_list(lines).into(), | ||
| self.ctx.new_str(source_path).into(), | ||
| ]); | ||
| cache.set_item(source_path, entry.into(), self)?; | ||
| Ok(()) | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# First, let's find linecache-related Python files to understand the format
fd 'linecache' --type f -e py 2>/dev/null | head -20Repository: RustPython/RustPython
Length of output: 108
🌐 Web query:
CPython linecache entry tuple format mtime None
💡 Result:
In CPython’s linecache, an eager/loaded cache entry is a 4-tuple:
(size, mtime, lines, fullname) — i.e., file size, modification time, list of source lines, and the resolved full path. The module-level comment documents this tuple format. [1]
mtime can be None when the source did not come from a normal filesystem file (e.g., it came via an import __loader__ / get_source). In that case, checkcache() explicitly treats the entry as a no-op (it skips os.stat() staleness checks) because there’s no meaningful on-disk mtime to compare against. [1][2]
Separately, linecache.cache[filename] can also temporarily hold a 1-tuple used for lazy loading (a thunk/function to fetch the lines later), which is why code sometimes checks len(entry) == 1. [2]
Sources:
[1] Debian’s packaged CPython Lib/linecache.py excerpt showing the cache tuple and mtime is None handling. (sources.debian.org)
[2] Stack Overflow explanation of linecache cache entry shapes (1-tuple lazy thunk vs 4-tuple) and why loader-based entries lack mtime. (stackoverflow.com)
Citations:
- 1: https://sources.debian.org/src/python3.14/3.14.0~rc1-1/Lib/linecache.py?utm_source=openai
- 2: https://stackoverflow.com/questions/51057882/why-does-linecache-check-for-the-length-of-the-tuple-elements-in-the-cache
🏁 Script executed:
#!/bin/bash
# Check the actual implementation in the code file
cat -n crates/vm/src/vm/python_run.rs | head -80Repository: RustPython/RustPython
Length of output: 3552
🏁 Script executed:
#!/bin/bash
# Search for linecache usage elsewhere in RustPython to understand expectations
rg 'linecache' --type rs -B2 -A2 | head -100Repository: RustPython/RustPython
Length of output: 91
🏁 Script executed:
#!/bin/bash
# Examine CPython's linecache.py to see the cache entry format
head -200 Lib/linecache.py | grep -A 20 -B 5 "cache\["Repository: RustPython/RustPython
Length of output: 1187
🏁 Script executed:
#!/bin/bash
# Check specifically how linecache handles the cache tuple
rg 'def.*cache|cache.*=.*\(' Lib/linecache.py -A 3 | head -80Repository: RustPython/RustPython
Length of output: 847
RustPython's linecache entry tuple format is correct, but trailing newline handling differs from CPython.
The tuple (size, mtime=None, lines, path) matches CPython's expectations. However, CPython's linecache explicitly ensures each line ends with '\n': if not lines[-1].endswith('\n'): lines[-1] += '\n'. RustPython's split_inclusive('\n') approach does not add trailing newlines when the source doesn't end with one, meaning the last line may lack a '\n'. This could affect traceback display for the final line. Add post-processing to match CPython's guarantee that all lines in the list end with a newline character.
🤖 Prompt for AI Agents
In `@crates/vm/src/vm/python_run.rs` around lines 47 - 62,
cache_source_for_traceback currently builds the `lines` list via
split_inclusive('\n') which can leave the final line without a trailing newline;
update the logic in cache_source_for_traceback to post-process the collected
`lines` (before converting to Py objects) and ensure the last element ends with
'\n' (append one if missing) so the resulting tuple `entry` matches CPython's
linecache invariant `(size, mtime=None, lines, path)` with every line ending in
newline; keep the existing creation of `entry` and call to cache.set_item as-is.
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Fix all issues with AI agents
In `@crates/codegen/src/compile.rs`:
- Around line 5186-5201: Capture the for-statement's source range at the start
of compile_for (e.g., store self.current_source_range into a local like
for_range) and replace uses of iter.range() used for setting attribution with
that for_range; specifically update the calls to
self.set_source_range(iter.range()) around the loop epilogue and implicit return
attribution to use for_range so LINE events remain tied to the for-statement
header rather than the iterable expression.
In `@crates/vm/src/frame.rs`:
- Around line 590-619: The current block replaces the original exception with
any error returned from monitoring::fire_reraise/fire_raise and other sites like
fire_py_unwind; change this so that only RAISE/RERAISE events replace the
propagated exception, but PY_UNWIND (and similar unwind events) preserve the
original exception on monitoring callback failure—i.e., call
monitoring::fire_py_unwind (or other unwind callers) and if it returns Err,
ignore or log the monitoring error and return the original exception instead of
monitor_exc. Update the logic around monitoring::EVENT_PY_UNWIND and the
corresponding match/Err handling (referencing fire_reraise, fire_raise,
fire_py_unwind, and monitoring::EVENT_* masks) to implement this differentiated
behavior.
- Around line 788-803: In gen_throw, don't silently discard
monitoring::fire_py_throw and monitoring::fire_raise results; capture their
Result and, on Err, propagate/replace the current exception just like the run()
loop does—i.e., call monitoring::fire_py_throw(vm, self.code, offset, &exc_obj)
and monitoring::fire_raise(vm, self.code, offset, &exc_obj), check the Result,
and if it returns Err(e) return or propagate that error (or replace the
propagating exception) so monitoring errors are not swallowed; update the
gen_throw logic accordingly to mirror run()'s error-handling behavior.
In `@crates/vm/src/stdlib/sys/monitoring.rs`:
- Around line 430-451: The FIRING TLS flag is set before running callbacks but
is reset only after the closure returns, so a panic inside the callback loop
will skip FIRING.with(|f| f.set(false)) and leave monitoring disabled for that
thread; wrap the FIRING set/reset in a panic-safe RAII guard (or use the
scopeguard crate) so that FIRING.with(|f| f.set(false)) runs in Drop even if a
callback panics: set FIRING to true, create a guard type or scopeguard that
calls FIRING.with(|f| f.set(false)) in its Drop, then run the existing callbacks
loop (the code using callbacks, cb.call(...), is unchanged) and let the guard
reset FIRING regardless of success/failure before returning the result and
before modifying vm.state.monitoring.lock().
- Around line 696-726: The doc comments for fire_branch_left and
fire_branch_right are reversed; update the comments so fire_branch_left
documents "fired when a branch is not taken (falls through)" and
fire_branch_right documents "fired when a branch is taken (jumps to
destination)" to match how frame.rs invokes these functions (reference
fire_branch_left and fire_branch_right to locate them and adjust their
docstrings accordingly).
🧹 Nitpick comments (11)
crates/derive-impl/src/pymodule.rs (1)
197-248: Submodule initialization looks correct; consider a minor robustness note on theNameValuecase.The overall logic — detect nested
#[pymodule]mods, skipsubmodules, generate cfg-guarded init code — is clean and consistent with the rest of the file.One small observation: at Line 213,
syn::Meta::NameValuesilently returnsOk(()), meaning a malformed#[pymodule = "..."]annotation would be silently ignored rather than producing a compile-time diagnostic. This is unlikely in practice but could cause confusion if someone misuses the attribute.Optional: emit an error for the NameValue case
- _ => return Ok(()), + syn::Meta::NameValue(nv) => { + bail_span!(nv, "#[pymodule] does not support `=` syntax"); + }crates/vm/src/stdlib/sys/mod.rs (1)
1072-1079: Inconsistency with non-threading_current_framesstub.The non-threading
_current_exceptionsreturns a dict with key0and the actual exception, while the non-threading_current_frames(line 1084) returns an empty dict. Consider aligning them — either both return meaningful data for the main thread or both return an empty dict. Currently CPython returns both with the main thread's data.crates/vm/src/vm/mod.rs (1)
1109-1155:resume_gen_framelacksscopeguardforrecursion_depthdecrement.If
f(frame)panics,recursion_depthat line 1153 won't be decremented, and the exception stack / frame chain won't be cleaned up either (lines 1141–1151). While panics from Python execution are rare,with_frame_excat least protects the recursion depth viawith_recursion'sscopeguard::defer!. Consider wrapping the cleanup in ascopeguardor a drop guard for consistency.Proposed fix — use scopeguard for cleanup
self.recursion_depth.update(|d| d + 1); self.frames.borrow_mut().push(frame.clone()); let frame_ptr: *const Frame = &***frame; let old_frame = crate::vm::thread::set_current_frame(frame_ptr); frame.previous.store( old_frame as *mut Frame, core::sync::atomic::Ordering::Relaxed, ); - // Inline exception push without thread exception update self.exceptions.borrow_mut().stack.push(exc); let old_owner = frame.owner.swap( crate::frame::FrameOwner::Thread as i8, core::sync::atomic::Ordering::AcqRel, ); let result = f(frame); - // SAFETY: frame_ptr is valid because self.frames holds a clone - unsafe { &*frame_ptr } - .owner - .store(old_owner, core::sync::atomic::Ordering::Release); - // Inline exception pop without thread exception update - self.exceptions - .borrow_mut() - .stack - .pop() - .expect("pop_exception() without nested exc stack"); - crate::vm::thread::set_current_frame(old_frame); - let _popped = self.frames.borrow_mut().pop(); - - self.recursion_depth.update(|d| d - 1); + scopeguard::defer! { + // SAFETY: frame_ptr is valid because self.frames holds a clone + unsafe { &*frame_ptr } + .owner + .store(old_owner, core::sync::atomic::Ordering::Release); + self.exceptions + .borrow_mut() + .stack + .pop() + .expect("pop_exception() without nested exc stack"); + crate::vm::thread::set_current_frame(old_frame); + let _popped = self.frames.borrow_mut().pop(); + self.recursion_depth.update(|d| d - 1); + } + resultcrates/vm/src/frame.rs (3)
2021-2056: Duplicated branch monitoring code — extract into a helper.The exact same branch-monitoring pattern (compute
src_offset/dest_offset, check mask, firebranch_left/branch_rightdepending onbranch_taken) appears in:
PopJumpIfNone(Lines 2029-2056)PopJumpIfNotNone(Lines 2067-2094)pop_jump_if(Lines 3109-3136)execute_for_iter(Lines 3150-3164, 3188-3202)This violates DRY — extract a small helper like
fire_branch_event(&self, vm, src_offset, dest_offset, branch_taken).♻️ Sketch of helper
/// Fire BRANCH_LEFT or BRANCH_RIGHT monitoring event. fn fire_branch_monitoring( &self, vm: &VirtualMachine, src_offset: u32, dest_offset: u32, branch_taken: bool, ) { let monitoring_mask = vm.state.monitoring_events.load(); if monitoring_mask & (crate::stdlib::sys::monitoring::EVENT_BRANCH_LEFT | crate::stdlib::sys::monitoring::EVENT_BRANCH_RIGHT) != 0 { if branch_taken { let _ = crate::stdlib::sys::monitoring::fire_branch_right( vm, self.code, src_offset, dest_offset, ); } else { let _ = crate::stdlib::sys::monitoring::fire_branch_left( vm, self.code, src_offset, dest_offset, ); } } }Then each call site becomes a single line:
-// Fire BRANCH monitoring events -let monitoring_mask = vm.state.monitoring_events.load(); -if monitoring_mask - & (crate::stdlib::sys::monitoring::EVENT_BRANCH_LEFT - | crate::stdlib::sys::monitoring::EVENT_BRANCH_RIGHT) - != 0 -{ - let dest_offset = if branch_taken { target.0 * 2 } else { self.lasti() * 2 }; - if branch_taken { - let _ = crate::stdlib::sys::monitoring::fire_branch_right( - vm, self.code, src_offset, dest_offset, - ); - } else { - let _ = crate::stdlib::sys::monitoring::fire_branch_left( - vm, self.code, src_offset, dest_offset, - ); - } -} +let dest_offset = if branch_taken { target.0 * 2 } else { self.lasti() * 2 }; +self.fire_branch_monitoring(vm, src_offset, dest_offset, branch_taken);As per coding guidelines: "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."
Also applies to: 2059-2094, 3096-3137
2908-2966:execute_callmonitoring:is_python_callcheck is shallow.Line 2930 uses
callable.downcast_ref::<PyFunction>().is_some()to decide whether to fireC_RETURN/C_RAISE. This misses cases where aPyFunctionis wrapped (e.g., bound methods,functools.partial, descriptors). A bound method wrapping a Python function would still be classified as a "C call" and fireC_RETURN/C_RAISEevents incorrectly. CPython checksPy_TYPE(callable) == &PyFunction_Typewhich is similarly shallow, so this may be acceptable for now, but it's worth noting.
2929-2944:call_arg0computation always allocates the MISSING sentinel when no args present, even when CALL monitoring is disabled.The
MISSINGsentinel is only fetched whenmonitoring_mask & EVENT_CALL != 0(Line 2933), so the guard is correct. However,get_missing(vm)acquires the monitoring lock every time. For the hot path (CALL with no args but monitoring active), consider caching MISSING at VM init time to avoid repeated lock acquisition.crates/vm/src/vm/thread.rs (2)
133-142:get_all_current_exceptionsholds the registry lock while cloning all exceptions.Line 137 acquires
vm.state.thread_frames.lock()and then iterates all slots callingslot.exception.to_owned()(Line 140). Ifto_owned()involves cloningPyObjectRef(ref-count bump), this holds the registry lock for O(threads) clone operations. For typical thread counts this is fine, but consider collecting just theArc<ThreadSlot>references first, then releasing the lock before reading exceptions, to minimize contention.
15-21:ThreadSlotfields arepubbut the struct itself should probably restrict visibility.Both
framesandexceptionarepub, allowing any code to directly lock/swap them. Consider restricting topub(crate)to match the existing visibility pattern in this module (e.g.,pub(crate) static CURRENT_FRAME).crates/vm/src/stdlib/sys/monitoring.rs (3)
248-278: Magic event IDs: hardcoded18,8,9for BRANCH/BRANCH_LEFT/BRANCH_RIGHT.Lines 265, 267, 268, 272-274 use raw numeric event IDs. These should use named constants derived from the bitflags to prevent silent breakage if event positions change.
♻️ Proposed fix
+// Event ID constants (bit position = trailing_zeros of the corresponding bit) +const EVENT_ID_BRANCH_LEFT: usize = MonitoringEvents::BRANCH_LEFT.bits().trailing_zeros() as usize; +const EVENT_ID_BRANCH_RIGHT: usize = MonitoringEvents::BRANCH_RIGHT.bits().trailing_zeros() as usize; +const EVENT_ID_BRANCH: usize = MonitoringEvents::BRANCH.bits().trailing_zeros() as usize; // In register_callback: - if event_id == 18 { - // BRANCH → BRANCH_LEFT (8) + BRANCH_RIGHT (9) - state.callbacks.insert((tool, 8), func.clone()); - state.callbacks.insert((tool, 9), func); + if event_id == EVENT_ID_BRANCH { + state.callbacks.insert((tool, EVENT_ID_BRANCH_LEFT), func.clone()); + state.callbacks.insert((tool, EVENT_ID_BRANCH_RIGHT), func); }
356-356: Remove decorative section separator.Line 356
// ── Event dispatch ──...is a decorative separator. As per coding guidelines: "Do not add decorative section separators (e.g.// -----------,// ===,/* *** */). Use///doc-comments or short//comments only when they add value."
84-107:MonitoringStateusesHashMapforlocal_events,callbacks, anddisabled— considerFxHashMapfor better performance.These maps are accessed under the monitoring lock on every event dispatch (
fire_event_inner, Lines 407-418). The standardHashMapuses SipHash which is slower thanFxHashMapfor small integer keys(usize, usize). Since this is a runtime VM, usingrustc_hash::FxHashMap(or similar) could reduce overhead when monitoring is active.
| // END_FOR + POP_ITER are on the `for` line, not the body's last line | ||
| let saved_range = self.current_source_range; | ||
| self.set_source_range(iter.range()); | ||
| emit!(self, Instruction::EndFor); | ||
| emit!(self, Instruction::PopIter); | ||
| self.set_source_range(saved_range); | ||
| } | ||
| self.compile_statements(orelse)?; | ||
|
|
||
| self.switch_to_block(after_block); | ||
|
|
||
| // Restore source range to the `for` line so any implicit return | ||
| // (LOAD_CONST None, RETURN_VALUE) is attributed to the `for` line, | ||
| // not the loop body's last line. | ||
| self.set_source_range(iter.range()); | ||
|
|
There was a problem hiding this comment.
Use the for statement range instead of iter.range() for line attribution.
For multi-line headers (for x in (\n y\n):), iter.range() can move LINE events to the iterable’s line. Capturing the statement range once (before compiling the loop) preserves correct attribution.
💡 Suggested fix
- let saved_range = self.current_source_range;
- self.set_source_range(iter.range());
+ let saved_range = self.current_source_range;
+ self.set_source_range(for_range);
emit!(self, Instruction::EndFor);
emit!(self, Instruction::PopIter);
self.set_source_range(saved_range);- self.set_source_range(iter.range());
+ self.set_source_range(for_range);// Add near the start of compile_for (before compiling iter/body):
let for_range = self.current_source_range;🤖 Prompt for AI Agents
In `@crates/codegen/src/compile.rs` around lines 5186 - 5201, Capture the
for-statement's source range at the start of compile_for (e.g., store
self.current_source_range into a local like for_range) and replace uses of
iter.range() used for setting attribution with that for_range; specifically
update the calls to self.set_source_range(iter.range()) around the loop epilogue
and implicit return attribution to use for_range so LINE events remain tied to
the for-statement header rather than the iterable expression.
| // Fire RAISE or RERAISE monitoring event. | ||
| // fire_reraise internally deduplicates: only the first | ||
| // re-raise after each EXCEPTION_HANDLED fires the event. | ||
| // If the callback raises (e.g. ValueError for illegal DISABLE), | ||
| // replace the original exception. | ||
| let exception = { | ||
| use crate::stdlib::sys::monitoring; | ||
| let monitoring_mask = vm.state.monitoring_events.load(); | ||
| if is_reraise { | ||
| if monitoring_mask & monitoring::EVENT_RERAISE != 0 { | ||
| let offset = idx as u32 * 2; | ||
| let exc_obj: PyObjectRef = exception.clone().into(); | ||
| match monitoring::fire_reraise(vm, self.code, offset, &exc_obj) { | ||
| Ok(()) => exception, | ||
| Err(monitor_exc) => monitor_exc, | ||
| } | ||
| } else { | ||
| exception | ||
| } | ||
| } else if monitoring_mask & monitoring::EVENT_RAISE != 0 { | ||
| let offset = idx as u32 * 2; | ||
| let exc_obj: PyObjectRef = exception.clone().into(); | ||
| match monitoring::fire_raise(vm, self.code, offset, &exc_obj) { | ||
| Ok(()) => exception, | ||
| Err(monitor_exc) => monitor_exc, | ||
| } | ||
| } else { | ||
| exception | ||
| } | ||
| }; |
There was a problem hiding this comment.
Inconsistent error handling: fire_reraise/fire_raise replace the original exception, but fire_py_unwind (and other sites) do the same — verify this is intentional.
When monitoring callbacks raise, you replace the original exception with the monitoring exception (Lines 604, 614, 636). This is reasonable for fire_raise/fire_reraise (the callback error becomes the propagated exception), but doing the same for fire_py_unwind means a buggy monitoring callback can silently swallow the real exception and replace it with its own. CPython's _Py_call_instrumentation_exc typically restores the original exception if the callback fails for unwind events.
Consider whether replacing the exception for PY_UNWIND is the correct behavior, or if the original exception should be preserved.
Also applies to: 624-640
🤖 Prompt for AI Agents
In `@crates/vm/src/frame.rs` around lines 590 - 619, The current block replaces
the original exception with any error returned from
monitoring::fire_reraise/fire_raise and other sites like fire_py_unwind; change
this so that only RAISE/RERAISE events replace the propagated exception, but
PY_UNWIND (and similar unwind events) preserve the original exception on
monitoring callback failure—i.e., call monitoring::fire_py_unwind (or other
unwind callers) and if it returns Err, ignore or log the monitoring error and
return the original exception instead of monitor_exc. Update the logic around
monitoring::EVENT_PY_UNWIND and the corresponding match/Err handling
(referencing fire_reraise, fire_raise, fire_py_unwind, and monitoring::EVENT_*
masks) to implement this differentiated behavior.
| // Fire PY_THROW event before raising the exception in the generator | ||
| { | ||
| use crate::stdlib::sys::monitoring; | ||
| let monitoring_mask = vm.state.monitoring_events.load(); | ||
| if monitoring_mask & monitoring::EVENT_PY_THROW != 0 { | ||
| let offset = idx as u32 * 2; | ||
| let exc_obj: PyObjectRef = exception.clone().into(); | ||
| let _ = monitoring::fire_py_throw(vm, self.code, offset, &exc_obj); | ||
| } | ||
| // Fire RAISE: the thrown exception is raised in this frame | ||
| if monitoring_mask & monitoring::EVENT_RAISE != 0 { | ||
| let offset = idx as u32 * 2; | ||
| let exc_obj: PyObjectRef = exception.clone().into(); | ||
| let _ = monitoring::fire_raise(vm, self.code, offset, &exc_obj); | ||
| } | ||
| } |
There was a problem hiding this comment.
Silently discarding monitoring errors in gen_throw can hide callback bugs.
fire_py_throw and fire_raise results are discarded with let _ = (Lines 795, 801). If a monitoring callback intentionally raises (e.g., ValueError to signal illegal DISABLE), that error is lost. Compare with the run() loop where fire_raise errors replace the propagating exception (Line 612-614). The inconsistency means the same callback behaves differently depending on the code path.
At minimum, consider matching the behavior in the run() loop where monitoring errors replace the exception.
🤖 Prompt for AI Agents
In `@crates/vm/src/frame.rs` around lines 788 - 803, In gen_throw, don't silently
discard monitoring::fire_py_throw and monitoring::fire_raise results; capture
their Result and, on Err, propagate/replace the current exception just like the
run() loop does—i.e., call monitoring::fire_py_throw(vm, self.code, offset,
&exc_obj) and monitoring::fire_raise(vm, self.code, offset, &exc_obj), check the
Result, and if it returns Err(e) return or propagate that error (or replace the
propagating exception) so monitoring errors are not swallowed; update the
gen_throw logic accordingly to mirror run()'s error-handling behavior.
| FIRING.with(|f| f.set(true)); | ||
| let result = (|| { | ||
| for (tool, cb) in callbacks { | ||
| let result = cb.call(args.clone(), vm)?; | ||
| if is_disable(&result, vm) { | ||
| // Only local events (event_id < LOCAL_EVENTS_COUNT) can be disabled. | ||
| // Non-local events (RAISE, EXCEPTION_HANDLED, PY_UNWIND, etc.) | ||
| // cannot be disabled per code object. | ||
| if event_id >= LOCAL_EVENTS_COUNT { | ||
| return Err(vm.new_value_error(format!( | ||
| "cannot disable {} events", | ||
| EVENT_NAMES[event_id] | ||
| ))); | ||
| } | ||
| let mut state = vm.state.monitoring.lock(); | ||
| state.disabled.insert((code_id, offset as usize, tool)); | ||
| } | ||
| } | ||
| Ok(()) | ||
| })(); | ||
| FIRING.with(|f| f.set(false)); | ||
| result |
There was a problem hiding this comment.
FIRING guard is not panic-safe — a panicking callback permanently disables monitoring for the thread.
If any callback at Line 433 panics (and the panic is caught by catch_unwind or similar), FIRING is never reset to false (Line 450 is skipped). All subsequent monitoring events on this thread will be silently suppressed due to the early return at Line 390-392.
Use a scopeguard or RAII wrapper to ensure cleanup:
🔒 Proposed fix using scopeguard pattern
FIRING.with(|f| f.set(true));
+ let _guard = scopeguard::guard((), |_| {
+ FIRING.with(|f| f.set(false));
+ });
let result = (|| {
for (tool, cb) in callbacks {
let result = cb.call(args.clone(), vm)?;
if is_disable(&result, vm) {
if event_id >= LOCAL_EVENTS_COUNT {
return Err(vm.new_value_error(format!(
"cannot disable {} events",
EVENT_NAMES[event_id]
)));
}
let mut state = vm.state.monitoring.lock();
state.disabled.insert((code_id, offset as usize, tool));
}
}
Ok(())
})();
- FIRING.with(|f| f.set(false));
result🤖 Prompt for AI Agents
In `@crates/vm/src/stdlib/sys/monitoring.rs` around lines 430 - 451, The FIRING
TLS flag is set before running callbacks but is reset only after the closure
returns, so a panic inside the callback loop will skip FIRING.with(|f|
f.set(false)) and leave monitoring disabled for that thread; wrap the FIRING
set/reset in a panic-safe RAII guard (or use the scopeguard crate) so that
FIRING.with(|f| f.set(false)) runs in Drop even if a callback panics: set FIRING
to true, create a guard type or scopeguard that calls FIRING.with(|f|
f.set(false)) in its Drop, then run the existing callbacks loop (the code using
callbacks, cb.call(...), is unchanged) and let the guard reset FIRING regardless
of success/failure before returning the result and before modifying
vm.state.monitoring.lock().
| /// BRANCH_LEFT: fired when a branch goes left (condition true) | ||
| pub fn fire_branch_left( | ||
| vm: &VirtualMachine, | ||
| code: &PyRef<PyCode>, | ||
| offset: u32, | ||
| destination: u32, | ||
| ) -> PyResult<()> { | ||
| let code_id = code.get_id(); | ||
| let args = FuncArgs::from(vec![ | ||
| code.clone().into(), | ||
| vm.ctx.new_int(offset).into(), | ||
| vm.ctx.new_int(destination).into(), | ||
| ]); | ||
| fire_event_inner(vm, 8, EVENT_BRANCH_LEFT, code_id, offset, &args) | ||
| } | ||
|
|
||
| /// BRANCH_RIGHT: fired when a branch goes right (condition false, falls through) | ||
| pub fn fire_branch_right( | ||
| vm: &VirtualMachine, | ||
| code: &PyRef<PyCode>, | ||
| offset: u32, | ||
| destination: u32, | ||
| ) -> PyResult<()> { | ||
| let code_id = code.get_id(); | ||
| let args = FuncArgs::from(vec![ | ||
| code.clone().into(), | ||
| vm.ctx.new_int(offset).into(), | ||
| vm.ctx.new_int(destination).into(), | ||
| ]); | ||
| fire_event_inner(vm, 9, EVENT_BRANCH_RIGHT, code_id, offset, &args) | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, find where fire_branch_left and fire_branch_right are called
fd -e rs -type f | xargs rg -n "fire_branch_(left|right)" --context=3Repository: RustPython/RustPython
Length of output: 4979
🏁 Script executed:
# Let's also check the frame.rs file specifically
fd frame.rs -type f | head -5Repository: RustPython/RustPython
Length of output: 235
🌐 Web query:
CPython 3.14 sys.monitoring BRANCH_LEFT BRANCH_RIGHT semantics documentation
💡 Result:
In CPython/Python 3.14, sys.monitoring.events.BRANCH_LEFT and BRANCH_RIGHT are local monitoring events emitted when a conditional branch is taken (i.e., control flow follows one of the two outcomes of a conditional). [1]
What “left” vs “right” means
- The VM labels the two outcomes as “left” and “right”, but does not guarantee which outcome corresponds to which label (e.g., “true” vs “false” is not specified). [1]
- The only guarantee is consistency for the duration of the program: once a given branch’s outcomes are assigned “left/right”, that assignment won’t flip mid-run. [1]
Callback signature / how to interpret it
For both BRANCH_LEFT and BRANCH_RIGHT, the callback is invoked as:
func(code: CodeType, instruction_offset: int, destination_offset: int) -> object [1]
instruction_offset: where the conditional branch decision happensdestination_offset: the next instruction offset that will execute (i.e., the target that was actually taken) [1]
Relationship to the old BRANCH event
- In Python 3.14,
BRANCHis deprecated, andBRANCH_LEFT/BRANCH_RIGHTare the intended replacement (and can be disabled independently for better performance). [1] - The 3.14 “What’s New” notes the addition of these two events and the deprecation of
BRANCH. [2]
Sources: Python 3.14 sys.monitoring docs [1], Python 3.14 “What’s New” [2].
Doc comments for fire_branch_left and fire_branch_right are inverted.
In frame.rs, fire_branch_right() is called when a branch is taken (jumps to destination), and fire_branch_left() is called when a branch is not taken (falls through). The current doc comments have these backwards:
-/// BRANCH_LEFT: fired when a branch goes left (condition true)
+/// BRANCH_LEFT: fired when a conditional branch is NOT taken (falls through)
pub fn fire_branch_left(
-/// BRANCH_RIGHT: fired when a branch goes right (condition false, falls through)
+/// BRANCH_RIGHT: fired when a conditional branch IS taken (jumps to target)
pub fn fire_branch_right(🤖 Prompt for AI Agents
In `@crates/vm/src/stdlib/sys/monitoring.rs` around lines 696 - 726, The doc
comments for fire_branch_left and fire_branch_right are reversed; update the
comments so fire_branch_left documents "fired when a branch is not taken (falls
through)" and fire_branch_right documents "fired when a branch is taken (jumps
to destination)" to match how frame.rs invokes these functions (reference
fire_branch_left and fire_branch_right to locate them and adjust their
docstrings accordingly).
- Add sys.call_tracing as func(*args) dispatch - Add sys._current_exceptions with per-thread exception tracking via thread_exceptions registry synced on push/pop/set_exception - Fix f_back to use previous_frame pointer and cross-thread lookup - Add module="sys" to float_info struct sequence - Fix sys.exit to unpack tuple args - Set default stdio_errors to surrogateescape - Remove noisy __del__ warning in object core
- Register latin-1/iso8859-1 codec aliases early for stdio bootstrap - Fix stdio_errors: use "strict" when encoding is explicitly set - Use traceback._print_exception_bltin in excepthook - Cache source in linecache for <string> tracebacks - Add StatelessIncrementalEncoder/Decoder to _io - Move sys.flags thread_inherit_context/context_aware_warnings to getset properties - Skip exception slot sync when no exceptions are active to avoid O(depth) topmost_exception() walk on every function call - Remove 5 expectedFailure markers in test_sys
- Rename submod -> child in pymodule.rs to fix cspell check - Update nested_double_async_with snapshot for codegen changes - Remove expectedFailure from test_line_number_implicit_return_after_async_for
- test_warnings, test_c_locale_coercion: remove stale expectedFailure - test_io: mark test_check_encoding_warning, test_newline_decoder - test_subprocess: mark test_encoding_warning - test_futures2: mark test_future_traceback - test_tasks: mark test_cancel_traceback_for_future_* - test_peepholer: mark test_constant_folding_remove_nop_location - test_linecache: mark test_linecache_python_string - test_sys: mark test_getwindowsversion tests - test_json: mark test_highly_nested_objects_encoding
- Remove ERROR_PROPAGATING thread-local: callback errors now propagate normally without suppressing all future events - Change DISABLE granularity from (code, tool, event) to (code, offset, tool) matching CPython's per-instruction remove_tools() - Add offset parameter to fire_event_inner and fire_line - Skip test_highly_nested_objects_encoding (timeout, not failure)
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
crates/vm/src/frame.rs (2)
466-502:⚠️ Potential issue | 🟠 MajorSeparate trace vs monitoring line tracking to avoid suppressing LINE events
prev_lineis updated in the tracing block before the monitoring block, so when tracing is enabled the monitoringEVENT_LINEcheck sees no change and never fires. Track trace and monitoring line numbers independently (or update ordering) so both subsystems get line-change notifications.💡 Suggested fix
- let mut prev_line: u32 = 0; + let mut prev_trace_line: u32 = 0; + let mut prev_monitor_line: u32 = 0; ... - && loc.line.get() as u32 != prev_line + && loc.line.get() as u32 != prev_trace_line { - prev_line = loc.line.get() as u32; + prev_trace_line = loc.line.get() as u32; vm.trace_event(crate::protocol::TraceEvent::Line, None)?; } ... - if monitoring_mask & monitoring::EVENT_LINE != 0 && line != prev_line { + if monitoring_mask & monitoring::EVENT_LINE != 0 && line != prev_monitor_line { monitoring::fire_line(vm, self.code, offset, line)?; } ... - prev_line = line; + prev_monitor_line = line; }
2909-2966:⚠️ Potential issue | 🟠 MajorC_RETURN/C_RAISE should fire even when EVENT_CALL is off
call_arg0is computed only whenEVENT_CALLis set, so C_RETURN/C_RAISE never fire unless EVENT_CALL is enabled. Also,fire_c_raiseerrors are discarded. Computearg0when any of CALL/C_RETURN/C_RAISE is enabled, and handlefire_c_raiseconsistently.💡 Suggested fix
- let monitoring_mask = vm.state.monitoring_events.load(); - let is_python_call = callable.downcast_ref::<PyFunction>().is_some(); - - // Compute arg0 once for CALL, C_RETURN, and C_RAISE events - let call_arg0 = if monitoring_mask & monitoring::EVENT_CALL != 0 { + let monitoring_mask = vm.state.monitoring_events.load(); + let is_python_call = callable.downcast_ref::<PyFunction>().is_some(); + let wants_call = monitoring_mask & monitoring::EVENT_CALL != 0; + let wants_c = monitoring_mask + & (monitoring::EVENT_C_RETURN | monitoring::EVENT_C_RAISE) + != 0; + + // Compute arg0 once for CALL / C_RETURN / C_RAISE + let call_arg0 = if wants_call || wants_c { let arg0 = final_args .args .first() .cloned() .unwrap_or_else(|| monitoring::get_missing(vm)); let offset = (self.lasti() - 1) * 2; - monitoring::fire_call(vm, self.code, offset, &callable, arg0.clone())?; + if wants_call { + monitoring::fire_call(vm, self.code, offset, &callable, arg0.clone())?; + } Some(arg0) } else { None }; ... - if let Some(arg0) = call_arg0 - && !is_python_call - { + if let Some(arg0) = call_arg0 + && !is_python_call + && wants_c + { let offset = (self.lasti() - 1) * 2; monitoring::fire_c_return(vm, self.code, offset, &callable, arg0)?; } ... - if let Some(arg0) = call_arg0 - && !is_python_call - { + if let Some(arg0) = call_arg0 + && !is_python_call + && wants_c + { let offset = (self.lasti() - 1) * 2; - let _ = monitoring::fire_c_raise(vm, self.code, offset, &callable, arg0); + monitoring::fire_c_raise(vm, self.code, offset, &callable, arg0)?; }
🤖 Fix all issues with AI agents
In `@crates/vm/src/frame.rs`:
- Around line 1407-1420: The monitoring call currently swallows errors when
firing jump events; update the block in function(s) using self.lasti(),
target.get(arg), self.jump(dest) so the call to
crate::stdlib::sys::monitoring::fire_jump returns its Result and any Err is
propagated instead of ignored—e.g., check vm.state.monitoring_events for
EVENT_JUMP as now but replace the ignored call with a propagating call (use the
? operator or map_err to convert/return the error) so failures from fire_jump
are not silently dropped; apply the same change to the other occurrences
referenced (around the blocks that call fire_* and inspect monitoring_events).
In `@crates/vm/src/vm/mod.rs`:
- Around line 1129-1176: resume_gen_frame currently performs manual cleanup
(recursion_depth.update decrement, frames.pop(), exceptions.stack.pop(),
owner.store(old_owner), crate::vm::thread::set_current_frame(old_frame)) after
calling f(frame), which will be skipped if f panics; wrap the post-call cleanup
in a panic-safe guard (either use scopeguard::defer! immediately after
successfully performing the setup steps or create a small RAII guard struct) so
that on unwind you always: decrement recursion_depth, pop self.frames, pop the
exceptions stack, restore frame.owner to old_owner, and reset the current thread
frame via crate::vm::thread::set_current_frame(old_frame); place the guard right
after pushing the FramePtr and pushing the exception so it has access to
old_owner and old_frame and remove the manual cleanup at the end or let the
guard perform it.
🧹 Nitpick comments (6)
crates/codegen/src/ir.rs (1)
253-273: Consider using an import forstd::collections::HashSet.
std::collections::HashSetis used three times inline (lines 254, 260, 261). A localuseor a module-level import would reduce verbosity.♻️ Suggested refactor
+ use std::collections::HashSet; + // Collect lines that have non-NOP instructions in this block - let non_nop_lines: std::collections::HashSet<_> = block + let non_nop_lines: HashSet<_> = block .instructions .iter() .filter(|ins| !matches!(ins.instr.real(), Some(Instruction::Nop))) .map(|ins| ins.location.line) .collect(); - let mut kept_nop_lines: std::collections::HashSet<OneIndexed> = - std::collections::HashSet::new(); + let mut kept_nop_lines: HashSet<OneIndexed> = HashSet::new();crates/vm/src/frame.rs (1)
2692-2707: EXCEPTION_HANDLED hook placement looks correctPlease also confirm fmt/clippy and the bytecode clean-build command were run since instruction handling changed.
Commands to run locally:
cargo fmt cargo clippy --all-targets --all-features rm -r target/debug/build/rustpython-* && find . | grep -E "\.pyc$" | xargs rm -rAs per coding guidelines: "Follow the default rustfmt code style by running
cargo fmtto format Rust code", "Always run clippy to lint Rust code (cargo clippy) before completing tasks and fix any warnings or lints that are introduced by your changes", and "When modifying bytecode instructions, perform a clean build by running:rm -r target/debug/build/rustpython-* && find . | grep -E "\.pyc$" | xargs rm -r".crates/vm/src/vm/mod.rs (1)
1148-1148: Triple deref(&***frame)is hard to follow — consider a helper or a brief inline comment.
frameis&FrameRef(&PyRef<Frame>), so&***framedereferences through&→PyRef<Frame>→Py<Frame>→Framethen re-borrows to&Frame. It works, but is a readability hazard. A named binding or a brief// &FrameRef → &Framecomment would help future readers.crates/vm/src/stdlib/sys/monitoring.rs (3)
456-726: Magic event-id literals scattered across allfire_*functions.Every
fire_*function passes a raw integer (e.g.,0,1,11,16) as theevent_idtofire_event_inner. A single typo would silently route callbacks to the wrong event with no compile-time protection. Consider defining named constants (e.g.,const EVENT_ID_PY_START: usize = 0;) or deriving the id from the bit position of the correspondingEVENT_*constant (EVENT_PY_START.trailing_zeros() as usize).Example using trailing_zeros
pub fn fire_py_start(vm: &VirtualMachine, code: &PyRef<PyCode>, offset: u32) -> PyResult<()> { let code_id = code.get_id(); let args = FuncArgs::from(vec![code.clone().into(), vm.ctx.new_int(offset).into()]); - fire_event_inner(vm, 0, EVENT_PY_START, code_id, offset, &args) + fire_event_inner(vm, EVENT_PY_START.trailing_zeros() as usize, EVENT_PY_START, code_id, offset, &args) }Or, have
fire_event_innerderiveevent_idfromevent_bitinternally, eliminating the redundant parameter.
234-240:free_tool_iddouble-locks:clear_tool_idlocks+unlocks, then line 237 locks again.This is both a minor performance issue and the same TOCTOU pattern flagged on
set_events. Consider inlining the clear logic or taking the lock once.Sketch
fn free_tool_id(tool_id: i32, vm: &VirtualMachine) -> PyResult<()> { let tool = check_valid_tool(tool_id, vm)?; - clear_tool_id(tool_id, vm)?; let mut state = vm.state.monitoring.lock(); + if state.tool_names[tool].is_some() { + state.global_events[tool] = 0; + state.local_events.retain(|(local_tool, _), _| *local_tool != tool); + state.callbacks.retain(|(cb_tool, _), _| *cb_tool != tool); + state.disabled.retain(|&(_, _, t)| t != tool); + } state.tool_names[tool] = None; + update_events_mask(vm, &state); Ok(()) }
84-107:MonitoringStatefields are allpub— consider restricting visibility.All fields of
MonitoringStateare public, allowing any code to mutate internal invariants (e.g.,global_events,disabled) without going through the validation functions. Since this struct is behind aPyMutex, direct field access from outside this module could bypass the event normalization and mask update logic. Making fieldspub(crate)or private with accessors would tighten the contract.
| let src_offset = (self.lasti() - 1) * 2; | ||
| let dest = target.get(arg); | ||
| self.jump(dest); | ||
| // JUMP events fire only for backward jumps (loop iterations) | ||
| if vm.state.monitoring_events.load() & crate::stdlib::sys::monitoring::EVENT_JUMP | ||
| != 0 | ||
| { | ||
| let _ = crate::stdlib::sys::monitoring::fire_jump( | ||
| vm, | ||
| self.code, | ||
| src_offset, | ||
| dest.0 * 2, | ||
| ); | ||
| } |
There was a problem hiding this comment.
Avoid swallowing monitoring errors for branch/jump events
These fire_* calls ignore callback failures, while other monitoring events propagate. Consider using ? (or a consistent replacement policy) so monitoring errors aren’t silently lost.
💡 Example adjustment (apply similarly to other sites)
- let _ = crate::stdlib::sys::monitoring::fire_jump(
+ crate::stdlib::sys::monitoring::fire_jump(
vm,
self.code,
src_offset,
dest.0 * 2,
- );
+ )?;Also applies to: 2023-2055, 2061-2094, 3104-3136, 3144-3202
🤖 Prompt for AI Agents
In `@crates/vm/src/frame.rs` around lines 1407 - 1420, The monitoring call
currently swallows errors when firing jump events; update the block in
function(s) using self.lasti(), target.get(arg), self.jump(dest) so the call to
crate::stdlib::sys::monitoring::fire_jump returns its Result and any Err is
propagated instead of ignored—e.g., check vm.state.monitoring_events for
EVENT_JUMP as now but replace the ignored call with a propagating call (use the
? operator or map_err to convert/return the error) so failures from fire_jump
are not silently dropped; apply the same change to the other occurrences
referenced (around the blocks that call fire_* and inspect monitoring_events).
| /// Lightweight frame execution for generator/coroutine resume. | ||
| /// Skips the thread frame stack (sys._current_frames()) and trace call/return events, | ||
| /// since generator frames manage their own PY_RESUME/PY_YIELD monitoring events. | ||
| pub fn resume_gen_frame<R, F: FnOnce(&FrameRef) -> PyResult<R>>( | ||
| &self, | ||
| frame: &FrameRef, | ||
| exc: Option<PyBaseExceptionRef>, | ||
| f: F, | ||
| ) -> PyResult<R> { | ||
| self.check_recursive_call("")?; | ||
| if self.check_c_stack_overflow() { | ||
| return Err(self.new_recursion_error(String::new())); | ||
| } | ||
| self.recursion_depth.update(|d| d + 1); | ||
|
|
||
| // SAFETY: frame (&FrameRef) stays alive for the duration, so NonNull is valid until pop. | ||
| self.frames | ||
| .borrow_mut() | ||
| .push(FramePtr(NonNull::from(&**frame))); | ||
| let old_frame = crate::vm::thread::set_current_frame((&***frame) as *const Frame); | ||
| frame.previous.store( | ||
| old_frame as *mut Frame, | ||
| core::sync::atomic::Ordering::Relaxed, | ||
| ); | ||
| // Inline exception push without thread exception update | ||
| self.exceptions.borrow_mut().stack.push(exc); | ||
| let old_owner = frame.owner.swap( | ||
| crate::frame::FrameOwner::Thread as i8, | ||
| core::sync::atomic::Ordering::AcqRel, | ||
| ); | ||
|
|
||
| let result = f(frame); | ||
|
|
||
| frame | ||
| .owner | ||
| .store(old_owner, core::sync::atomic::Ordering::Release); | ||
| // Inline exception pop without thread exception update | ||
| self.exceptions | ||
| .borrow_mut() | ||
| .stack | ||
| .pop() | ||
| .expect("pop_exception() without nested exc stack"); | ||
| crate::vm::thread::set_current_frame(old_frame); | ||
| self.frames.borrow_mut().pop(); | ||
|
|
||
| self.recursion_depth.update(|d| d - 1); | ||
| result | ||
| } |
There was a problem hiding this comment.
resume_gen_frame lacks panic-safe cleanup — recursion_depth, frames, and exception stack will leak on unwind.
In with_recursion (line 1047), recursion_depth is decremented via scopeguard::defer!. Here, the manual decrement at line 1174 is skipped if f(frame) panics. The same applies to frames.pop(), exception stack pop, frame-chain restore, and owner restore.
At minimum, wrap the recursion depth in a scopeguard to match with_recursion:
Proposed fix (minimal — recursion_depth guard)
self.recursion_depth.update(|d| d + 1);
+ scopeguard::defer! { self.recursion_depth.update(|d| d - 1) }
// SAFETY: frame (&FrameRef) stays alive for the duration, so NonNull is valid until pop.
self.frames
.borrow_mut()
.push(FramePtr(NonNull::from(&**frame)));
let old_frame = crate::vm::thread::set_current_frame((&***frame) as *const Frame);
frame.previous.store(
old_frame as *mut Frame,
core::sync::atomic::Ordering::Relaxed,
);
// Inline exception push without thread exception update
self.exceptions.borrow_mut().stack.push(exc);
let old_owner = frame.owner.swap(
crate::frame::FrameOwner::Thread as i8,
core::sync::atomic::Ordering::AcqRel,
);
let result = f(frame);
frame
.owner
.store(old_owner, core::sync::atomic::Ordering::Release);
// Inline exception pop without thread exception update
self.exceptions
.borrow_mut()
.stack
.pop()
.expect("pop_exception() without nested exc stack");
crate::vm::thread::set_current_frame(old_frame);
self.frames.borrow_mut().pop();
- self.recursion_depth.update(|d| d - 1);
resultIdeally, the frame/exception/owner cleanup would also be wrapped in a scopeguard (or a single RAII guard struct) for full panic safety, as is done for the FIRING flag issue in monitoring.rs.
🤖 Prompt for AI Agents
In `@crates/vm/src/vm/mod.rs` around lines 1129 - 1176, resume_gen_frame currently
performs manual cleanup (recursion_depth.update decrement, frames.pop(),
exceptions.stack.pop(), owner.store(old_owner),
crate::vm::thread::set_current_frame(old_frame)) after calling f(frame), which
will be skipped if f panics; wrap the post-call cleanup in a panic-safe guard
(either use scopeguard::defer! immediately after successfully performing the
setup steps or create a small RAII guard struct) so that on unwind you always:
decrement recursion_depth, pop self.frames, pop the exceptions stack, restore
frame.owner to old_owner, and reset the current thread frame via
crate::vm::thread::set_current_frame(old_frame); place the guard right after
pushing the FramePtr and pushing the exception so it has access to old_owner and
old_frame and remove the manual cleanup at the end or let the guard perform it.
📦 Library DependenciesThe following Lib/ modules were modified. Here are their dependencies: [ ] lib: cpython/Lib/asyncio dependencies:
dependent tests: (7 tests)
[ ] test: cpython/Lib/test/test_c_locale_coercion.py dependencies: dependent tests: (no tests depend on c_locale_coercion) [ ] test: cpython/Lib/test/test_compile.py (TODO: 23) dependencies: dependent tests: (no tests depend on compile) [x] lib: cpython/Lib/io.py dependencies:
dependent tests: (88 tests)
[x] lib: cpython/Lib/linecache.py dependencies:
dependent tests: (65 tests)
[x] test: cpython/Lib/test/test_monitoring.py (TODO: 8) dependencies: dependent tests: (no tests depend on monitoring) [x] lib: cpython/Lib/re dependencies:
dependent tests: (60 tests)
[ ] lib: cpython/Lib/subprocess.py dependencies:
dependent tests: (52 tests)
[ ] test: cpython/Lib/test/test_sys.py (TODO: 6) dependencies: dependent tests: (211 tests)
[ ] test: cpython/Lib/test/test_unicodedata.py (TODO: 18) dependencies: dependent tests: (no tests depend on unicode) [x] lib: cpython/Lib/urllib dependencies:
dependent tests: (27 tests)
[x] lib: cpython/Lib/warnings.py dependencies:
dependent tests: (49 tests)
Legend:
|
Summary by CodeRabbit
New Features
Bug Fixes
Refactor