Provide detailed error for circular from imports#5972
Provide detailed error for circular from imports#5972youknowone merged 5 commits intoRustPython:mainfrom
from imports#5972Conversation
|
Warning Rate limit exceeded@ever0de has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 6 minutes and 36 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (1)
WalkthroughThe Changes
Poem
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
9c09aef to
dfb4db1
Compare
dfb4db1 to
d86644e
Compare
from importsfrom imports
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: .coderabbit.yml
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
Lib/test/test_import/__init__.pyis excluded by!Lib/**
📒 Files selected for processing (3)
vm/src/builtins/module.rs(3 hunks)vm/src/frame.rs(2 hunks)vm/src/import.rs(2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.rs
Instructions used from:
Sources:
📄 CodeRabbit Inference Engine
- .github/copilot-instructions.md
🧠 Learnings (2)
vm/src/builtins/module.rs (1)
Learnt from: CR
PR: RustPython/RustPython#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-06-30T10:08:48.858Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
vm/src/import.rs (1)
Learnt from: CR
PR: RustPython/RustPython#0
File: .github/copilot-instructions.md:0-0
Timestamp: 2025-06-30T10:08:48.858Z
Learning: Applies to **/*.rs : Use the macro system (`pyclass`, `pymodule`, `pyfunction`, etc.) when implementing Python functionality in Rust
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Run snippets and cpython tests (windows-latest)
🔇 Additional comments (4)
vm/src/builtins/module.rs (2)
1-1: LGTM! Atomic boolean field properly added for initialization tracking.The
AtomicBoolimport and field addition are appropriate for tracking module initialization state in a thread-safe manner.Also applies to: 57-57
80-80: Field initialization is correct in both constructors.The
initializingfield is properly initialized tofalsein bothnew()andfrom_def()constructors, ensuring modules start in a non-initializing state.Also applies to: 88-88
vm/src/import.rs (2)
3-3: Imports correctly added for atomic operations.The
Orderingimport andPyModuleaddition are necessary for the initialization tracking implementation.Also applies to: 7-7
161-178: Well-implemented guard pattern for initialization tracking.The guard ensures the
initializingflag is always reset, even if module code execution fails or panics. The use ofOrdering::Relaxedis appropriate here.However, this sets an internal field that isn't exposed to Python code. The
frame.rscode checks__spec__._initializing, which is a different attribute. Consider exposing this state to Python.
| fn import_from(&mut self, vm: &VirtualMachine, idx: bytecode::NameIdx) -> PyResult { | ||
| let module = self.top_value(); | ||
| let name = self.code.names[idx as usize]; | ||
| let err = || vm.new_import_error(format!("cannot import name '{name}'"), name.to_owned()); | ||
|
|
||
| // Load attribute, and transform any error into import error. | ||
| if let Some(obj) = vm.get_attribute_opt(module.to_owned(), name)? { | ||
| return Ok(obj); | ||
| } | ||
| // fallback to importing '{module.__name__}.{name}' from sys.modules | ||
| let mod_name = module | ||
| .get_attr(identifier!(vm, __name__), vm) | ||
| .map_err(|_| err())?; | ||
| let mod_name = mod_name.downcast::<PyStr>().map_err(|_| err())?; | ||
| let full_mod_name = format!("{mod_name}.{name}"); | ||
| let sys_modules = vm.sys_module.get_attr("modules", vm).map_err(|_| err())?; | ||
| sys_modules.get_item(&full_mod_name, vm).map_err(|_| err()) | ||
|
|
||
| let fallback_result: Option<PyResult> = module | ||
| .get_attr(&vm.ctx.new_str("__name__"), vm) | ||
| .ok() | ||
| .and_then(|mod_name| mod_name.downcast_ref::<PyStr>().map(|s| s.to_owned())) | ||
| .and_then(|mod_name_str| { | ||
| let full_mod_name = format!("{}.{}", mod_name_str.as_str(), name.as_str()); | ||
| vm.sys_module | ||
| .get_attr("modules", vm) | ||
| .ok() | ||
| .and_then(|sys_modules| sys_modules.get_item(&full_mod_name, vm).ok()) | ||
| }) | ||
| .map(Ok); | ||
|
|
||
| if let Some(Ok(sub_module)) = fallback_result { | ||
| return Ok(sub_module); | ||
| } | ||
|
|
||
| if is_module_initializing(module, vm) { | ||
| let module_name = module | ||
| .get_attr(&vm.ctx.new_str("__name__"), vm) | ||
| .ok() | ||
| .and_then(|n| n.downcast_ref::<PyStr>().map(|s| s.as_str().to_owned())) | ||
| .unwrap_or_else(|| "<unknown>".to_owned()); | ||
|
|
||
| let msg = format!( | ||
| "cannot import name '{}' from partially initialized module '{}' (most likely due to a circular import)", | ||
| name.as_str(), | ||
| module_name | ||
| ); | ||
| Err(vm.new_import_error(msg, name.to_owned())) | ||
| } else { | ||
| Err(vm.new_import_error( | ||
| format!("cannot import name '{}'", name.as_str()), | ||
| name.to_owned(), | ||
| )) | ||
| } | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Critical: Disconnect between internal initializing field and Python-level __spec__._initializing attribute.
The refactored import logic looks correct, but there's a fundamental issue: this code checks __spec__._initializing (line 1389 via is_module_initializing), but the import_code_obj function in import.rs only sets the internal PyModule.initializing field. There's no code that exposes this internal state to Python as __spec__._initializing.
Without this connection, the circular import detection won't work as intended.
Consider either:
- Setting
__spec__._initializinginimport_code_objwhen setting the internal field - Modifying
is_module_initializingto check the internalPyModule.initializingfield instead
Run the following script to verify if __spec__._initializing is set anywhere in the codebase:
🏁 Script executed:
#!/bin/bash
# Search for any code that sets _initializing on spec objects
rg -A 5 "_initializing.*="
# Search for any references to the initializing field on PyModule
ast-grep --pattern 'initializing'Length of output: 62
🏁 Script executed:
#!/bin/bash
set -e
# Search for any occurrences of the internal `initializing` field in Rust code
rg -n "initializing" -t rs
# Search for any usage of `__spec__` or `_initializing` on spec objects
rg -n "__spec__" -t rs
rg -n "_initializing" -t rsLength of output: 62
🏁 Script executed:
#!/bin/bash
set -e
# Search for occurrences of 'initializing' in all Rust source files
rg -n "initializing" --glob '*.rs'
# Search for any usage of '__spec__' in Rust source
rg -n "__spec__" --glob '*.rs'
# Search for any explicit '_initializing' assignments or checks in Rust source
rg -n "_initializing" --glob '*.rs'Length of output: 1656
Critical: Circular‐import detection is broken—spec._initializing is never set
The new is_module_initializing path in vm/src/frame.rs (lines 1389, 2405–2415) checks the Python‐level __spec__._initializing flag, but no code ever assigns _initializing on the module’s __spec__. We only toggle the internal PyModule.initializing field in vm/src/import.rs (lines 168, 172), so the “partially initialized module” branch will never fire.
To resolve this, choose one of the following fixes:
• In vm/src/import.rs (around lines 168–172), after calling
module.initializing.store(true/false, Ordering::Relaxed);also set the Python‐level flag:
let spec = module.get_attr(&vm.ctx.new_str("__spec__"), vm)?;
spec.set_attr(&vm.ctx.new_str("_initializing"), vm.ctx.new_bool(true), vm)?;(and similarly for false)
• Or in vm/src/frame.rs’s is_module_initializing (lines 2405–2415), downcast module to PyModule and read its internal initializing flag directly instead of querying __spec__.
Add appropriate tests to verify circular‐import error messages now appear.
🤖 Prompt for AI Agents
In vm/src/frame.rs lines 1363 to 1408, the circular-import detection relies on
the Python-level __spec__._initializing flag, but this flag is never set,
causing the detection to fail. To fix this, either update vm/src/import.rs
around lines 168-172 to set the _initializing attribute on the module's __spec__
object whenever the internal PyModule.initializing flag is changed, or modify
the is_module_initializing function in vm/src/frame.rs (lines 2405-2415) to
downcast the module to PyModule and check the internal initializing field
directly. Additionally, add tests to confirm that circular-import errors are
correctly detected and reported.
| fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool { | ||
| let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else { | ||
| return false; | ||
| }; | ||
| if vm.is_none(&spec) { | ||
| return false; | ||
| } | ||
| let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else { | ||
| return false; | ||
| }; | ||
| initializing_attr.try_to_bool(vm).unwrap_or(false) | ||
| } |
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Helper function implementation is sound but relies on unset attribute.
The is_module_initializing function has proper error handling, but it checks for __spec__._initializing which isn't being set by the import system. This function needs to either:
- Check the internal
PyModule.initializingfield if the module is aPyModuleinstance - Ensure that
__spec__._initializingis properly set during import
Consider checking the internal field directly:
fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool {
+ // First try to check the internal field if this is a PyModule
+ if let Ok(py_module) = module.downcast_ref::<PyModule>() {
+ use std::sync::atomic::Ordering;
+ return py_module.initializing.load(Ordering::Relaxed);
+ }
+
+ // Fallback to checking __spec__._initializing
let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else {
return false;
};
if vm.is_none(&spec) {
return false;
}
let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else {
return false;
};
initializing_attr.try_to_bool(vm).unwrap_or(false)
}Run the following script to check how CPython handles module initialization tracking:
🌐 Web query:
How does CPython track module initialization state for circular import detection? Does it use __spec__._initializing?
💡 Result:
CPython tracks module initialization state for circular import detection by leveraging the sys.modules cache. When a module is first imported, a module object is placed into sys.modules immediately—before its code is executed. This ensures that if the module is imported again (such as via a circular import), Python will return this already-existing (but possibly only partially initialized) module object, rather than starting a new import. This mechanism is essential for breaking import cycles and preventing infinite recursion during import[3][7].
The __spec__._initializing attribute (or similar such as __spec__._initialized) is an internal implementation detail and is not the primary mechanism for detection of circular imports in CPython. There have been discussions and experimental ideas about using such an attribute to mark a module’s initialization state for more fine-grained control or for advanced import hacks like async imports[6][8]. However, in normal CPython operation and as of current mainstream Python versions, this attribute is either not present or not used by the import machinery for circular import detection.
Instead, here is how it works:
- When a module is being imported, it is immediately inserted into
sys.modules, signaling "import in progress." - If another module tries to import it during this time, it receives a reference to this partially initialized module[3][7].
- If the code tries to access names that haven’t been defined yet (because initialization hasn't finished), you get errors like:
ImportError: cannot import name 'X' from 'Y' (most likely due to a circular import)[7]
- There is no robust, public-facing per-module "initializing" flag used by the import system; the presence of the module object in
sys.modulesbefore its code has finished running is the primary state signal.
Summary Table
| Mechanism | Used by CPython for circular import detection? | Notes |
|---|---|---|
sys.modules early insert |
Yes | Inserted immediately, acts as the signal for "import in progress" |
__spec__._initializing |
No (not in standard import) | May be used in experimental or advanced import schemes, not standard in CPython as of 2025 |
| Explicit module state flag | No | Not part of public API or normal workflow |
In conclusion:
CPython does not use __spec__._initializing to track module initialization state for the purpose of circular import detection; it relies on the presence of the module in sys.modules during import.[3][7]
Citations:
- 1: https://github.com/bndr/pycycle
- 2: https://github.com/python/cpython/blob/master/Lib/test/test_import/__init__.py
- 3: https://rollbar.com/blog/how-to-fix-circular-import-in-python/
- 4: https://realpython.com/python-import/
- 5: https://www.geeksforgeeks.org/python/python-circular-imports/
- 6: https://discuss.python.org/t/async-imports-to-reduce-startup-times/69732
- 7: https://www.datacamp.com/tutorial/python-circular-import
- 8: Improve AttributeError message for partially initialized module python/cpython#77418
- 9: https://labex.io/tutorials/python-how-to-resolve-circular-imports-in-python-418812
- 10: https://docs.kanaries.net/topics/Python/python-circular-import
Streamline is_module_initializing to use internal flag and drop __spec__._initializing fallback.
- File: vm/src/frame.rs (lines 2405–2416)
- CPython doesn’t use
__spec__._initializingfor import tracking—modules insys.modulessignal “in progress.” - RustPython’s
PyModulealready has aninitializingflag; use that and returnfalsefor all other objects.
fn is_module_initializing(module: &PyObject, vm: &VirtualMachine) -> bool {
- // Fallback to checking __spec__._initializing
- let Ok(spec) = module.get_attr(&vm.ctx.new_str("__spec__"), vm) else {
- return false;
- };
- if vm.is_none(&spec) {
- return false;
- }
- let Ok(initializing_attr) = spec.get_attr(&vm.ctx.new_str("_initializing"), vm) else {
- return false;
- };
- initializing_attr.try_to_bool(vm).unwrap_or(false)
+ // Use internal initializing flag on PyModule
+ if let Ok(py_module) = module.downcast_ref::<PyModule>() {
+ use std::sync::atomic::Ordering;
+ return py_module.initializing.load(Ordering::Relaxed);
+ }
+ false
}🤖 Prompt for AI Agents
In vm/src/frame.rs around lines 2405 to 2416, refactor the
is_module_initializing function to remove the fallback logic that checks
__spec__._initializing. Instead, check if the given module is a PyModule and
directly return its initializing flag. For all other object types, return false.
This aligns with CPython's approach and simplifies the function by relying on
the internal initializing flag of PyModule.
2d99040 to
69c0294
Compare
Summary by CodeRabbit
Bug Fixes
New Features