You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm a great fan of py-spy for profiling but have been having issues using it with greenlets because greenlets truncate their Python stack at the frame where the greenlet is created so none of the parent frames are available. I've gotten around this with the following patch for greenlet 1.x versions:
Presumably something similar for greenlet 2.x but in greenlet_greenlet.hpp in set_initial_state(). I haven't been able to test on 2.x because I've been unable to get gevent to work with it. Any reasons this approach would be a bad idea?
The text was updated successfully, but these errors were encountered:
Hmm, interesting idea. Though, I'm also a big py-spy user and haven't had any problems using it with greenlets.
I have concerns about whether it's even reliably possible, though, because the frames that created the greenlet can have been exited while the greenlet is still alive. In the best case scenario, we would wind up keeping references to all those frames alive, preventing deallocating any objects referenced by them. That can be a serious problem. (I haven't worked through all the details, but worse scenarios could be anything from confusing/corrupted tracebacks to improper unwinding behaviour [exceptions, try/finally, context managers] or interpreter crashes.)
In normal circumstances, this reports that it required 8MB of memory to run, and runs in <4s. If I capture the frame when the greenlet is started, memory usage balloons to nearly 2GB! (And runtime doubles because of GC overhead.)
Even if this were to prove reliably possible, and no one was concerned about keeping all those extra objects around, it would be a substantial behaviour change visible to users. Not just in things like traceback.print_stack() or sys._current_frames(), but I expect things like debuggers to be affected. So it's not something we could just slip in, it would need a major version bump.
I'm a great fan of py-spy for profiling but have been having issues using it with greenlets because greenlets truncate their Python stack at the frame where the greenlet is created so none of the parent frames are available. I've gotten around this with the following patch for greenlet 1.x versions:
Presumably something similar for greenlet 2.x but in greenlet_greenlet.hpp in set_initial_state(). I haven't been able to test on 2.x because I've been unable to get gevent to work with it. Any reasons this approach would be a bad idea?
The text was updated successfully, but these errors were encountered: