Skip to content

Parallel execution of multiple IronPython engines with one runtime #1826

@FSchu

Description

@FSchu

Description

Hi,
we found a changed behaviour in IP 3.4.1 compared to the IP 2.x implementation.
We are using IP to execute python code in multiple threads.
To improve performance the work threads are precreated and share the same (global) runtime.
After switching to IP 3 we found that addings an assembly via the clr methodes and also imports to e.g. the decimal module sometimes fail. This is different for what happend in IP 2.x, there was no issue, running using mutiple python enginens in parallel threads with one global runtime.

We have been able to mitigate the issue with the addition of a process wide lock object used in the methods responsible for imports and adding assembly references. However, we can't tell for sure that all potential methods are

Steps to Reproduce

  1. Create a shared runtime
  2. Create multipe thread having their private python engine
  3. Ensure that the threads are started at the same time
  4. Have the engine execute script code including clr.AddReference calls and "from decimal import Decimal"
  5. Attached is a C# sample program (renamed to .cs.txt to allow upload) and a patch file with the changes we made to try to fix the issue
    Program.cs.txt
    mutliThreadedSuport.patch

Expected behavior:
All threads execute the script code without errors

Actual behavior:
Some threads fail to execute the script code with error like "module 'numbers' not in sys.modules" or failure to add the assembly via the clr method. The actual error can be different.

Version Information

  • .NET platform used: .NET Framework
  • Version 4.6.2
  • Windows 11
  • 3.4.1 DEBUG (3.4.1.1000) / [.NETFramework,Version=v4.6.2 on .NET Framework 4.8.9282.0 (64-bit)]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions