Separate OpenMP from Pyccel's core functionalities and connect it as a Plugin by rmahjoubi · Pull Request #2338 · pyccel/pyccel

rmahjoubi · 2025-06-12T00:33:04Z

This PR reintroduces OpenMP in Pyccel as a standalone plugin, interfacing with Pyccel via a plugins utility.

Plugins Utility

The plugins module provides the basic classes from which every plugin should inherit, and it's the main interface to the plugins from Pyccel.

Core Functionality

Via the plugins module, a Pyccel class instance can register itself to be patched with the plugin's functionalities, then each plugin picks its targets from the registered objects
Currently, the plugins are loaded and used in no particular order; when a plugin fails to load, a warning is raised and the loader resumes with the rest
When an object registers itself with the plugins, every plugin that is targeting the object will patch it in the order the loading happened
When an object unregisters itself from the plugins, every plugin that is targeting the object undoes its patches in the order the loading happened
Some plugins may need to change their behavior after set_options is called; for the changes to take place, set_options should be called with refresh set to True

OpenMP Plugin

Plugin Behavior

The plugin keeps track of any applied versions and only applies other versions when specified in the options; any applied version persists until the unregister method is called

Classes

Treating TextX objects is done in the syntactic stage via classes that inherit from OmpTxNode, which itself inherits from OmpNode; visiting these classes results in instances of their respective classes that inherit from OmpNode
Each OpenMP version has dedicated syntactic/semantic parsers and printers
Printers use the raw statement of an OpenMP object provided in the comment; the segmentation of these statements allows for the suppression or tweaking of printing of segments correlated to the desired OpenMP object

the support for exit is limited to calls with integral arguments or without arguments (same as exit(0))

Replace every class in the ast/omp.py with new classes the new classes have the same name as the textx grammar rules because these classes are used as the base classes for the textx instead of the ones found in pyccel/parser/syntax/openmp.py

the version field will be used to store the version of the omp grammer used for a Construct/Clause this is done by making the grammer sets a version field for the Clause/Construct if there are multile versions of the grammer

the grammer is not finished an only supports parallel constructs and its clauses

instead of hardcoding the rules in the syntactic/semantic/printing stages we just delegate the work to the omp class new functions there will be 5 functions _visit_syntatic, _visit_semantic, _cprint, _fprint, _pprint those functions will be called from the syntactic/semantic/printing stages and the omp class will be responsible for checking the if everything is corrent and provide the correct printing

this way we can support multiple version of omp withouth having to duplicate the code and just use inheritance and override the methods to change the behavior of the previous version

- add a version setter for the OmpAnnotatedComment class to be able to check if the syntax is supported by the version of omp - fixing liter errors

rmahjoubi · 2025-07-24T16:42:46Z

/bot run linux intel macosx windows

rmahjoubi · 2025-07-24T17:31:56Z

/bot run linux

pyccel-bot · 2025-07-24T19:31:59Z

I can't seem to find your checklist to confirm that you have completed all necessary tasks. Please request one using /bot checklist.

rmahjoubi · 2025-07-24T22:30:30Z

/bot run intel macosx windows

rmahjoubi · 2025-07-24T23:00:31Z

/bot run coverage

pyccel-bot

Good job ! Your PR is using all the code it added/changed.

rmahjoubi · 2025-07-24T23:06:49Z

Here is your checklist. Please tick items off when you have completed them or determined that they are not necessary for this pull request:

Write a clear PR description
Add tests to check your code works as expected
Update documentation if necessary
Update Changelog
Ensure any relevant issues are linked
Ensure new tests are passing
Update AUTHORS file

pyccel-bot

Good job ! Your PR is using all the code it added/changed.

pyccel-bot

Good job ! Your PR is using all the code it added/changed.

EmilyBourne · 2025-07-28T06:47:37Z

@rmahjoubi

rmahjoubi · 2025-07-28T15:31:35Z

@EmilyBourne I'm not sure about this error, as original_method is an argument to the constructor of the OmpPatchInfo class. I think this might disappear if we run the Codacy tests again.

rmahjoubi · 2025-08-04T12:58:58Z

/bot run linux

EmilyBourne

Do you think there is anyway to split this PR? It really is enormous and quite hard to review. I would love to split it into a Plugin part and an OpenMP part but I guess that would be tricky for testing? Could we have fake plugin for the tests in tests/plugin to get round this?

EmilyBourne · 2025-08-11T07:39:57Z

+
+## Overview
+
+The Pyccel plugin system allows Pyccel's functionality to be extended through plugins. Plugins can modify the behaviour of existing classes by patching their methods or adding new methods. This is particularly useful for adding support for new language features, optimisations, or code generation targets without modifying the core codebase.


Plugins can modify the behaviour of existing classes by patching their methods or adding new methods

Is this true for all classes or only certain classes (SyntaxParser, SemanticParser, Codegen) ?

EmilyBourne · 2025-08-11T07:41:00Z

+
+1. `register(instances)` - Register instances with the plugin. This method should apply patches to the provided instances.
+2. `refresh()` - Refresh all registered targets. This method should reapply existing patches to registered targets and apply any new patches, if necessary.
+3. `deregister(instances)` - Deregister instances from the plugin. This method should remove all patches applied to the provided instances.


I thought you had decided to remove the deregister functionality given that new class instances are created when we run Pyccel anyway?

EmilyBourne · 2025-08-11T07:42:18Z

+1. `register(instances)` - Register instances with the plugin. This method should apply patches to the provided instances.
+2. `refresh()` - Refresh all registered targets. This method should reapply existing patches to registered targets and apply any new patches, if necessary.
+3. `deregister(instances)` - Deregister instances from the plugin. This method should remove all patches applied to the provided instances.
+4. `set_options(options)` - Set options for the plugin. This method should update the plugin's behaviour based on the provided options.


Should there also be a get_options method? Maybe get_options_kwargs and get_parser_options to handle options passed via epyccel or pyccel?

EmilyBourne · 2025-08-11T07:43:46Z

+    def load_plugins(self, plugins_dir=None):
+        """Discover and load all plugins from the plugins directory."""
+        # ...


Have you timed this? What exactly is loaded here? Will it result in the grammar for all OpenMP versions being calculated? The call to redbaron is usually quite costly so if we can avoid it we probably should

Is plugins_dir pyccel/plugins/MyPlugin/ or pyccel/plugins/?

EmilyBourne · 2025-08-11T07:45:39Z

+def register(self, registry):
+    """Apply patches to the target."""
+    target = registry.target
+
+    # Example: Patch the 'parse' method
+    original_method = getattr(target, 'parse', None)
+
+    def parse(self, *args, **kwargs):
+        # Do something before the original method
+        print("Before parsing")
+
+        # Call the original method
+        result = original_method(self, *args, **kwargs)
+
+        # Do something after the original method
+        print("After parsing")
+
+        return result
+
+    # Apply the patch


Ok. Maybe add a comment stating that functions should usually be defined in the module not the function otherwise people are likely to take this as the best practice example

EmilyBourne · 2025-08-11T09:00:07Z

+OmpTxIntegerExpr: value=InnerContent | ParenthesisContent;
+OmpTxConstantPositiveInteger: value=/[0-9]+/;
+OmpTxList: throw_away+=ID[','];
+ParenthesizedExpression: (('(' ParenthesizedExpression ')') | /[^()]*/)*; // TODO: find a better way to do this as this will not escape the parenthesis inside of a string


Can you elaborate more on this TODO? Maybe I can help? Usually .tx files are designed to handle parsing in a way that is more explicit than regexes to avoid this kind of issue.

Maybe you are just missing something that matches a string?

EmilyBourne · 2025-08-11T09:00:34Z

+OmpTxList: throw_away+=ID[','];
+ParenthesizedExpression: (('(' ParenthesizedExpression ')') | /[^()]*/)*; // TODO: find a better way to do this as this will not escape the parenthesis inside of a string
+DataSharingAttribute: ('shared' | 'none' | 'private' | 'firstprivate');
+ReductionOperator: ('+' | '-' | '*' | '&' | '|' | '^' | '&&' | '||' | 'min' | 'max'); // TODO: those are only the implicitly declared ones for C and C++. Add the others for Fortran and support user-defined ones (see specs https://www.openmp.org/spec-html/5.0/openmpsu107.html#x140-581002)


Forgotten TODO

EmilyBourne · 2025-08-11T09:19:17Z

+    position : tuple
+        The start and end positions of the OpenMP syntax in the source code, 
+        used to print errors.


Ok. Please add a comment about this

Hellio404 added 30 commits September 11, 2022 17:08

adding support for sys.exit

f02f6b8

the support for exit is limited to calls with integral arguments or without arguments (same as exit(0))

fixing linter

c844fd3

allowing exit to get variables and adding test for exit

f3ee113

Merge branch 'master' into fix-931

6e43f42

fixing

36312fe

fixing codacy issues

dfc8f83

using stop instead of call exit

2fd5bb5

support printing to the stderror

a5f087e

use PythonPrint to print non-integer args of exit to the stderr

e75f1ea

update some docstrings and fix codacy issues

c09822c

Merge branch 'master' into fix-931

f58d3d7

fixing some codacy issues

30dadfd

spliting long line

b24a1bc

fixing a bug in the iteration over map

4a1cfb5

fixing trailing whitespace

c062d7d

fixing requested changes

719c57c

fixing a bug in _print_SysExit for fortran

7154d66

fixing bug when exit arg is an array of integers

337eb85

fixing a typo

052a345

fixing codacy problems

500b251

forgot the status= in the exit

c2eb998

Merge branch 'master' into fix-931

3830402

removing shebang form sysext.py

f51e6d4

Replace the omp ast classes with new ones

0df0b31

Replace every class in the ast/omp.py with new classes the new classes have the same name as the textx grammar rules because these classes are used as the base classes for the textx instead of the ones found in pyccel/parser/syntax/openmp.py

fix a mistake (wrong inhereted class)

9b385b7

adding version field to the OmpAnnotatedComment class

a4cf9e1

the version field will be used to store the version of the omp grammer used for a Construct/Clause this is done by making the grammer sets a version field for the Clause/Construct if there are multile versions of the grammer

adding new textx grammer for parsing openmp

e1f63d4

the grammer is not finished an only supports parallel constructs and its clauses

removing the old omp classes used in for the textx and create new ones

0e1729f

this way we can support multiple version of omp withouth having to duplicate the code and just use inheritance and override the methods to change the behavior of the previous version

last changes before start implementing the visit functions

f33df5c

- add a version setter for the OmpAnnotatedComment class to be able to check if the syntax is supported by the version of omp - fixing liter errors

rmahjoubi added 2 commits July 24, 2025 17:24

fix test

2f467cd

Merge remote-tracking branch 'origin/devel' into rewrite-omp

16ba225

replace accelerators attribute with _extra_compilation_tools

09cbe5a

EmilyBourne marked this pull request as ready for review July 24, 2025 19:31

github-actions bot marked this pull request as draft July 24, 2025 19:32

pyccel-bot bot approved these changes Jul 24, 2025

View reviewed changes

remove extra line

9d5cd74

rmahjoubi marked this pull request as ready for review July 24, 2025 23:08

fix typo

1ee7a15

rmahjoubi requested a review from EmilyBourne July 24, 2025 23:21

pyccel-bot bot approved these changes Jul 24, 2025

View reviewed changes

github-actions bot marked this pull request as draft July 25, 2025 10:11

rmahjoubi marked this pull request as ready for review July 25, 2025 16:02

pyccel-bot bot approved these changes Jul 25, 2025

View reviewed changes

EmilyBourne added the needs_initial_review label Jul 31, 2025

Merge branch 'devel' into rewrite-omp

ea15787

EmilyBourne requested changes Aug 11, 2025

View reviewed changes

EmilyBourne mentioned this pull request Sep 18, 2025

Improve Windows installation and fix CI #2451

Merged

EmilyBourne mentioned this pull request Oct 22, 2025

Use Python3 style class declarations #2471

Merged

EmilyBourne mentioned this pull request Jan 9, 2026

[EPIC] GPU Support #2539

Open

EmilyBourne mentioned this pull request Jan 21, 2026

[Feature Request] Improve OpenMP offload support #2550

Open

3 tasks


		## Overview

		The Pyccel plugin system allows Pyccel's functionality to be extended through plugins. Plugins can modify the behaviour of existing classes by patching their methods or adding new methods. This is particularly useful for adding support for new language features, optimisations, or code generation targets without modifying the core codebase.

Conversation

rmahjoubi commented Jun 12, 2025

Plugins Utility

OpenMP Plugin

Uh oh!

rmahjoubi commented Jul 24, 2025

Uh oh!

rmahjoubi commented Jul 24, 2025

Uh oh!

pyccel-bot bot commented Jul 24, 2025

Uh oh!

rmahjoubi commented Jul 24, 2025

Uh oh!

rmahjoubi commented Jul 24, 2025

Uh oh!

pyccel-bot bot left a comment

Choose a reason for hiding this comment

Uh oh!

rmahjoubi commented Jul 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pyccel-bot bot left a comment

Choose a reason for hiding this comment

Uh oh!

pyccel-bot bot left a comment

Choose a reason for hiding this comment

Uh oh!

EmilyBourne commented Jul 28, 2025

Uh oh!

rmahjoubi commented Jul 28, 2025

Uh oh!

rmahjoubi commented Aug 4, 2025

Uh oh!

EmilyBourne left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rmahjoubi commented Jul 24, 2025 •

edited

Loading