i'm working on overhauling NOvA's grid infrastructure, and i have a feature request for jobsub_lite that would significantly streamline our job submission infrastructure – namely, the ability to invoke it from python directly.

from reading the jobsub_lite code, my understanding is that the lib directory defines an importable python package, in which the guts of the machinery are located, and then the command line interface is provided via jobsub_submit in the bin directory. according to its docstring, this script is responsible for the following:

• parse args
• get credentials
• handle tarfile options
• set added values from environment, etc.
• convert/render template files to submission files
• launch

NOvA users typically interface with jobsub_lite via python scripts, which provide a layer of abstraction to assemble the raw jobsub_submit command for a few common workflows (mostly Art and CAFAna jobs), and then execute it via a call to subprocess. so essentially, we run python scripts that assemble bash commands that execute a python script.

my request is this: would it be possible to take all of the functionality of the main function in jobsub_submit that comes after argument parsing and migrate it into lib, so it can be imported and invoked from python scripts, taking arguments as kwargs? then the jobsub_submit executable itself would simply parse the command line args and then pass them as kwargs into this submission function. that way, our python submission scripts can import and run this submission function directly, without needing to invoke the subprocess package.

the ability to run jobsub submission and query commands from python natively would be a game-changer. i'm happy to implement these changes and submit them as a PR for review, if this idea is something you'd be open to. thank you!