GErunner, the Python tool for Sun/Univa GridEngine enables to describe jobs or processing pipeline and submit them from Python.
This tool is not yet packaged. Just clone this repository into your directory.
GEJob object has almost of all attributes and functions correspond to qsub command.
>>> from GErunner import GEJob
>>> myjob = GEJob("ls", args=["-l"], cwd=True, binary=True)
>>> myjob.submit()
ls -> 12345 [ SUBMITTED ]
12345
mem is shortcut attribute for setting memory resouce and slot is for def_slot.
>>> myjob = GEJob("hoge.sh", mem="4G", slot=8)
>>> myjob.resource
{'s_vmem': '4G', 'mem_req': '4G'}
>>> myjob.parallel_env
{'def_slot': [8]}
GESeriesJob and GEParallelJob are automatically assign job id to -hold_jid (-hold_jid_ad) option.
>>> from GErunner import GEJob, GESeriesJob
>>> job1 = GEJob("pwd", binary=True)
>>> job2 = GEJob("ls", binary=True)
>>> pipeline = GESeriesJob([job1, job2])
>>> pipeline.submit()
pwd -> 12345 [ SUBMITTED ]
ls -> 12346 [ SUBMITTED ] (waiting 12345)
12346
They can be nested.
>>> from GErunner import GEJob, GESeriesJob,GEParallelJob
>>> job1_1 = GEJob("ls", binary=True)
>>> job1_2 = GEJob("ls", args=["-l"], binary=True)
>>> job2 = GEJob("pwd", binary=True)
>>> parallel = GEParallelJob([job1_1, job1_2])
>>> pipeline = GESeriesJob([parallel, job2])
>>> pipeline.submit()
ls -> 12345 [ SUBMITTED ]
ls -> 12346 [ SUBMITTED ]
pwd -> 12347 [ SUBMITTED ] (waiting 12345,12346)
12347
GEArrayJob makes it easy to submit array job from arguments list.
>>> from GErunner import GEArrayJob
>>> args = ("spam", "ham", "egg")
>>> myjob = GEArrayJob("grep", args=["{1}", "huga.txt"], arg1=args, binary=True, cwd=True)
>>> myjob.submit()
grep -> 12345.1-3:1 [ SUBMITTED ]
12345
GErunner.qsubparse has the argument parser for qsub command.
>>> from GErunner.qsubparse import command2GEJob
>>> myjob = command2GEJob("-cwd -b y ls -l".split())