sge¶
-
class
anadama2.grid.sge.
SGE
(partition, tmpdir, benchmark_on=None, options=None, environment=None)[source]¶ This class enables the Workflow class to dispatch tasks to Sun Grid Engine and its lookalikes. Use it like so:
from anadama2 import Workflow from anadama2.sge import SGE ctx = Workflow(grid_powerup=SGE(queue="general")) ctx.do("wget " "ftp://public-ftp.hmpdacc.org/" "HMMCP/finalData/hmp1.v35.hq.otu.counts.bz2 " "-O @{input/hmp1.v35.hq.otu.counts.bz2}") # run on sge with 200 MB of memory, 4 cores, and 60 minutes t1 = ctx.grid_do("pbzip2 -d -p 4 < #{input/hmp1.v35.hq.otu.counts.bz2} " "> @{input/hmp1.v35.hq.otu.counts}", mem=200, cores=4, time=60) # run on sge on the serial_requeue queue ctx.grid_add_task("some_huge_analysis {depends[0]} {targets[0]}", depends=t1.targets, targets="output.txt", mem=4000, cores=1, time=300, partition="serial_requeue") ctx.go()
Parameters: - partition (str) – The name of the SLURM partition to submit tasks to
- tmpdir (str) – A directory to store temporary files in. All
machines in the cluster must be able to read the contents of
this directory; uses
anadama2.picklerunner
to create self-contained scripts to run individual tasks and callssrun
to run the script on the cluster. - benchmark_on – Option to turn on/off benchmarking
: type benchmark_on: bool
Parameters: options (str) – Grid specific options to apply to each job