helpers

Contains functions that help create tasks. All functions contained herein are intended for use with anadama2.workflow.Workflow.add_task(). This means that the functions in here don’t immediately do what they say; they return functions that, when called, do that they say (they’re closures). Sorry if that breaks your brain.

Using closures lets you add tasks like this:

from anadama2 import Workflow
from anadama2.helpers import sh

ctx = Workflow()
ctx.add_task(sh("my fancy shell command"),
             targets="foobaz.txt")

Instead of this:

from anadama2 import Workflow
from anadama2.util import sh # <--- note the different import

ctx = Workflow()
ctx.add_task(lambda task: sh("my fancy shell command"),
             targets="foobaz.txt")

anadama2.helpers.apply_sh(actions)

Add the shell function to any actions that are strings

anadama2.helpers.file_size(depends)

Return the size of the file in GB

anadama2.helpers.format_command(command, **kwargs)

Format the shell command to allow for special variables Here’s a synopsis of common use cases:

• [targets[0]] is formatted to the first target
• [depends[2]] is formatted to the third dependency

Extra keyword arguments are also added to the formatting keyword arguments. Thus, adding a keyword argument of threads=1 makes [threads] be formatted to 1 in the shell command. :type s: str

anadama2.helpers.parse_sh(s, **kwargs)

Do the same thing as anadama2.helpers.sh(), but do some extra interpreting and formatting of the shell command before handing it over to the shell. For formatting information, see anadama2.helpers.format_command(). :type s: str

anadama2.helpers.rm(to_rm, ignore_missing=True)

Remove files using os.remove().

Parameters:

• to_rm (str or list of str) – The filename or filenames to remove.
• ignore_missing (bool) – If one of the filenames isn’t a file, don’t raise an exception

anadama2.helpers.rm_r(to_rm, ignore_missing=True)

Recursively remove files and directories using shutil.rmtree().

Parameters:

• to_rm (str or list of str) – The filename or filenames to remove.
• ignore_missing (bool) – If one of the filenames isn’t a file, don’t raise an exception

anadama2.helpers.sh(s, log_command=True, **kwargs)

Execute a shell command. All further keywords are passed to subprocess.Popen

Parameters:s (str) – The command to execute. Passed directly to a shell, so be careful about doing things like sh('df -h > data; rm -rf /'); both commands are executed and bad things will happen.

anadama2.helpers.system(args_list, stdin=None, stdout=None, stdout_clobber=None, stderr=None, stderr_clobber=None, working_dir=None, **kwargs)

Execute a system call (no shell will be used). All further keywords are passed to subprocess.Popen

Parameters:

• args_list (list) – The argv to be passed to the system call.
• stdin (str) – If provided, the name of the file to open and send to the subprocess’ standard input. By default no data is sent to the process.
• stdout (str) – If provided, the name of the file to send output from the subprocess’ standard output. Standard output is appended to the file. By default all data from the subprocess standard out is sent to the standard out of the executing process
• stdout_clobber (str) – If provided, the name of the file to send output from the subprocess’ standard output. If the file already exists, it will be truncated before it receives writes.
• stderr (str) – If provided, the name of the file to send output from the subprocess’ standard error output. Standard error output is appended to the file. By default all data from the subprocess standard error is sent to the standard error of the executing process.
• stderr_clobber (str) – If provided, the name of the file to send output from the subprocess’ standard error output. If the file already exists, it will be truncated before it receives writes.