adaptivemd.Task

class adaptivemd.Task(generator=None)[source]

A description for a task running on an HPC

Variables:
  • worker (WorkingScheduler) – the currently assigned Worker instance (not the scheduler!)
  • generator (TaskGenerator) – if given the TaskGenerator that was used to create this task
  • state (str) – a string representing the current state of the execution. One of - ‘create’ : task has been created and is available for execution - ‘running’: task is currently executed by a scheduler - ‘queued’ : task has been captured by a worker for execution - ‘fail’ : task has completed but failed. You can restart it - ‘succedd` : task has completed and succeeded. - ‘halt’ : task has been halted by user. You can restart it - ‘cancelled’ : task has been cancelled by user. You CANNOT restart it
  • stdout (LogEntry) – After completion you can access the stdout of the task here
  • stderr (LogEntry) – After completion you can access the stderr of the task here
__init__(generator=None)[source]

Methods

__init__([generator])
add_cb(event, cb) Add a custom callback
add_conda_env(name) Add loading a conda env to all tasks of this resource
add_files(files) Add additional files to the task execution
add_path(path)
param path:a (list of) path(s) to be added to the $PATH variable before task execution
append(cmd) Append a command to this task
args() Return a list of args of the __init__ function of a class
base() Return the most parent class actually derived from StorableMixin
cancel() Mark a task as cancelled if it it not running or has been halted
descendants() Return a list of all subclassed objects
fire(event, scheduler) Fire an event like success or failed.
from_dict(dct)
get(f[, name]) Get a file and make it available to the task in the main directory
get_uuid() Create a new unique ID
has_failed() Check if the task is done executing and has failed
idx(store) Return the index which is used for the object in the given store.
is_done() Check if the task is done executing.
link(f[, name]) Add an action to create a link to a file (under a new name)
named(name) Attach a .name property to an object
objects() Returns a dictionary of all storable objects
prepend(cmd) Append a command to this task
put(f, target) Put a file back and make it persistent
remove(f) Add an action to remove a file or folder
restart() Mark a task as being runnable if it was stopped or failed before
setenv(key, value) Set an environment variable for the task
to_dict()
touch(f) Add an action to create an empty file or folder at a given location
was_successful() Check if the task is done executing and was successful

Attributes

ACTIVE_LONG
CREATION_COUNT
FINAL_STATES
INSTANCE_UUID
RESTARTABLE_STATES
RUNNABLE_STATES
additional_files list of Location
base_cls Return the base class
base_cls_name Return the name of the base class
cls Return the class name as a string
dependency_okay Check if all dependency tasks are successful
description Return a lengthy description of the task for debugging and information
environment dict str – str
main list of str or Action
modified_files A set of all input files whose names match output names and hence will be overwritten
new_files Return a set of all files the will be newly created by this task
pre_add_paths list of str
pre_exec list of str or Action
ready Check if this task is ready to be executed
script list of str or Action
source_locations Return a set of all required file urls
sources Return a set of all required input files
staged_files Set of all staged files by the tasks generator
state
stderr
stdout
target_locations Return a set of all new and overwritten file urls
targets Return a set of all new and overwritten files
unstaged_input_files Return a set of File objects that are used but are not part of the generator stage
worker
restart()[source]

Mark a task as being runnable if it was stopped or failed before

cancel()[source]

Mark a task as cancelled if it it not running or has been halted

dependency_okay

Check if all dependency tasks are successful

Returns:True if all dependencies are fulfilled
Return type:bool
ready

Check if this task is ready to be executed

Usually this only checks dependencies but might involve more elaborate checks for specific Task classes

Returns:if True the task can now be executed
Return type:bool
description

Return a lengthy description of the task for debugging and information

Returns:the information text
Return type:str
fire(event, scheduler)[source]

Fire an event like success or failed.

Notes

You should never have to call this yourself. The scheduler does that.

Parameters:
  • event (str) – the events name like fail, success, submit
  • scheduler (Scheduler) – the scheduler that issued the events to be fired
is_done()[source]

Check if the task is done executing. Can be failed, successful or cancelled

Returns:True if the task has finished its execution
Return type:bool
was_successful()[source]

Check if the task is done executing and was successful

Returns:True if the task has finished successfully
Return type:bool
has_failed()[source]

Check if the task is done executing and has failed

Returns:True if the task has finished but failed
Return type:bool
add_cb(event, cb)[source]

Add a custom callback

Parameters:
  • event (str) – name of the event to be called upon firing
  • cb (function) – the function to be called. It must be a function that takes a task and a scheduler
additional_files

list of Location return the list of files created other than taken care of by actions. Should usually not be necessary. If you do some bad hacks with the bash you can add files that you transferred yourself to the project folders.

add_files(files)[source]

Add additional files to the task execution

Should usually not be necessary. If you do some bad hacks with the bash you can add files that you transferred yourself to the project folders.

Parameters:files (list of File) – the list of files to be added to the task
targets

Return a set of all new and overwritten files

Returns:the list of files that are created or overwritten by this task
Return type:set of File
target_locations

Return a set of all new and overwritten file urls

Returns:the list of file urls that are created or overwritten by this task
Return type:set of str
sources

Return a set of all required input files

Returns:the list of files that are required by this task
Return type:set of File
source_locations

Return a set of all required file urls

Returns:the list of file urls that are required by this task
Return type:set of str
new_files

Return a set of all files the will be newly created by this task

Returns:the set of files that are created by this task
Return type:set of File
modified_files

A set of all input files whose names match output names and hence will be overwritten

Returns:the list of potentially overwritten input files
Return type:list of File
staged_files

Set of all staged files by the tasks generator

Returns:files that are staged by the tasks generator
Return type:set of File

Notes

There might be more files stages by other generators

unstaged_input_files

Return a set of File objects that are used but are not part of the generator stage

Usually a task requires some reused files from staging and specific others. This function lists all the files that this task will stage to its working directory but will not be available from the set of staged files of the tasks generator

Returns:the set of File objects that are needed and not staged
Return type:set of File
setenv(key, value)[source]

Set an environment variable for the task

Parameters:
  • key (str) –
  • value (str) –
append(cmd)[source]

Append a command to this task

prepend(cmd)[source]

Append a command to this task

get(f, name=None)[source]

Get a file and make it available to the task in the main directory

Parameters:
  • f (File) –
  • name (Location or str) –
Returns:

the file instance of the file to be created in the unit

Return type:

File

touch(f)[source]

Add an action to create an empty file or folder at a given location

Parameters:f (Location) – the location (file or folder) to be used

Add an action to create a link to a file (under a new name)

Parameters:
  • f (Location) – the source location (file or folder) to be used
  • name (Location or str) – the target location to be used. For source files and target folders the basename is copied
Returns:

the actual target location

Return type:

Location

put(f, target)[source]

Put a file back and make it persistent

Corresponds to output_staging

Parameters:
  • f (File) – the file to be used
  • target (str or File) – the target location. Need to contain a URL like staging:// or file:// for application side files
Returns:

the actual target location

Return type:

Location

remove(f)[source]

Add an action to remove a file or folder

Parameters:f (File) – the location to be removed
Returns:the actual location
Return type:Location
add_conda_env(name)[source]

Add loading a conda env to all tasks of this resource

This calls resource.wrapper.append(‘source activate {name}’) :param name: name of the conda environment :type name: str