adaptivemd.Task¶
-
class
adaptivemd.
Task
(generator=None)[source]¶ A description for a task running on an HPC
Variables: - worker (
WorkingScheduler
) – the currently assigned Worker instance (not the scheduler!) - generator (
TaskGenerator
) – if given theTaskGenerator
that was used to create this task - state (str) – a string representing the current state of the execution. One of - ‘create’ : task has been created and is available for execution - ‘running’: task is currently executed by a scheduler - ‘queued’ : task has been captured by a worker for execution - ‘fail’ : task has completed but failed. You can restart it - ‘succedd` : task has completed and succeeded. - ‘halt’ : task has been halted by user. You can restart it - ‘cancelled’ : task has been cancelled by user. You CANNOT restart it
- stdout (
LogEntry
) – After completion you can access the stdout of the task here - stderr (
LogEntry
) – After completion you can access the stderr of the task here
Methods
__init__
([generator])x.__init__(…) initializes x; see help(type(x)) for signature add_cb
(event, cb)Add a custom callback add_conda_env
(name)Add loading a conda env to all tasks of this resource add_files
(files)Add additional files to the task execution add_path
(path)param path: a (list of) path(s) to be added to the $PATH variable before task execution append
(cmd)Append a command to this task args
()Return a list of args of the __init__ function of a class base
()Return the most parent class actually derived from StorableMixin cancel
()Mark a task as cancelled if it it not running or has been halted descendants
()Return a list of all subclassed objects fire
(event, scheduler)Fire an event like success or failed. from_dict
(dct)Reconstruct an object from a dictionary representation get
(f[, name])Get a file and make it available to the task in the main directory get_uuid
()Create a new unique ID :returns: the unique number for an object in the project :rtype: long has_failed
()Check if the task is done executing and has failed idx
(store)Return the index which is used for the object in the given store. is_done
()Check if the task is done executing. link
(f[, name])Add an action to create a link to a file (under a new name) named
(name)Attach a .name property to an object objects
()Returns a dictionary of all storable objects prepend
(cmd)Append a command to this task put
(f, target)Put a file back and make it persistent remove
(f)Add an action to remove a file or folder restart
()Mark a task as being runnable if it was stopped or failed before setenv
(key, value)Set an environment variable for the task to_dict
()Convert object into a dictionary representation touch
(f)Add an action to create an empty file or folder at a given location was_successful
()Check if the task is done executing and was successful Attributes
ACTIVE_LONG
CREATION_COUNT
FINAL_STATES
INSTANCE_UUID
RESTARTABLE_STATES
RUNNABLE_STATES
additional_files
list of Location return the list of files created other than taken care of by actions. base_cls
Return the base class base_cls_name
Return the name of the base class cls
Return the class name as a string dependency_okay
Check if all dependency tasks are successful description
Return a lengthy description of the task for debugging and information environment
dict str – str the dict of environment variables and their assigned value main
list of str or Action the main part of the script modified_files
A set of all input files whose names match output names and hence will be overwritten new_files
Return a set of all files the will be newly created by this task pre_add_paths
list of str the list of added paths to the $PATH variable by this task pre_exec
list of str or Action the list of actions to be run before the main script. ready
Check if this task is ready to be executed script
list of str or Action the full script of this task. source_locations
Return a set of all required file urls sources
Return a set of all required input files staged_files
Set of all staged files by the tasks generator state
stderr
stdout
target_locations
Return a set of all new and overwritten file urls targets
Return a set of all new and overwritten files unstaged_input_files
Return a set of File objects that are used but are not part of the generator stage worker
-
dependency_okay
¶ Check if all dependency tasks are successful
Returns: True if all dependencies are fulfilled Return type: bool
-
ready
¶ Check if this task is ready to be executed
Usually this only checks dependencies but might involve more elaborate checks for specific Task classes
Returns: if True the task can now be executed Return type: bool
-
description
¶ Return a lengthy description of the task for debugging and information
Returns: the information text Return type: str
-
fire
(event, scheduler)[source]¶ Fire an event like success or failed.
Notes
You should never have to call this yourself. The scheduler does that.
Parameters: - event (str) – the events name like fail, success, submit
- scheduler (Scheduler) – the scheduler that issued the events to be fired
-
is_done
()[source]¶ Check if the task is done executing. Can be failed, successful or cancelled
Returns: True if the task has finished its execution Return type: bool
-
was_successful
()[source]¶ Check if the task is done executing and was successful
Returns: True if the task has finished successfully Return type: bool
-
has_failed
()[source]¶ Check if the task is done executing and has failed
Returns: True if the task has finished but failed Return type: bool
-
add_cb
(event, cb)[source]¶ Add a custom callback
Parameters: - event (str) – name of the event to be called upon firing
- cb (function) – the function to be called. It must be a function that takes a task and a scheduler
-
additional_files
¶ list of Location return the list of files created other than taken care of by actions. Should usually not be necessary. If you do some bad hacks with the bash you can add files that you transferred yourself to the project folders.
-
add_files
(files)[source]¶ Add additional files to the task execution
Should usually not be necessary. If you do some bad hacks with the bash you can add files that you transferred yourself to the project folders.
Parameters: files (list of File) – the list of files to be added to the task
-
targets
¶ Return a set of all new and overwritten files
Returns: the list of files that are created or overwritten by this task Return type: set of File
-
target_locations
¶ Return a set of all new and overwritten file urls
Returns: the list of file urls that are created or overwritten by this task Return type: set of str
-
sources
¶ Return a set of all required input files
Returns: the list of files that are required by this task Return type: set of File
-
source_locations
¶ Return a set of all required file urls
Returns: the list of file urls that are required by this task Return type: set of str
-
new_files
¶ Return a set of all files the will be newly created by this task
Returns: the set of files that are created by this task Return type: set of File
-
modified_files
¶ A set of all input files whose names match output names and hence will be overwritten
Returns: the list of potentially overwritten input files Return type: list of File
-
staged_files
¶ Set of all staged files by the tasks generator
Returns: files that are staged by the tasks generator Return type: set of File Notes
There might be more files stages by other generators
-
unstaged_input_files
¶ Return a set of File objects that are used but are not part of the generator stage
Usually a task requires some reused files from staging and specific others. This function lists all the files that this task will stage to its working directory but will not be available from the set of staged files of the tasks generator
Returns: the set of File objects that are needed and not staged Return type: set of File
-
setenv
(key, value)[source]¶ Set an environment variable for the task
Parameters: - key (str) –
- value (str) –
-
get
(f, name=None)[source]¶ Get a file and make it available to the task in the main directory
Parameters: - f (File) –
- name (Location or str) –
Returns: the file instance of the file to be created in the unit
Return type: File
-
touch
(f)[source]¶ Add an action to create an empty file or folder at a given location
Parameters: f (Location) – the location (file or folder) to be used
-
link
(f, name=None)[source]¶ Add an action to create a link to a file (under a new name)
Parameters: - f (Location) – the source location (file or folder) to be used
- name (Location or str) – the target location to be used. For source files and target folders the basename is copied
Returns: the actual target location
Return type: Location
-
put
(f, target)[source]¶ Put a file back and make it persistent
Corresponds to output_staging
Parameters: - f (File) – the file to be used
- target (str or File) – the target location. Need to contain a URL like staging:// or file:// for application side files
Returns: the actual target location
Return type: Location
- worker (