fireworks.user_objects.firetasks package — FireWorks 2.0.4 documentation

# fireworks.user_objects.firetasks package

## Subpackages

- fireworks.user_objects.firetasks.tests package
  - Submodules
  - fireworks.user_objects.firetasks.tests.test_dataflow_tasks module
    - `CommandLineTaskTest`
      - `CommandLineTaskTest.test_command_line_task_1()`
      - `CommandLineTaskTest.test_command_line_task_2()`
      - `CommandLineTaskTest.test_command_line_task_3()`
      - `CommandLineTaskTest.test_command_line_task_4()`
    - `ForeachTaskTest`
      - `ForeachTaskTest.test_foreach_commandlinetask()`
      - `ForeachTaskTest.test_foreach_pytask()`
    - `ImportDataTaskTest`
      - `ImportDataTaskTest.test_import_data_task()`
    - `JoinDictTaskTest`
      - `JoinDictTaskTest.test_join_dict_task()`
    - `JoinListTaskTest`
      - `JoinListTaskTest.test_join_list_task()`
    - `afunc()`
  - fireworks.user_objects.firetasks.tests.test_fileio_tasks module
    - `CompressDecompressArchiveDirTest`
      - `CompressDecompressArchiveDirTest.setUp()`
      - `CompressDecompressArchiveDirTest.tearDown()`
      - `CompressDecompressArchiveDirTest.test_archive_dir()`
      - `CompressDecompressArchiveDirTest.test_compress_dir()`
    - `FileWriteDeleteTest`
      - `FileWriteDeleteTest.test_init()`
      - `FileWriteDeleteTest.test_run()`
  - fireworks.user_objects.firetasks.tests.test_filepad_tasks module
    - `FilePadTasksTest`
      - `FilePadTasksTest.setUp()`
      - `FilePadTasksTest.tearDown()`
      - `FilePadTasksTest.test_addfilesfrompatterntask_run()`
      - `FilePadTasksTest.test_addfilestask_run()`
      - `FilePadTasksTest.test_deletefilestask_run()`
      - `FilePadTasksTest.test_getfilesbyquerytask_ignore_degenerate_file_name()`
      - `FilePadTasksTest.test_getfilesbyquerytask_ignore_empty_result_run()`
      - `FilePadTasksTest.test_getfilesbyquerytask_metafile_run()`
      - `FilePadTasksTest.test_getfilesbyquerytask_raise_degenerate_file_name()`
      - `FilePadTasksTest.test_getfilesbyquerytask_raise_empty_result_run()`
      - `FilePadTasksTest.test_getfilesbyquerytask_run()`
      - `FilePadTasksTest.test_getfilesbyquerytask_run_some_identifier()`
      - `FilePadTasksTest.test_getfilesbyquerytask_sort_ascending_name_run()`
      - `FilePadTasksTest.test_getfilesbyquerytask_sort_descending_name_run()`
      - `FilePadTasksTest.test_getfilestask_run()`
  - fireworks.user_objects.firetasks.tests.test_script_task module
    - `PyTaskTest`
      - `PyTaskTest.test_task()`
      - `PyTaskTest.test_task_auto_kwargs()`
      - `PyTaskTest.test_task_data_flow()`
    - `ScriptTaskTest`
      - `ScriptTaskTest.test_scripttask()`
    - `afunc()`
  - fireworks.user_objects.firetasks.tests.test_templatewriter_task module
    - `TemplateWriterTaskTest`
      - `TemplateWriterTaskTest.test_task()`
  - Module contents

## Submodules

## fireworks.user_objects.firetasks.dataflow_tasks module

This module includes dataflow firetask tasks.

class fireworks.user_objects.firetasks.dataflow_tasks.CommandLineTask(*args, **kwargs)

  Bases: `FiretaskBase`

  A Firetask to execute external commands in a shell.

  Required params:

    - command_spec (dict): a dictionary specification of the command (see below for details)

  Optional params:

    - inputs ([str]): list of labels, one for each input argument

    - outputs ([str]): list of labels, one for each output argument

    - chunk_number (int): the serial number of the firetask when it is part of a series generated by a ForeachTask

  command_spec = {

    ‘command’: [str], # mandatory label_1: dict_1, # optional label_2: dict_2, # optional …

  } The ‘command’ is a representation of the command as to be used with subprocess package. The optional keys label_1, label_2, etc. are the actual labels used in the inputs and outputs. The dictionaries dict_1, dict_2, etc. have the following schema: {

  > ‘binding’: {

  >   prefix: str or None, separator: str or None

  > }, ‘source’: {

  > > ‘type’: ‘path’ or ‘data’ or ‘identifier’

  > >   or ‘stdin’ or ‘stdout’ or ‘stderr’ or None,

  > > ‘value’: str or int or float

  > }, ‘target’: {

  > > ‘type’: ‘path’ or ‘data’ or ‘identifier’

  > >   or ‘stdin’ or ‘stdout’ or ‘stderr’ or None,

  > > ‘value’: str

  > }

  }

  ### Remarks

  - If the ‘type’ in the ‘source’ field is ‘data’ the ‘value’ can be of

  types ‘str’, ‘int’ and ‘float’.

  - When a str is found instead of dict for some ‘source’, for example

  ‘source’: ‘string’, ‘string’ is replaced with spec[‘string’] which must be available and of the schema of the ‘source’.

  - When a str is found instead of dict for some label, for example

  label: ‘string’, ‘string’ is replaced with spec[‘string’] which can be a dictionary with this schema or a list of such dictionaries.

  static command_line_tool(command: list[str], inputs: list[Any] | None = None, outputs: list[Any] | None = None) → list[dict]

    This function composes and executes a command from provided specifications.

    Required parameters:

      - command ([str]): the command as to be passed to subprocess.Popen

    Optional parameters:

      - inputs ([dict, [dict]]): list of the specifications for inputs; multiple inputs may be passed in one list of dictionaries

      - outputs ([dict]): list of the specifications for outputs

    Returns:

      > ‘target’: {
      >   ‘type’: ‘path’ or ‘data’ or ‘identifier’

      >     or ‘stdin’ or ‘stdout’ or ‘stderr’ or None

      >   ’value’: str

      > }

      If outputs is None then an empty list is returned.

    Return type:

      - list of target dictionaries for each output

  optional_params = ['inputs', 'outputs', 'chunk_number']
  required_params = ['command_spec']
  run_task(fw_spec: dict[str, Any]) → FWAction

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.dataflow_tasks.ForeachTask(*args, **kwargs)

  Bases: `FiretaskBase`

  This firetask branches the workflow creating parallel fireworks using FWAction: one firework for each element or each chunk from the split list. Each firework in this generated list contains the firetask specified in the task dictionary. If the number of chunks is specified the split list will be divided into this number of chunks and each chunk will be processed by one of the generated child fireworks.

  Required params:

    - task (dict): a dictionary version of the firetask

    - split (str): a label of an input list; it must be available both in the inputs list of the specified task and in the spec.

  Optional params:

    - number of chunks (int): if provided the split input list will be divided into this number of sublists and each will be processed by a separate child firework

  optional_params = ['number of chunks']
  required_params = ['task', 'split']
  run_task(fw_spec: dict[str, Any]) → FWAction

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.dataflow_tasks.ImportDataTask(*args, **kwargs)

  Bases: `FiretaskBase`

  Update the spec with data from file in a nested dictionary at a position specified by a mapstring = maplist[0]/maplist[1]/… i.e. spec[maplist[0]][maplist[1]]… = data.

  optional_params = []
  required_params = ['filename', 'mapstring']
  run_task(fw_spec: dict[str, Any]) → FWAction

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.dataflow_tasks.JoinDictTask(*args, **kwargs)

  Bases: `FiretaskBase`

  combines specified spec fields into a dictionary.

  optional_params = ['rename']
  required_params = ['inputs', 'output']
  run_task(fw_spec: dict[str, Any]) → FWAction

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.dataflow_tasks.JoinListTask(*args, **kwargs)

  Bases: `FiretaskBase`

  combines specified spec fields into a list.

  required_params = ['inputs', 'output']
  run_task(fw_spec: dict[str, Any]) → FWAction

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

## fireworks.user_objects.firetasks.fileio_tasks module

File I/O related firetasks for writing, deleting, transferring and archiving files.

class fireworks.user_objects.firetasks.fileio_tasks.ArchiveDirTask(*args, **kwargs)

  Bases: `FiretaskBase`

  Wrapper around shutil.make_archive to make tar archives.

  Parameters:

    - base_name (str) – Name of the file to create, including the path, minus any format-specific extension.

    - format (str) – Optional. one of “zip”, “tar”, “bztar” or “gztar”. Defaults to gztar.

  optional_params = ['format']
  required_params = ['base_name']
  run_task(fw_spec) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.CompressDirTask(*args, **kwargs)

  Bases: `FiretaskBase`

  Compress all files in a directory.

  Parameters:

    - dest (str) – Optional. Path to compress.

    - compression (str) – Optional. Can only be gz or bz2. Defaults to gz.

    - ignore_errors (bool) – Optional. Whether to ignore errors. Defaults to False.

  optional_params = ['compression', 'dest', 'ignore_errors']
  run_task(fw_spec) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.DecompressDirTask(*args, **kwargs)

  Bases: `FiretaskBase`

  Decompress all files in a directory. Autodetects gz, bz2 and z file extensions.

  Parameters:

    - dest (str) – Optional. Path to decompress.

    - ignore_errors (bool) – Optional. Whether to ignore errors. Defaults to False.

  optional_params = ['dest', 'ignore_errors']
  run_task(fw_spec) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.FileDeleteTask(*args, **kwargs)

  Bases: `FiretaskBase`

  A Firetask to delete files.

  Required params:

    - files_to_delete: ([str]) Filenames to delete

  Optional params:

    - dest: (str) Shared path for files

    - ignore_errors (bool): Whether to ignore errors. Defaults to True.

  optional_params = ['dest', 'ignore_errors']
  required_params = ['files_to_delete']
  run_task(fw_spec: dict[str, Any]) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.FileTransferTask(*args, **kwargs)

  Bases: `FiretaskBase`

  A Firetask to Transfer files. Note that.

  Required params:

    - mode: (str) - move, mv, copy, cp, copy2, copytree, copyfile, rtransfer

    - files: ([str]) or ([(str, str)]) - list of source files, or dictionary containing

        ‘src’ and ‘dest’ keys

  Optional params:

    - dest: (str) destination directory, if not specified within files parameter (else optional)

    - server: (str) server host for remote transfer

    - user: (str) user to authenticate with on remote server

    - key_filename: (str) optional SSH key location for remote transfer

    - max_retry: (int) number of times to retry failed transfers; defaults to 0 (no retries)

    - retry_delay: (int) number of seconds to wait between retries; defaults to 10

    - ignore_errors (bool): Optional. Whether to ignore errors. Defaults to False.

  fn_list = {'copy': <function copy>, 'copy2': <function copy2>, 'copyfile': <function copyfile>, 'copytree': <function copytree>, 'cp': <function copy>, 'move': <function move>, 'mv': <function move>}
  optional_params = ['dest', 'server', 'user', 'key_filename', 'max_retry', 'retry_delay', 'ignore_errors']
  required_params = ['mode', 'files']
  run_task(fw_spec: dict[str, Any]) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.fileio_tasks.FileWriteTask(*args, **kwargs)

  Bases: `FiretaskBase`

  A Firetask to write files.

  Required params:
    - files_to_write: ([{filename:(str), contents:(str)}]) List of dicts with filenames

        and contents

  Optional params:

    - dest: (str) Shared path for files

  optional_params = ['dest']
  required_params = ['files_to_write']
  run_task(fw_spec: dict[str, Any]) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

## fireworks.user_objects.firetasks.filepad_tasks module

Firetasks for interacting with FilePad: add, fetch, query, and delete files.

class fireworks.user_objects.firetasks.filepad_tasks.AddFilesTask(*args, **kwargs)

  Bases: `FiretaskBase`

  A Firetask to add files to the filepad.

  Required params:

    - paths (list/str): either list of paths or a glob pattern string.

  Optional params:

    - identifiers ([str]): list of identifiers, one for each file in the paths list

    - directory (str): path to directory where the pattern matching is to be done.

    - filepad_file (str): path to the filepad db config file

    - compress (bool): whether or not to compress the file before inserting to gridfs

    - metadata (dict): metadata to store along with the file, stored in ‘metadata’ key

  optional_params = ['identifiers', 'directory', 'filepad_file', 'compress', 'metadata']
  required_params = ['paths']
  run_task(fw_spec: dict[str, Any]) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.filepad_tasks.DeleteFilesTask(*args, **kwargs)

  Bases: `FiretaskBase`

  A Firetask to delete files from the filepad.

  Required params:

    - identifiers ([str]): identifiers of files to delete

  Optional params:

    - filepad_file (str): path to the filepad db config file

  optional_params = ['filepad_file']
  required_params = ['identifiers']
  run_task(fw_spec: dict[str, Any]) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.filepad_tasks.GetFilesByQueryTask(*args, **kwargs)

  Bases: `FiretaskBase`

  A Firetask to query files from the filepad and write them to specified directory (current working directory if not specified).

  Required params:

    - query (dict): mongo db query identifying files to fetch. Same as within fireworks.utilities.dict_mods, use ‘->’ in dict keys for querying nested documents, instead of MongoDB ‘.’ (dot) separator. Do use ‘.’ and NOT ‘->’ within the ‘sort_key’ field.

  Optional params:

    - sort_key (str): sort key, don’t sort per default

    - sort_direction (int): sort direction, default ‘pymongo.DESCENDING’

    - limit (int): maximum number of files to write, default: no limit

    - fizzle_empty_result (bool): fizzle if no file found, default: True

    - fizzle_degenerate_file_name (bool): fizzle if more than one of the resulting files are to be written to the same local file (i.e. the filepad’s ‘original_file_name’ entries overlap), default: True

    - filepad_file (str): path to the filepad db config file

    - dest_dir (str): destination directory, default is the current working directory.

    - new_file_names ([str]): if provided, the retrieved files will be renamed. Not recommended as order and number of queried files not fixed.

    - meta_file (bool): default: False. If True, then metadata of queried files written to a .yaml file of same name as file itself, suffixed by…

    - meta_file_suffix (str): if not None, metadata for each file is written to a YAML file of the same name, suffixed by this string. Default: “.meta.yaml”

  The options ‘fizzle_degenerate_file_name’, ‘limit’, ‘sort_key’, and ‘sort_direction’ are all inntended to help dealing with the following special case: Querying by metadata leads to an a priori unknown number of files in the general case. Thus, it is advisable to either ‘limit’ the number of files and/or avoid explicitly specifying a list of ‘new_file_names’. In the latter case, files will be written to ‘dest_dir’ using their ‘original_file_name’ recorded within the attached FilePad object. When more than one queried file share the same ‘original_file_name’, the order of processing matters: subsequent files will overwrite their predecessor of same name. ‘sort_key’ and ‘sort_direction’ can help to assure deterministic behavior, e.g. by always processing newer files later.

  optional_params = ['dest_dir', 'filepad_file', 'fizzle_degenerate_file_name', 'fizzle_empty_result', 'limit', 'meta_file', 'meta_file_suffix', 'new_file_names', 'sort_direction', 'sort_key']
  required_params = ['query']
  run_task(fw_spec: dict[str, Any]) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.filepad_tasks.GetFilesTask(*args, **kwargs)

  Bases: `FiretaskBase`

  A Firetask to fetch files from the filepad and write it to specified directory (current working directory if not specified).

  Required params:

    - identifiers ([str]): identifiers of files to fetch

  Optional params:

    - filepad_file (str): path to the filepad db config file

    - dest_dir (str): destination directory, default is the current working directory

    - new_file_names ([str]): if provided, the retrieved files will be renamed

  optional_params = ['filepad_file', 'dest_dir', 'new_file_names']
  required_params = ['identifiers']
  run_task(fw_spec: dict[str, Any]) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

fireworks.user_objects.firetasks.filepad_tasks.get_fpad(fpad_file: str | None) → FilePad

  Return a FilePad instance from file if provided, else auto-load one.

## fireworks.user_objects.firetasks.script_task module

This module includes tasks to integrate scripts and python functions.

class fireworks.user_objects.firetasks.script_task.PyTask(*args, **kwargs)

  Bases: `FiretaskBase`

  Runs any python function! Extremely powerful, which allows you to essentially run any accessible method on the system. The optional inputs and outputs lists may contain spec keys to add to args list and to make the function output available in the current and in children fireworks.

  Required parameters:
    - func (str): Fully qualified python method. E.g., json.dump, or shutil

        .copy, or some other function that is not part of the standard library!

  Optional parameters:

    - args (list): List of args. Default is empty.

    - kwargs (dict): Dictionary of keyword args. Default is empty.

    - auto_kwargs (bool): If True, all other params not starting with ‘_’ are supplied as keyword args

    - stored_data_varname (str): Whether to store the output in FWAction. If this is a string that does not evaluate to False, the output of the function will be stored as FWAction(stored_data={stored_data_varname: output}). The name is deliberately long to avoid potential name conflicts.

    - inputs ([str]): a list of keys in spec which will be used as inputs; the generated arguments list will be appended to args

    - outputs ([str]): a list of spec keys that will be used to pass the function’s outputs to child fireworks

    - chunk_number (int): a serial number of the Firetask within a group of Firetasks generated by a ForeachTask

  other_params = ['args', 'kwargs', 'auto_kwargs', 'stored_data_varname', 'inputs', 'outputs', 'chunk_number']
  required_params = ['func']
  run_task(fw_spec: dict[str, list[int] | int]) → FWAction | None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

class fireworks.user_objects.firetasks.script_task.ScriptTask(*args, **kwargs)

  Bases: `FiretaskBase`

  Runs a user-defined script.

  classmethod from_str(shell_cmd, parameters=None)
  required_params = ['script']
  run_task(fw_spec)

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

## fireworks.user_objects.firetasks.templatewriter_task module

This module contains the TemplateWriterTask, which writes files based on a template file and a Context using Jinja2’s templating engine.

class fireworks.user_objects.firetasks.templatewriter_task.TemplateWriterTask(*args, **kwargs)

  Bases: `FiretaskBase`

  Task to write templated files via Jinja2 library Required parameters:

  > - template_file: (str) - path to template file

  > - context: (dict) - variable replacements for the template file

  > - output_file: (str) - output file

  Optional parameters:

    - append: (bool) - append to output file (instead of overwrite)

    - template_dir: (str) - directory in which to find the template file.

  run_task(fw_spec) → None

    This method gets called when the Firetask is run. It can take in a Firework spec, perform some task using that data, and then return an output in the form of a FWAction.

    Parameters:

      fw_spec (dict) – A Firework spec. This comes from the master spec. In addition, this spec contains a special “_fw_env” key that contains the env settings of the FWorker calling this method. This provides for abstracting out certain commands or settings. For example, “foo” may be named “foo1” in resource 1 and “foo2” in resource 2. The FWorker env can specify { “foo”: “foo1”}, which maps an abstract variable “foo” to the relevant “foo1” or “foo2”. You can then write a task that uses fw_spec[“_fw_env”][“foo”] that will work across all these multiple resources.

    Returns:

      (FWAction)

## fireworks.user_objects.firetasks.unittest_tasks module

Helpers used in unit tests to validate serialization behavior.

class fireworks.user_objects.firetasks.unittest_tasks.ExportTestSerializer(a: Any)

  Bases: `FWSerializable`

  Serializer to test exporting with explicit _fw_name entries.

  __init__(a: Any) → None

    Construct with a generic payload.

  classmethod from_dict(m_dict: dict[str, Any]) → ExportTestSerializer

    Recreate instance from a serialized dict.

  to_dict() → dict[str, Any]

    Serialize to a dict including the FireWorks name.

class fireworks.user_objects.firetasks.unittest_tasks.UnitTestSerializer(a: Any, m_date: datetime)

  Bases: `FWSerializable`

  Serializable object used for testing round-trip serialization.

  __init__(a: Any, m_date: datetime) → None

    Construct with a generic payload and a datetime stamp.

  classmethod from_dict(m_dict: dict[str, Any]) → UnitTestSerializer

    Recreate instance from a serialized dict.

  to_dict(*args, **kwargs)

## Module contents

Collection of built-in Firetasks for common workflows.