autosubmit — autosubmit 3.9.1 documentation

# autosubmit

class `autosubmit.autosubmit.` `Autosubmit`

  Interface class for autosubmit.

  static `archive`(expid, clean=True, compress=True)

    Archives an experiment: call clean (if experiment is of version 3 or later), compress folder to tar.gz and moves to year’s folder

    Parameters: | clean,compress – expid (str) – experiment identifier
    Returns: |

  static `change_status`(final, final_status, job, save)

    Set job status to final

    Parameters: | final – final_status – job –

  static `check`(experiment_id, notransitive=False)

    Checks experiment configuration and warns about any detected error or inconsistency.

    Parameters: | experiment_id (str) – experiment identifier:

  static `clean`(expid, project, plot, stats, create_log_file=True)

    Clean experiment’s directory to save storage space. It removes project directory and outdated plots or stats.

    Parameters: | create_log_file (bool) – if true, creates log file expid (str) – identifier of experiment to clean project (bool) – set True to delete project directory plot (bool) – set True to delete outdated plots stats (bool) – set True to delete outdated stats

  static `configure`(advanced, database_path, database_filename, local_root_path, platforms_conf_path, jobs_conf_path, smtp_hostname, mail_from, machine, local)

    Configure several paths for autosubmit: database, local root and others. Can be configured at system, user or local levels. Local level configuration precedes user level and user level precedes system configuration.

    Parameters: | database_path (str) – path to autosubmit database database_filename (str) – database filename local_root_path (str) – path to autosubmit’s experiments’ directory platforms_conf_path (str) – path to platforms conf file to be used as model for new experiments jobs_conf_path (str) – path to jobs conf file to be used as model for new experiments machine (bool) – True if this configuration has to be stored for all the machine users local (bool) – True if this configuration has to be stored in the local path mail_from (str) – smtp_hostname (str) –

  static `configure_dialog`()

    Configure several paths for autosubmit interactively: database, local root and others. Can be configured at system, user or local levels. Local level configuration precedes user level and user level precedes system configuration.

  static `create`(expid, noplot, hide, output='pdf', group_by=None, expand=[], expand_status=[], notransitive=False, check_wrappers=False, detail=False)

    Creates job list for given experiment. Configuration files must be valid before realizing this process.

    Parameters: | expid (str) – experiment identifier noplot – if True, method omits final plotting of the jobs list. Only needed on large experiments when

    plotting time can be much larger than creation time. :type noplot: bool :return: True if successful, False if not :rtype: bool :param hide: hides plot window :type hide: bool :param hide: hides plot window :type hide: bool :param output: plot’s file format. It can be pdf, png, ps or svg :type output: str

  static `delete`(expid, force)

    Deletes and experiment from database and experiment’s folder

    Parameters: | expid (str) – identifier of the experiment to delete force (bool) – if True, does not ask for confirmation
    Returns: | True if succesful, False if not
    Return type: | bool

  static `describe`(experiment_id)

    Show details for specified experiment

    Parameters: | experiment_id (str) – experiment identifier:

  static `expid`(hpc, description, copy_id='', dummy=False, test=False, operational=False, root_folder='')

    Creates a new experiment for given HPC

    Parameters: | operational (bool) – if true, creates an operational experiment hpc (str) – name of the main HPC for the experiment description (str) – short experiment’s description. copy_id (str) – experiment identifier of experiment to copy dummy (bool) – if true, writes a default dummy configuration for testing test – if true, creates an experiment for testing
    Returns: | experiment identifier. If method fails, returns ‘’.
    Return type: | str

  static `generate_scripts_andor_wrappers`(as_conf, job_list, jobs_filtered, packages_persistence, only_wrappers=False)

    Parameters: | as_conf (AutosubmitConfig() Object) – Class that handles basic configuration parameters of Autosubmit. job_list (JobList() Object) – Representation of the jobs of the experiment, keeps the list of jobs inside. jobs_filtered (List() of Job Objects) – list of jobs that are relevant to the process. packages_persistence (JobPackagePersistence() Object) – Object that handles local db persistence. only_wrappers (Boolean) – True when coming from Autosubmit.create(). False when coming from Autosubmit.inspect(),
    Returns: | Nothing
    Return type: |

  static `inspect`(expid, lst, filter_chunks, filter_status, filter_section, notransitive=False, force=False, check_wrapper=False)

    Generates cmd files experiment.

    Parameters: | expid (str) – identifier of experiment to be run
    Returns: | True if run to the end, False otherwise
    Return type: | bool

  static `install`()

    Creates a new database instance for autosubmit at the configured path

  static `migrate`(experiment_id, offer, pickup)

    Migrates experiment files from current to other user. It takes mapping information for new user from config files.

    Parameters: | experiment_id – experiment identifier: pickup – offer –

  static `monitor`(expid, file_format, lst, filter_chunks, filter_status, filter_section, hide, txt_only=False, group_by=None, expand=[], expand_status=[], hide_groups=False, notransitive=False, check_wrapper=False, txt_logfiles=False, detail=False)

    Plots workflow graph for a given experiment with status of each job coded by node color. Plot is created in experiment’s plot folder with name <expid>_<date>_<time>.<file_format>

    Parameters: | expid (str) – identifier of the experiment to plot file_format (str) – plot’s file format. It can be pdf, png, ps or svg lst (str) – list of jobs to change status filter_chunks (str) – chunks to change status filter_status (str) – current status of the jobs to change status filter_section (str) – sections to change status hide (bool) – hides plot window

  static `parse_args`()

    Parse arguments given to an executable and start execution of command given

  static `recovery`(expid, noplot, save, all_jobs, hide, group_by=None, expand=[], expand_status=[], notransitive=False, no_recover_logs=False, detail=False)

    Method to check all active jobs. If COMPLETED file is found, job status will be changed to COMPLETED, otherwise it will be set to WAITING. It will also update the jobs list.

    Parameters: | expid (str) – identifier of the experiment to recover save (bool) – If true, recovery saves changes to the jobs list all_jobs (bool) – if True, it tries to get completed files for all jobs, not only active. hide (bool) – hides plot window

  static `refresh`(expid, model_conf, jobs_conf)

    Refresh project folder for given experiment

    Parameters: | model_conf (bool) – jobs_conf (bool) – expid (str) – experiment identifier

  static `rerun_recovery`(expid, job_list, rerun_list, as_conf)

    Method to check all active jobs. If COMPLETED file is found, job status will be changed to COMPLETED, otherwise it will be set to WAITING. It will also update the jobs list.

    Parameters: | expid (str) – identifier of the experiment to recover save (bool) – If true, recovery saves changes to the jobs list all_jobs (bool) – if True, it tries to get completed files for all jobs, not only active. hide (bool) – hides plot window

  static `run_experiment`(expid, notransitive=False, update_version=False)

    Runs and experiment (submitting all the jobs properly and repeating its execution in case of failure).

    Parameters: | expid (str) – identifier of experiment to be run
    Returns: | True if run to the end, False otherwise
    Return type: | bool

  static `set_status`(expid, noplot, save, final, lst, filter_chunks, filter_status, filter_section, filter_type_chunk, hide, group_by=None, expand=[], expand_status=[], notransitive=False, check_wrapper=False, detail=False)

    Set status

    Parameters: | expid (str) – experiment identifier save (bool) – if true, saves the new jobs list final (str) – status to set on jobs lst (str) – list of jobs to change status filter_chunks (str) – chunks to change status filter_status (str) – current status of the jobs to change status filter_section (str) – sections to change status hide (bool) – hides plot window

  static `statistics`(expid, filter_type, filter_period, file_format, hide, notransitive=False)

    Plots statistics graph for a given experiment. Plot is created in experiment’s plot folder with name <expid>_<date>_<time>.<file_format>

    Parameters: | expid (str) – identifier of the experiment to plot filter_type – type of the jobs to plot filter_period – period to plot file_format (str) – plot’s file format. It can be pdf, png, ps or svg hide (bool) – hides plot window

  static `submit_ready_jobs`(as_conf, job_list, platforms_to_test, packages_persistence, inspect=False, only_wrappers=False, hold=False)

    Gets READY jobs and send them to the platforms if there is available space on the queues

    Parameters: | as_conf (AutosubmitConfig object) – autosubmit config object job_list (JobList object) – job list to check platforms_to_test (set of Platform Objects, e.g. SgePlatform(), LsfPlatform()) – platforms used packages_persistence (JobPackagePersistence object) – Handles database per experiment. inspect (Boolean) – True if coming from generate_scripts_andor_wrappers(). only_wrappers (Boolean) – True if it comes from create -cw, False if it comes from inspect -cw.
    Returns: | True if at least one job was submitted, False otherwise
    Return type: | Boolean

  static `test`(expid, chunks, member=None, start_date=None, hpc=None, branch=None)

    Method to conduct a test for a given experiment. It creates a new experiment for a given experiment with a given number of chunks with a random start date and a random member to be run on a random HPC.

    Parameters: | expid (str) – experiment identifier chunks (int) – number of chunks to be run by the experiment member (str) – member to be used by the test. If None, it uses a random one from which are defined on the experiment. start_date (str) – start date to be used by the test. If None, it uses a random one from which are defined on the experiment. hpc (str) – HPC to be used by the test. If None, it uses a random one from which are defined on the experiment. branch (str) – branch or revision to be used by the test. If None, it uses configured branch.
    Returns: | True if test was succesful, False otherwise
    Return type: | bool

  static `testcase`(copy_id, description, chunks=None, member=None, start_date=None, hpc=None, branch=None)

    Method to create a test case. It creates a new experiment whose id starts by ‘t’.

    Parameters: | copy_id (str) – experiment identifier description (str) – test case experiment description chunks (int) – number of chunks to be run by the experiment. If None, it uses configured chunk(s). member (str) – member to be used by the test. If None, it uses configured member(s). start_date (str) – start date to be used by the test. If None, it uses configured start date(s). hpc (str) – HPC to be used by the test. If None, it uses configured HPC. branch (str) – branch or revision to be used by the test. If None, it uses configured branch.
    Returns: | test case id
    Return type: | str

  static `unarchive`(experiment_id, compress=True, overwrite=False)

    Unarchives an experiment: uncompress folder from tar.gz and moves to experiments root folder

    Parameters: | experiment_id (str) – experiment identifier

  static `update_version`(expid)

    Refresh experiment version with the current autosubmit version :param expid: experiment identifier :type expid: str

`autosubmit.autosubmit.` `signal_handler`(signal_received, frame)

  Used to handle interrupt signals, allowing autosubmit to clean before exit

  Parameters: | signal_received – frame –

`autosubmit.autosubmit.` `signal_handler_create`(signal_received, frame)

  Used to handle KeyboardInterrumpt signals while the create method is being executed

  Parameters: | signal_received – frame –