autosubmit.config — autosubmit 3.9.1 documentation

# autosubmit.config

## autosubmit.config.basicConfig

class `autosubmit.config.basicConfig.` `BasicConfig`

  Class to manage configuration for autosubmit path, database and default values for new experiments

  static `read`()

    Reads configuration from .autosubmitrc files, first from /etc, then for user directory and last for current path.

## autosubmit.config.config_common

class `autosubmit.config.config_common.` `AutosubmitConfig`(expid, basic_config, parser_factory)

  Bases: `object`

  Class to handle experiment configuration coming from file or database

  Parameters: | expid (str) – experiment identifier

  `check_autosubmit_conf`()

    Checks experiment’s autosubmit configuration file.

    Returns: | True if everything is correct, False if it founds any error
    Return type: | bool

  `check_conf_files`()

    Checks configuration files (autosubmit, experiment jobs and platforms), looking for invalid values, missing required options. Prints results in log

    Returns: | True if everything is correct, False if it finds any error
    Return type: | bool

  `check_expdef_conf`()

    Checks experiment’s experiment configuration file.

    Returns: | True if everything is correct, False if it founds any error
    Return type: | bool

  `check_jobs_conf`()

    Checks experiment’s jobs configuration file.

    Returns: | True if everything is correct, False if it founds any error
    Return type: | bool

  `check_platforms_conf`()

    Checks experiment’s queues configuration file.

    Returns: | True if everything is correct, False if it founds any error
    Return type: | bool

  `check_proj`()

    Checks project config file

    Returns: | True if everything is correct, False if it founds any error
    Return type: | bool

  `check_proj_file`()

    Add a section header to the project’s configuration file (if not exists)

  `experiment_file`

    Returns experiment’s config file name

  `get_chunk_ini`(default=1)

    Returns the first chunk from where the experiment will start

    Parameters: | default –
    Returns: | initial chunk
    Return type: | int

  `get_chunk_list`()

    Returns chunk list from experiment’s config file

    Returns: | experiment’s chunks
    Return type: | list

  `get_chunk_size_unit`()

    Unit for the chunk length

    Returns: | Unit for the chunk length Options: {hour, day, month, year}
    Return type: | str

  `get_communications_library`()

    Returns the communications library from autosubmit’s config file. Paramiko by default.

    Returns: | communications library
    Return type: | str

  `get_copy_remote_logs`()

    Returns if the user has enabled the logs local copy from autosubmit’s config file

    Returns: | if logs local copy
    Return type: | bool

  `get_current_host`(section)

    Returns the user to be changed from platform config file.

    Returns: | migrate user to
    Return type: | str

  `get_current_project`(section)

    Returns the project to be changed from platform config file.

    Returns: | migrate user to
    Return type: | str

  `get_current_user`(section)

    Returns the user to be changed from platform config file.

    Returns: | migrate user to
    Return type: | str

  `get_custom_directives`(section)

    Gets custom directives needed for the given job type :param section: job type :type section: str :return: custom directives needed :rtype: str

  `get_date_list`()

    Returns startdates list from experiment’s config file

    Returns: | experiment’s startdates
    Return type: | list

  `get_default_job_type`()

    Returns the default job type from experiment’s config file

    Returns: | default type such as bash, python, r..
    Return type: | str

  `get_fetch_single_branch`()

    Returns fetch single branch from experiment’s config file Default is -single-branch :return: fetch_single_branch(Y/N) :rtype: boolean

  `get_file_jobs_conf`()

    Returns path to project config file from experiment config file

    Returns: | path to project config file
    Return type: | str

  `get_file_project_conf`()

    Returns path to project config file from experiment config file

    Returns: | path to project config file
    Return type: | str

  `get_git_project_branch`()

    Returns git branch from experiment’s config file

    Returns: | git branch
    Return type: | str

  `get_git_project_commit`()

    Returns git commit from experiment’s config file

    Returns: | git commit
    Return type: | str

  `get_git_project_origin`()

    Returns git origin from experiment config file

    Returns: | git origin
    Return type: | str

  `get_git_remote_project_root`()

    Returns remote machine ROOT PATH

    Returns: | git commit
    Return type: | str

  `get_jobs_sections`()

    Returns the list of sections defined in the jobs config file

    Returns: | sections
    Return type: | list

  `get_local_project_path`()

    Gets path to origin for local project

    Returns: | path to local project
    Return type: | str

  `get_mails_to`()

    Returns the address where notifications will be sent from autosubmit’s config file

    Returns: | mail address
    Return type: | [str]

  `get_max_processors`()

    Returns max processors from autosubmit’s config file

    Return type: | str

  `get_max_waiting_jobs`()

    Returns max number of waiting jobs from autosubmit’s config file

    Returns: | main platforms
    Return type: | int

  `get_max_wallclock`()

    Returns max wallclock

    Return type: | str

  `get_max_wrapped_jobs`()

    Returns the maximum number of jobs that can be wrapped together as configured in autosubmit’s config file

    Returns: | maximum number of jobs (or total jobs)
    Return type: | int

  `get_member_list`()

    Returns members list from experiment’s config file

    Returns: | experiment’s members
    Return type: | list

  `get_memory`(section)

    Gets memory needed for the given job type :param section: job type :type section: str :return: memory needed :rtype: str

  `get_memory_per_task`(section)

    Gets memory per task needed for the given job type :param section: job type :type section: str :return: memory per task needed :rtype: str

  `get_migrate_host_to`(section)

    Returns the host to change to from platform config file.

    Returns: | host_to
    Return type: | str

  `get_migrate_project_to`(section)

    Returns the project to change to from platform config file.

    Returns: | migrate project to
    Return type: | str

  `get_migrate_user_to`(section)

    Returns the user to change to from platform config file.

    Returns: | migrate user to
    Return type: | str

  `get_min_wrapped_jobs`()

    > Returns the minim number of jobs that can be wrapped together as configured in autosubmit’s config file

    Returns: | minim number of jobs (or total jobs)
    Return type: | int

  `get_notifications`()

    Returns if the user has enabled the notifications from autosubmit’s config file

    Returns: | if notifications
    Return type: | string

  `get_num_chunks`()

    Returns number of chunks to run for each member

    Returns: | number of chunks
    Return type: | int

  `get_output_type`()

    Returns default output type, pdf if none

    Returns: | output type
    Return type: | string

  static `get_parser`(parser_factory, file_path)

    Gets parser for given file

    Parameters: | parser_factory – file_path (str) – path to file to be parsed
    Returns: | parser
    Return type: | SafeConfigParser

  `get_platform`()

    Returns main platforms from experiment’s config file

    Returns: | main platforms
    Return type: | str

  `get_processors`(section)

    Gets processors needed for the given job type :param section: job type :type section: str :return: wallclock time :rtype: str

  `get_project_destination`()

    Returns git commit from experiment’s config file

    Returns: | git commit
    Return type: | str

  `get_project_dir`()

    Returns experiment’s project directory

    Returns: | experiment’s project directory
    Return type: | str

  `get_project_type`()

    Returns project type from experiment config file

    Returns: | project type
    Return type: | str

  `get_remote_dependencies`()

    Returns if the user has enabled the PRESUBMISSION configuration parameter from autosubmit’s config file

    Returns: | if remote dependencies
    Return type: | bool

  `get_rerun`()

    Returns startdates list from experiment’s config file

    Returns: | rerurn value
    Return type: | list

  `get_retrials`()

    Returns max number of retrials for job from autosubmit’s config file

    Returns: | safety sleep time
    Return type: | int

  `get_safetysleeptime`()

    Returns safety sleep time from autosubmit’s config file

    Returns: | safety sleep time
    Return type: | int

  `get_scratch_free_space`(section)

    Gets scratch free space needed for the given job type :param section: job type :type section: str :return: percentage of scratch free space needed :rtype: int

  `get_storage_type`()

    Returns the communications library from autosubmit’s config file. Paramiko by default.

    Returns: | communications library
    Return type: | str

  `get_submodules_list`()

    Returns submodules list from experiment’s config file Default is –recursive :return: submodules to load :rtype: list

  `get_svn_project_revision`()

    Get revision for subversion project

    Returns: | revision for subversion project
    Return type: | str

  `get_svn_project_url`()

    Gets subversion project url

    Returns: | subversion project url
    Return type: | str

  `get_synchronize`(section)

    Gets wallclock for the given job type :param section: job type :type section: str :return: wallclock time :rtype: str

  `get_tasks`(section)

    Gets tasks needed for the given job type :param section: job type :type section: str :return: tasks (processes) per host :rtype: str

  `get_threads`(section)

    Gets threads needed for the given job type :param section: job type :type section: str :return: threads needed :rtype: str

  `get_total_jobs`()

    Returns max number of running jobs from autosubmit’s config file

    Returns: | max number of running jobs
    Return type: | int

  `get_version`()

    Returns version number of the current experiment from autosubmit’s config file

    Returns: | version
    Return type: | str

  `get_wallclock`(section)

    Gets wallclock for the given job type :param section: job type :type section: str :return: wallclock time :rtype: str

  `get_wrapper_check_time`()

    Returns time to check the status of jobs in the wrapper

    Returns: | wrapper check time
    Return type: | int

  `get_wrapper_jobs`()

    Returns the jobs that should be wrapped, configured in the autosubmit’s config

    Returns: | expression (or none)
    Return type: | string

  `get_wrapper_machinefiles`()

    Returns the strategy for creating the machinefiles in wrapper jobs

    Returns: | machinefiles function to use
    Return type: | string

  `get_wrapper_method`()

    Returns the method of make the wrapper

    Returns: | method
    Return type: | string

  `get_wrapper_queue`()

    Returns the wrapper queue if not defined, will be the one of the first job wrapped

    Returns: | expression (or none)
    Return type: | string

  `get_wrapper_type`()

    Returns what kind of wrapper (VERTICAL, MIXED-VERTICAL, HORIZONTAL, HYBRID, NONE) the user has configured in the autosubmit’s config

    Returns: | wrapper type (or none)
    Return type: | string

  `jobs_file`

    Returns project’s jobs file name

  `load_parameters`()

    Load parameters from experiment and autosubmit config files. If experiment’s type is not none, also load parameters from model’s config file

    Returns: | a dictionary containing tuples [parameter_name, parameter_value]
    Return type: | dict

  `load_project_parameters`()

    Loads parameters from model config file

    Returns: | dictionary containing tuples [parameter_name, parameter_value]
    Return type: | dict

  `platforms_file`

    Returns experiment’s platforms config file name

    Returns: | platforms config file’s name
    Return type: | str

  `platforms_parser`

    Returns experiment’s platforms parser object

    Returns: | platforms config parser object
    Return type: | SafeConfigParser

  `project_file`

    Returns project’s config file name

  `reload`()

    Creates parser objects for configuration files

  `set_expid`(exp_id)

    Set experiment identifier in autosubmit and experiment config files

    Parameters: | exp_id (str) – experiment identifier to store

  `set_git_project_commit`(as_conf)

    Function to register in the configuration the commit SHA of the git project version. :param as_conf: Configuration class for exteriment :type as_conf: AutosubmitConfig

  `set_new_host`(section, new_host)

    Sets new host for given platform :param new_host: :param section: platform name :type: str

  `set_new_project`(section, new_project)

    Sets new project for given platform :param new_project: :param section: platform name :type: str

  `set_new_user`(section, new_user)

    Sets new user for given platform :param new_user: :param section: platform name :type: str

  `set_platform`(hpc)

    Sets main platforms in experiment’s config file

    Parameters: | hpc – main platforms
    Type: | str

  `set_safetysleeptime`(sleep_time)

    Sets autosubmit’s version in autosubmit’s config file

    Parameters: | sleep_time (int) – value to set

  `set_version`(autosubmit_version)

    Sets autosubmit’s version in autosubmit’s config file

    Parameters: | autosubmit_version (str) – autosubmit’s version