Download Methods — Rucio 1.2 documentation

# Download Methods

class `rucio.client.downloadclient.` `DownloadClient`(client=None, logger=None, check_admin=False)
  `download_aria2c`(items, trace_custom_fields={})

    Uses aria2c to download the items with given DIDs. This function can also download datasets and wildcarded DIDs. It only can download files that are available via https/davs. Aria2c needs to be installed and X509_USER_PROXY needs to be set!

    Parameters: | items – List of dictionaries. Each dictionary describing an item to download. Keys: did - DID string of this file (e.g. ‘scope:file.name’). Wildcards are not allowed rse - Optional: rse name (e.g. ‘CERN-PROD_DATADISK’) or rse expression from where to download base_dir - Optional: base directory where the downloaded files will be stored. (Default: ‘.’) no_subdir - Optional: If true, files are written directly into base_dir and existing files are overwritten. (Default: False) nrandom - Optional: if the DID addresses a dataset, nrandom files will be randomly choosen for download from the dataset ignore_checksum - Optional: If true, skips the checksum validation between the downloaded file and the rucio catalouge. (Default: False) trace_custom_fields – Custom key value pairs to send with the traces
    Returns: | a list of dictionaries with an entry for each file, containing the input options, the did, and the clientState
    Raises: | InputValidationError – if one of the input items is in the wrong format NoFilesDownloaded – if no files could be downloaded NotAllFilesDownloaded – if not all files could be downloaded RucioException – if something went wrong during the download (e.g. aria2c could not be started)

  `download_dids`(items, num_threads=2, trace_custom_fields={})

    Download items with given DIDs. This function can also download datasets and wildcarded DIDs.

    Parameters: | items – List of dictionaries. Each dictionary describing an item to download. Keys: did - DID string of this file (e.g. ‘scope:file.name’). Wildcards are not allowed rse - Optional: rse name (e.g. ‘CERN-PROD_DATADISK’) or rse expression from where to download resolve_archives - Optional: bool indicating whether archives should be considered for download (Default: False) force_scheme - Optional: force a specific scheme to download this item. (Default: None) base_dir - Optional: base directory where the downloaded files will be stored. (Default: ‘.’) no_subdir - Optional: If true, files are written directly into base_dir and existing files are overwritten. (Default: False) nrandom - Optional: if the DID addresses a dataset, nrandom files will be randomly choosen for download from the dataset ignore_checksum - Optional: If true, skips the checksum validation between the downloaded file and the rucio catalouge. (Default: False) transfer_timeout - Optional: Timeout time for the download protocols. (Default: None) num_threads – Suggestion of number of threads to use for the download. It will be lowered if it’s too high. trace_custom_fields – Custom key value pairs to send with the traces
    Returns: | a list of dictionaries with an entry for each file, containing the input options, the did, and the clientState
    Raises: | InputValidationError – if one of the input items is in the wrong format NoFilesDownloaded – if no files could be downloaded NotAllFilesDownloaded – if not all files could be downloaded RucioException – if something unexpected went wrong during the download

  `download_file_from_archive`(items, trace_custom_fields={})

    Download items with a given PFN. This function can only download files, no datasets.

    Parameters: | items – List of dictionaries. Each dictionary describing a file to download. Keys: did - DID string of the archive file (e.g. ‘scope:file.name’). Wildcards are not allowed archive - DID string of the archive from which the file should be extracted rse - Optional: rse name (e.g. ‘CERN-PROD_DATADISK’). RSE Expressions are allowed base_dir - Optional: Base directory where the downloaded files will be stored. (Default: ‘.’) no_subdir - Optional: If true, files are written directly into base_dir and existing files are overwritten. (Default: False) trace_custom_fields – Custom key value pairs to send with the traces
    Returns: | a list of dictionaries with an entry for each file, containing the input options, the did, and the clientState clientState can be one of the following: ALREADY_DONE, DONE, FILE_NOT_FOUND, FAIL_VALIDATE, FAILED
    Raises: | InputValidationError – if one of the input items is in the wrong format NoFilesDownloaded – if no files could be downloaded NotAllFilesDownloaded – if not all files could be downloaded SourceNotFound – if xrdcp was unable to find the PFN ServiceUnavailable – if xrdcp failed RucioException – if something unexpected went wrong during the download

  `download_pfns`(items, num_threads=2, trace_custom_fields={})

    Download items with a given PFN. This function can only download files, no datasets.

    Parameters: | items – List of dictionaries. Each dictionary describing a file to download. Keys: pfn - PFN string of this file did - DID string of this file (e.g. ‘scope:file.name’). Wildcards are not allowed rse - rse name (e.g. ‘CERN-PROD_DATADISK’). RSE Expressions are not allowed base_dir - Optional: Base directory where the downloaded files will be stored. (Default: ‘.’) no_subdir - Optional: If true, files are written directly into base_dir and existing files are overwritten. (Default: False) ignore_checksum - Optional: If true, the checksum validation is skipped (for pfn downloads the checksum must be given explicitly). (Default: True) transfer_timeout - Optional: Timeout time for the download protocols. (Default: None) num_threads – Suggestion of number of threads to use for the download. It will be lowered if it’s too high. trace_custom_fields – Custom key value pairs to send with the traces
    Returns: | a list of dictionaries with an entry for each file, containing the input options, the did, and the clientState clientState can be one of the following: ALREADY_DONE, DONE, FILE_NOT_FOUND, FAIL_VALIDATE, FAILED
    Raises: | InputValidationError – if one of the input items is in the wrong format NoFilesDownloaded – if no files could be downloaded NotAllFilesDownloaded – if not all files could be downloaded RucioException – if something unexpected went wrong during the download