MIRROR-DEDUPE-SCAN(1)

NAME
mirror-dedupe-scan - scan and generate repository configuration for mirror-dedupe

SYNOPSIS
mirror-dedupe-scan [--name NAME] [--dest DEST] [--config-dir DIR] [-r DIST] [-R DISTS] [-a ARCH] [-A ARCHES] [-c COMPONENT] [-C COMPONENTS] [-G URL] [-g PATH] [--method METHOD] UPSTREAM-URL

DESCRIPTION
mirror-dedupe-scan automatically scans a Debian/Ubuntu repository and generates a configuration file for use with mirror-dedupe. It detects the sync method (rsync or HTTPS), available architectures, components, distributions, and GPG key locations.

The generated configuration is saved to /etc/mirror-dedupe/repos-available/ and can be enabled by creating a symlink in /etc/mirror-dedupe/repos-enabled/.

OPTIONS

--name NAME
    Repository name (required). Used for the configuration filename and as the default destination path when --dest is omitted.

--dest DEST
    Destination path relative to repo_root. Defaults to the value of --name if not specified.

--config-dir DIR
    Configuration directory (default: /etc/mirror-dedupe).

-r, --dist, --release  DIST
    Add a distribution/suite. May be specified multiple times. When any -r/--dist/--release/-R/--releases option is supplied, the scanner uses exactly the suites given (with special handling for the value “all”; see below).

-R, --releases  DISTS
    Comma-separated list of distributions/suites. This is equivalent to specifying multiple -r options. If any item is the literal “all”, the scanner writes every discovered suite into the generated configuration and disables automatic distribution expansion.

-a, --arch ARCH
    Architecture to include. May be specified multiple times.

-A, --architectures ARCHES
    Comma-separated list of architectures to include. When present, these values override any architectures parsed from Release files.

-c, --component COMPONENT
    Component to include (for example, main or universe). May be specified multiple times.

-C, --components COMPONENTS
    Comma-separated list of components to include. When present, these values override any components parsed from Release files.

-G, --gpg-key-url URL
    Explicit GPG key URL for this repository. Must be used together with -g/--gpg-key-path. If given, the scanner validates that the URL is reachable before writing the configuration.

-g, --gpg-key-path PATH
    Relative GPG key path under repo_root (for example, keys/vendor.asc). Must be used together with -G/--gpg-key-url.

--method METHOD
    Force the sync method to use either rsync or https. When omitted, the sync method is auto-detected.

UPSTREAM-URL
    URL of the upstream repository to scan.

EXAMPLES
Scan the Ubuntu archive (noble):

    mirror-dedupe-scan --name ubuntu --dest ubuntu \
        --release noble \
        http://archive.ubuntu.com/ubuntu

Scan the Grafana repository:

    mirror-dedupe-scan --name grafana --dest grafana \
        --gpg-key-url https://apt.grafana.com/gpg.key \
        --gpg-key-path gpg.key \
        https://apt.grafana.com

Scan the Docker repository (Ubuntu, noble):

    mirror-dedupe-scan --name docker --dest docker \
        --release noble \
        --gpg-key-url https://download.docker.com/linux/ubuntu/gpg \
        --gpg-key-path gpg \
        https://download.docker.com/linux/ubuntu

AUTO-DETECTION
The scanner automatically detects:

Sync method
    Attempts to discover a suitable rsync daemon/module for the upstream host using IPv4-only probes, selecting rsync only when a working dists/ tree is found; otherwise falls back to HTTPS. If --method is supplied, its value is used instead of auto-detection.

Architectures
    Parsed from the Release file

Components
    Parsed from the Release file

Distributions
    Discovered by listing the dists/ directory. When explicit distributions are provided via -r/-R, those values are used directly (with the special value “all” causing all discovered suites to be written to the configuration with distribution expansion disabled). When no distribution options are supplied at all, the scanner behaves as if -R all had been specified: every discovered suite is written to distributions: and distribution expansion is disabled.

GPG keys
    Checks common GPG key locations and per-suite Release.gpg locations. Explicit GPG key overrides provided via -G/-g are honoured directly after a reachability check.

ENABLING A REPOSITORY
After scanning, enable the repository:

    cd /etc/mirror-dedupe/repos-enabled
    ln -s ../repos-available/NAME.conf .

FILES

/etc/mirror-dedupe/repos-available/
    Directory containing available repository configurations

/etc/mirror-dedupe/repos-enabled/
    Directory containing enabled repository configurations (symlinks)

SEE ALSO
mirror-dedupe(1)

AUTHOR
Copyright (c) 2025 Tim Hosking
Email: tim@mungerware.com
Website: https://github.com/munger
Licence: MIT