.. _mapproxy-util:

#############
mapproxy-util
#############


The commandline tool ``mapproxy-util`` provides sub-commands that are helpful when working with MapProxy.

To get a list of all sub-commands call::

 mapproxy-util


To call a sub-command::

  mapproxy-util subcommand


Each sub-command provides additional information::

  mapproxy-util subcommand --help


The current sub-commands are:

- :ref:`mapproxy_util_create`
- :ref:`mapproxy_util_serve_develop`
- :ref:`mapproxy_util_serve_multiapp_develop`
- :ref:`mapproxy_util_scales`
- :ref:`mapproxy_util_wms_capabilities`
- :ref:`mapproxy_util_grids`
- :ref:`mapproxy_util_export`
- :ref:`mapproxy_defrag_compact_cache`
- ``autoconfig`` (see :ref:`mapproxy_util_autoconfig`)
- :ref:`mapproxy_util_gridconf_from_ogcapitilematrixset`

.. _mapproxy_util_create:

``create``
==========

This sub-command creates example configurations for you. There are templates for each configuration file.


.. program:: mapproxy-util create

.. cmdoption:: -l, --list-templates

  List names of all available configuration templates.

.. cmdoption:: -t <name>, --template <name>

  Create a configuration with the named template.

.. cmdoption:: -f <mapproxy.yaml>, --mapproxy-conf <mapproxy.yaml>

  The path of the MapProxy configuration. Required for some templates.

.. cmdoption:: --force

  Overwrite any existing configuration with the same output filename.



Configuration templates
-----------------------

Available templates are:

base-config:
  Creates an example ``mapproxy.yaml`` and ``seed.yaml`` file. You need to pass the destination directory to the command.


log-ini:
  Creates an example logging configuration. You need to pass the target filename to the command (i.e. `my-app/log.ini`).

wsgi-app:
  Creates an example server script for the given MapProxy configuration (:option:`--f/--mapproxy-conf<mapproxy-util create -f>`) . You need to pass the target filename to the command.



Example
-------

::

  mapproxy-util create -t base-config ./


.. index:: testing, development, server
.. _mapproxy_util_serve_develop:

``serve-develop``
=================

This sub-command starts a MapProxy instance of your configuration as a stand-alone server.

You need to pass the MapProxy configuration as an argument. The server will automatically reload if you change the configuration or any of the MapProxy source code.


.. program:: mapproxy-util serve-develop

.. cmdoption:: -b <address>, --bind <address>

  The server address where the HTTP server should listen for incomming connections. Can be a port (``:8080``), a host (``localhost``) or both (``localhost:8081``). The default is ``localhost:8080``. You need to use ``0.0.0.0`` to be able to connect to the server from external clients.

.. cmdoption:: --debug

  The server outputs debug logging information to the console.

.. cmdoption:: --log-config

  Supply a logging config.


Example
-------

::

  mapproxy-util serve-develop ./mapproxy.yaml

.. index:: testing, development, server, multiapp
.. _mapproxy_util_serve_multiapp_develop:

``serve-multiapp-develop``
==========================

.. versionadded:: 1.3.0


This sub-command is similar to ``serve-develop`` but it starts a :ref:`MultiMapProxy <multimapproxy>` instance.

You need to pass a directory of your MapProxy configurations as an argument. The server will automatically reload if you change any configuration or any of the MapProxy source code.


.. program:: mapproxy-util serve-multiapp-develop

.. cmdoption:: -b <address>, --bind <address>

  The server address where the HTTP server should listen for incomming connections. Can be a port (``:8080``), a host (``localhost``) or both (``localhost:8081``). The default is ``localhost:8080``. You need to use ``0.0.0.0`` to be able to connect to the server from external clients.


Example
-------

::

  mapproxy-util serve-multiapp-develop my_projects/




.. index:: scales, resolutions
.. _mapproxy_util_scales:

``scales``
==========

.. versionadded:: 1.2.0

This sub-command helps to convert between scales and resolutions.

Scales are ambiguous when the resolution of the output device (LCD, printer, mobile, etc) is unknown and therefore MapProxy only uses resolutions for configuration (see :ref:`scale_resolution`). You can use the ``scales`` sub-command to calculate between known scale values and resolutions.

The command takes a list with one or more scale values and returns the corresponding resolution value.

.. program:: mapproxy-util scales

.. cmdoption:: --unit <m|d>

  Return resolutions in this unit per pixel (default meter per pixel).

.. cmdoption:: -l <n>, --levels <n>

  Calculate resolutions for ``n`` levels. This will double the resolution of the last scale value if ``n`` is larger than the number of the provided scales.

.. cmdoption:: -d <dpi>, --dpi <dpi>

  The resolution of the output display to use for the calculation. You need to set this to the same value of the client/server software you are using. Common values are 72 and 96. The default value is the equivalent of a pixel size of .28mm, which is around 91 DPI. This is the value the OGC uses since the WMS 1.3.0 specification.

.. cmdoption:: --as-res-config

  Format the output so that it can be pasted into a MapProxy grid configuration.

.. cmdoption:: --res-to-scale

  Calculate from resolutions to scale.


Example
-------


For multiple levels as MapProxy configuration snippet:
::

  mapproxy-util scales -l 4 --as-res-config 100000

.. code-block:: yaml

    res: [
         #  res            level        scale
           28.0000000000, #  0      100000.00000000
           14.0000000000, #  1       50000.00000000
            7.0000000000, #  2       25000.00000000
            3.5000000000, #  3       12500.00000000
    ]



With multiple scale values and custom DPI:
::

  mapproxy-util scales --dpi 96 --as-res-config \
      100000 50000 25000 10000

.. code-block:: yaml

  res: [
       #  res            level        scale
         26.4583333333, #  0      100000.00000000
         13.2291666667, #  1       50000.00000000
          6.6145833333, #  2       25000.00000000
          2.6458333333, #  3       10000.00000000
  ]

.. _mapproxy_util_wms_capabilities:

``wms-capabilities``
====================

.. versionadded:: 1.5.0

This sub-command parses a valid capabilites document from a URL and displays all available layers.

This tool does not create a MapProxy configuration, but the output should help you to set up or modify your MapProxy configuration.

The command takes a valid URL GetCapabilities URL.

.. program:: mapproxy-util wms_capabilities

.. cmdoption:: --host <URL>

  Display all available Layers for this service. Each new layer will be marked with a hyphen and all sublayers are indented.

.. cmdoption:: --version <versionnumber>

  Parse the Capabilities-document for the given version. Only version 1.1.1 and 1.3.0 are supported. The default value is 1.1.1



Example
-------

With the following MapProxy layer configuration:

.. code-block:: yaml

  layers:
    - name: osm
      title: My OSM Layer
      sources: [osm_cache]
    - name: foo
      title: Group Layer
      layers:
        - name: layer1a
          title: Title of Layer 1a
          sources: [osm_cache]
        - name: layer1b
          title: Title of Layer 1b
          sources: [osm_cache]

Parsed capabilities document:
::

  mapproxy-util wms-capabilities http://127.0.0.1:8080/service?REQUEST=GetCapabilities

.. code-block:: yaml

  Capabilities Document Version 1.1.1
  Root-Layer:
    - title: MapProxy WMS Proxy
      url: http://127.0.0.1:8080/service?
      opaque: False
      srs: ['EPSG:31467', 'EPSG:31466', 'EPSG:4326', 'EPSG:25831', 'EPSG:25833',
            'EPSG:25832', 'EPSG:31468', 'EPSG:900913', 'CRS:84', 'EPSG:4258']
      bbox:
          EPSG:900913: [-20037508.3428, -20037508.3428, 20037508.3428, 20037508.3428]
          EPSG:4326: [-180.0, -85.0511287798, 180.0, 85.0511287798]
      queryable: False
      llbbox: [-180.0, -85.0511287798, 180.0, 85.0511287798]
      layers:
        - name: osm
          title: My OSM Layer
          url: http://127.0.0.1:8080/service?
          opaque: False
          srs: ['EPSG:31467', 'EPSG:31466', 'EPSG:25832', 'EPSG:25831', 'EPSG:25833',
                'EPSG:4326', 'EPSG:31468', 'EPSG:900913', 'CRS:84', 'EPSG:4258']
          bbox:
              EPSG:900913: [-20037508.3428, -20037508.3428, 20037508.3428, 20037508.3428]
              EPSG:4326: [-180.0, -85.0511287798, 180.0, 85.0511287798]
          queryable: False
          llbbox: [-180.0, -85.0511287798, 180.0, 85.0511287798]
        - name: foobar
          title: Group Layer
          url: http://127.0.0.1:8080/service?
          opaque: False
          srs: ['EPSG:31467', 'EPSG:31466', 'EPSG:25832', 'EPSG:25831', 'EPSG:25833',
                'EPSG:4326', 'EPSG:31468', 'EPSG:900913', 'CRS:84', 'EPSG:4258']
          bbox:
              EPSG:900913: [-20037508.3428, -20037508.3428, 20037508.3428, 20037508.3428]
              EPSG:4326: [-180.0, -85.0511287798, 180.0, 85.0511287798]
          queryable: False
          llbbox: [-180.0, -85.0511287798, 180.0, 85.0511287798]
          layers:
            - name: layer1a
              title: Title of Layer 1a
              url: http://127.0.0.1:8080/service?
              opaque: False
              srs: ['EPSG:31467', 'EPSG:31466', 'EPSG:25832', 'EPSG:25831', 'EPSG:25833',
                    'EPSG:4326', 'EPSG:31468', 'EPSG:900913', 'CRS:84', 'EPSG:4258']
              bbox:
                  EPSG:900913: [-20037508.3428, -20037508.3428, 20037508.3428, 20037508.3428]
                  EPSG:4326: [-180.0, -85.0511287798, 180.0, 85.0511287798]
              queryable: False
              llbbox: [-180.0, -85.0511287798, 180.0, 85.0511287798]
            - name: layer1b
              title: Title of Layer 1b
              url: http://127.0.0.1:8080/service?
              opaque: False
              srs: ['EPSG:31467', 'EPSG:31466', 'EPSG:25832', 'EPSG:25831', 'EPSG:25833',
                    'EPSG:4326', 'EPSG:31468', 'EPSG:900913', 'CRS:84', 'EPSG:4258']
              bbox:
                  EPSG:900913: [-20037508.3428, -20037508.3428, 20037508.3428, 20037508.3428]
                  EPSG:4326: [-180.0, -85.0511287798, 180.0, 85.0511287798]
              queryable: False
              llbbox: [-180.0, -85.0511287798, 180.0, 85.0511287798]


.. _mapproxy_util_grids:

``grids``
=========

.. versionadded:: 1.5.0

This sub-command displays information about configured grids.

The command takes a MapProxy configuration file and returns all configured grids.

Furthermore, default values for each grid will be displayed if they are not defined explicitly.
All default values are marked with an asterisk in the output.

.. program:: mapproxy-util grids

.. cmdoption:: -f <path/to/config>, --mapproxy-config <path/to/config>

  Display all configured grids for this MapProxy configuration with detailed information.
  If this option is not set, the sub-command will try to use the last argument as the mapproxy config.

.. cmdoption:: -l, --list

  Display only the names of the grids for the given configuration, which are used by any grid.

.. cmdoption:: --all

  Show also grids that are not referenced by any cache.

.. cmdoption:: -g <grid_name>, --grid <grid_name>

  Display information only for a single grid.
  The tool will exit, if the grid name is not found.

.. cmdoption:: -c <coverage name>, --coverage <coverage name>

  Display an approximation of the number of tiles for each level that  which are within this coverage.
  The coverage must be defined in Seed configuration.

.. cmdoption:: -s <seed.yaml>, --seed-conf <seed.yaml>

  This option loads the seed configuration and is needed if you use the ``--coverage`` option.

Example
-------

With the following MapProxy grid configuration:
.. code-block:: yaml

  grids:
    localgrid:
      srs: EPSG:31467
      bbox: [5,50,10,55]
      bbox_srs: EPSG:4326
      min_res: 10000
    localgrid2:
      base: localgrid
      srs: EPSG:25832
      res_factor: sqrt2
      tile_size: [512, 512]


List all configured grids:
::

  mapproxy-util grids --list --mapproxy-config /path/to/mapproxy.yaml

::

    GLOBAL_GEODETIC
    GLOBAL_MERCATOR
    localgrid
    localgrid2


Display detailed information for one specific grid:
::

  mapproxy-util grids --grid localgrid --mapproxy-conf /path/to/mapproxy.yaml

.. code-block:: yaml

    localgrid:
        Configuration:
            bbox: [5, 50, 10, 55]
            bbox_srs: 'EPSG:4326'
            min_res: 10000
            origin*: 'sw'
            srs: 'EPSG:31467'
            tile_size*: [256, 256]
        Levels: Resolutions, # x * y = total tiles
            00:  10000,             #      1 * 1      =        1
            01:  5000.0,            #      1 * 1      =        1
            02:  2500.0,            #      1 * 1      =        1
            03:  1250.0,            #      2 * 2      =        4
            04:  625.0,             #      3 * 4      =       12
            05:  312.5,             #      5 * 8      =       40
            06:  156.25,            #      9 * 15     =      135
            07:  78.125,            #     18 * 29     =      522
            08:  39.0625,           #     36 * 57     =   2.052K
            09:  19.53125,          #     72 * 113    =   8.136K
            10:  9.765625,          #    144 * 226    =  32.544K
            11:  4.8828125,         #    287 * 451    = 129.437K
            12:  2.44140625,        #    574 * 902    = 517.748K
            13:  1.220703125,       #   1148 * 1804   =   2.071M
            14:  0.6103515625,      #   2295 * 3607   =   8.278M
            15:  0.30517578125,     #   4589 * 7213   =  33.100M
            16:  0.152587890625,    #   9178 * 14426  = 132.402M
            17:  0.0762939453125,   #  18355 * 28851  = 529.560M
            18:  0.03814697265625,  #  36709 * 57701  =   2.118G
            19:  0.019073486328125, #  73417 * 115402 =   8.472G


.. _mapproxy_util_export:

``export``
==========

This sub-command exports tiles from one cache to another. This is similar to the seed tool, but you don't need to edit the configuration. The destination cache, grid and the coverage can be defined on the command line.


.. program:: mapproxy-util export


Required arguments:

.. cmdoption:: -f, --mapproxy-conf

  The path of the MapProxy configuration of the source cache.

.. cmdoption:: --source

  Name of the source or cache to export.

.. cmdoption:: --levels

  Comma separated list of levels to export. You can also define a range of levels. For example ``'1,2,3,4,5'``, ``'1..10'`` or ``'1,3,4,6..8'``.

.. cmdoption:: --grid

  The tile grid for the export. The option can either be the name of the grid as defined in the in the MapProxy configuration, or it can be the grid definition itself. You can define a grid as a single string of the key-value pairs. The grid definition :ref:`supports all grid parameters <grids>`. See below for examples.

.. cmdoption:: --dest

  Destination of the export. Can be a filename, directory or URL, depending on the export ``--type``.

.. cmdoption:: --type

  Choose the export type. See below for a list of all options.

Other options:

.. cmdoption:: --fetch-missing-tiles

  If MapProxy should request missing tiles from the source. By default, the export tool will only existing tiles.

.. cmdoption:: --coverage, --srs, --where

  Limit the export to this coverage. You can use a BBOX, WKT files or OGR datasources. See :doc:`coverages`.

.. option:: -c N, --concurrency N

  The number of concurrent export processes.


Export types
------------

``tms``:
    Export tiles in a TMS like directory structure.

``mapproxy`` or ``tc``:
    Export tiles like the internal cache directory structure. This is compatible with TileCache.

``mbtile``:
    Export tiles into a MBTile file.

``sqlite``:
    Export tiles into SQLite level files.

``geopackage``:
    Export tiles into a GeoPackage file.

``arcgis``:
    Export tiles in a ArcGIS exploded cache directory structure.

``compact-v1``:
    Export tiles as ArcGIS compact cache bundle files (version 1).


Examples
--------

Export tiles into a TMS directory structure under ``./cache/``. Limit export to the BBOX and levels 0 to 6.

::

    mapproxy-util export -f mapproxy.yaml --grid osm_grid \
        --source osm_cache --dest ./cache/ \
        --levels 1..6 --coverage 5,50,10,60 --srs 4326

Export tiles into an MBTiles file. Limit export to a shape coverage.

::

    mapproxy-util export -f mapproxy.yaml --grid osm_grid \
        --source osm_cache --dest osm.mbtiles --type mbtile \
        --levels 1..6 --coverage boundaries.shp \
        --where 'CNTRY_NAME = "Germany"' --srs 3857

Export tiles into an MBTiles file using a custom grid definition.

::

    mapproxy-util export -f mapproxy.yaml --levels 1..6 \
        --grid "srs='EPSG:4326' bbox=[5,50,10,60] tile_size=[512,512]" \
        --source osm_cache --dest osm.mbtiles --type mbtile \



.. _mapproxy_defrag_compact_cache:

``defrag-compact-cache``
========================


The ArcGIS compact cache format version 1 and 2 are append only. Updating existing tiles will increase the file size. Bundle files become larger and fragmented with time. The ``defrag-compact-cache`` sub-command compacts existing bundle files by rewriting and reorganizing each bundle file.


.. program:: mapproxy-util defrag-compact-cache


Required arguments:

.. cmdoption:: -f, --mapproxy-conf

  The path of the MapProxy configuration with the configured compact caches.

Optional arguments:

.. cmdoption:: --caches

  Comma separated list of caches to defragment. By default all configured compact caches will be defragmented.

.. cmdoption:: --min-percent, --min-mb

  Bundle files with only a minmal fragmentation are skipped. You can define this threshold with ``--min-percent`` as the required minimal percentage of unused space and ``--min-mb`` as the minimal required unused space in megabytes. Both thresholds must be exceeded. Defaults to 10% and 1MB.

.. option:: -n, --dry-run

  This will simulate the defragmentation process.


Examples
--------

Defragment bundle files from ``map1_cache`` and ``map2_cache`` when they have more than 20% and 5MB of unused space. E.g. a 20 MB bundle file only gets rewritten if it becomes smaller then 15MB after defragmentation; a 500MB bundle file only gets rewritten if it becomes smaller then 400MB after defragmentation.

::

  mapproxy-util defrag-compact-cache -f mapproxy.yaml \
    --min-percent 20 \
    --min-mb 5 \
    --caches map1_cache,map2_cache


.. _mapproxy_util_gridconf_from_ogcapitilematrixset:

``gridconf-from-ogcapitilematrixset``
=====================================

This sub-command let you output the tile matrix sets exposed in an OGC API
service that implements OGC API Tiles in the standard format of the ``grids``
section of the :file:`mapproxy.yml` configuration file. Only tilesets that are
compatible with grids supported by MapProxy will be exported.

.. program:: mapproxy-util gridconf-from-ogcapitilematrixset


Required arguments:

.. cmdoption::   --url=URL

  The URL to the OGC API landing page.


Examples
--------

::

    mapproxy-util gridconf-from-ogcapitilematrixset   https://maps.gnosis.earth/ogcapi

    Cannot handle https://maps.gnosis.earth/ogcapi/tileMatrixSets/CDB1GlobalGrid?f=json: Tile matrix with variableMatrixWidths are not supported
    Cannot handle https://maps.gnosis.earth/ogcapi/tileMatrixSets/GNOSISGlobalGrid?f=json: Tile matrix with variableMatrixWidths are not supported
    Cannot handle https://maps.gnosis.earth/ogcapi/tileMatrixSets/GoogleCRS84Quad?f=json: Tile matrix set with varying pointOfOrigin depending on tile matrix
    Cannot handle https://maps.gnosis.earth/ogcapi/tileMatrixSets/ISEA9R?f=json: CRS http://www.opengis.net/def/crs/OGC/0/153456 is not supported
    Configuration to add to mapproxy.yml:

    grids:
      GlobalCRS84Pixel:
        bbox: [-180, -422, 332, 90]
        origin: ul
        res: [2, 1, 0.5, 0.333333333333333, 0.166666666666667, 0.0833333333333333, 0.0333333333333333,
          0.0166666666666667, 0.00833333333333333, 0.00416666666666667, 0.0013888888888889,
          0.00083333333333333, 0.00027777777777778, 0.0001388888888889, 8.333333333333e-05,
          2.777777777778e-05, 8.33333333333e-06, 2.77777777778e-06, 8.3333333333e-07,
          2.7777777778e-07, 8.333333333e-08, 2.777777778e-08, 8.33333333e-09, 2.77777778e-09,
          1.3888889e-09]
        srs: EPSG:4326
        tile_size: [256, 256]
      GlobalCRS84Scale:
        bbox: [-180, -231.95619782843647, 463.91239565687295, 90]
        origin: ul
        res: [1.25764139776733, 0.628820698883665, 0.251528279553466, 0.125764139776733,
          0.0628820698883665, 0.0251528279553466, 0.0125764139776733, 0.00628820698883665,
          0.00251528279553466, 0.00125764139776733, 0.00062882069888367, 0.00025152827955347,
          0.00012576413977673, 6.288206988837e-05, 2.515282795535e-05, 1.257641397767e-05,
          6.28820698884e-06, 2.51528279553e-06, 1.25764139777e-06, 6.2882069888e-07, 2.5152827955e-07,
          1.2576413978e-07, 6.28820699e-08, 2.515282796e-08, 1.257641398e-08, 6.288207e-09,
          2.5152828e-09]
        srs: EPSG:4326
        tile_size: [256, 256]
      WebMercatorQuad:
        bbox: [-20037508.3427892, -20037508.342789043, 20037508.342789043, 20037508.3427892]
        origin: ul
        res: [156543.03392804, 78271.5169640205, 39135.7584820102, 19567.879241005, 9783.93962050256,
          4891.96981025128, 2445.98490512564, 1222.99245256282, 611.49622628141, 305.748113140705,
          152.874056570353, 76.4370282851763, 38.218514142588, 19.109257071294, 9.55462853564703,
          4.77731426782352, 2.38865713391176, 1.19432856695588, 0.59716428347794, 0.29858214173897,
          0.149291070869485, 0.0746455354347424, 0.0373227677173712, 0.0186613838586856,
          0.0093306919293428, 0.0046653459646714, 0.0023326729823357, 0.00116633649116785,
          0.00058316824558393, 0.00029158412279196]
        srs: EPSG:3857
        tile_size: [256, 256]
      WorldCRS84Quad:
        bbox: [-180, -90.0, 180.0, 90]
        origin: ul
        res: [0.703125, 0.3515625, 0.17578125, 0.087890625, 0.0439453125, 0.02197265625,
          0.010986328125, 0.0054931640625, 0.00274658203125, 0.001373291015625, 0.0006866455078125,
          0.00034332275390625, 0.00017166137695312, 8.583068847656e-05, 4.291534423828e-05,
          2.145767211914e-05, 1.072883605957e-05, 5.3644180298e-06, 2.6822090149e-06,
          1.34110450745e-06, 6.7055225372e-07, 3.3527612686e-07, 1.6763806343e-07, 8.381903172e-08,
          4.190951586e-08, 2.095475793e-08, 1.047737896e-08, 5.23868948e-09, 2.61934474e-09,
          1.30967237e-09]
        srs: EPSG:4326
        tile_size: [256, 256]
      WorldMercatorWGS84Quad:
        bbox: [-20037508.3427892, -20037508.342789043, 20037508.342789043, 20037508.3427892]
        origin: ul
        res: [156543.03392804, 78271.5169640205, 39135.7584820102, 19567.879241005, 9783.93962050256,
          4891.96981025128, 2445.98490512564, 1222.99245256282, 611.49622628141, 305.748113140705,
          152.874056570353, 76.4370282851763, 38.218514142588, 19.109257071294, 9.55462853564703,
          4.77731426782352, 2.38865713391176, 1.19432856695588, 0.59716428347794, 0.29858214173897,
          0.149291070869485, 0.0746455354347424, 0.0373227677173712, 0.0186613838586856,
          0.0093306919293428, 0.0046653459646714, 0.0023326729823357, 0.00116633649116785,
          0.00058316824558393, 0.00029158412279196]
        srs: EPSG:3395
        tile_size: [256, 256]
