oceanum.datamesh.Connector — oceanum 0.1.0 documentation

# oceanum.datamesh.Connector

class oceanum.datamesh.Connector(token=None, service='https://datamesh.oceanum.io', gateway=None)

  Datamesh connector class.

  All datamesh operations are methods of this class

  Attributes

  `host` | Datamesh host

  Methods

  __init__(token=None, service='https://datamesh.oceanum.io', gateway=None)

    Datamesh connector constructor

    Parameters

      - token (string) – Your datamesh access token. Defaults to os.environ.get(“DATAMESH_TOKEN”, None).

      - service (string, optional) – URL of datamesh service. Defaults to os.environ.get(“DATAMESH_SERVICE”, “https://datamesh.oceanum.io”).

      - gateway (string, optional) – URL of gateway service. Defaults to os.environ.get(“DATAMESH_GATEWAY”, “https://gateway.<datamesh_service_domain>”).

    Raises

      ValueError – Missing or invalid arguments

  get_catalog(filter={})

    Get datamesh catalog

    Parameters

      filter (dict, optional) – Set of filters to apply. Defaults to {}.

    Returns

      A datamesh catalog instance

    Return type

      `oceanum.datamesh.Catalog`

  async get_catalog_async(filter={})

    Get datamesh catalog asynchronously

    Parameters

      - filter (dict, optional) – Set of filters to apply. Defaults to {}.

      - loop – event loop. default=None will use `asyncio.get_running_loop()`

      - executor – `concurrent.futures.Executor` instance. default=None will use the default executor

    Returns

      A datamesh catalog instance

    Return type

      Coroutine<`oceanum.datamesh.Catalog`>

  get_datasource(datasource_id)

    Get a Datasource instance from the datamesh. This does not load the actual data.

    Parameters

      datasource_id (string) – Unique datasource id

    Returns

      A datasource instance

    Return type

      `oceanum.datamesh.Datasource`

    Raises

      DatameshConnectError – Datasource cannot be found or is not authorized for the datamesh key

  async get_datasource_async(datasource_id)

    Get a Datasource instance from the datamesh asynchronously. This does not load the actual data.

    Parameters

      - datasource_id (string) – Unique datasource id

      - loop – event loop. default=None will use `asyncio.get_running_loop()`

      - executor – `concurrent.futures.Executor` instance. default=None will use the default executor

    Returns

      A datasource instance

    Return type

      Coroutine<`oceanum.datamesh.Datasource`>

    Raises

      DatameshConnectError – Datasource cannot be found or is not authorized for the datamesh key

  load_datasource(datasource_id, use_dask=True)

    Load a datasource into the work environment

    Parameters

      - datasource_id (string) – Unique datasource id

      - use_dask (bool, optional) – Load datasource as a dask enabled datasource if possible. Defaults to True.

    Returns

      The datasource container

    Return type

      Union[`pandas.DataFrame`, `geopandas.GeoDataFrame`, `xarray.Dataset`]

  async load_datasource_async(datasource_id, use_dask=True)

    Load a datasource asynchronously into the work environment

    Parameters

      - datasource_id (string) – Unique datasource id

      - use_dask (bool, optional) – Load datasource as a dask enabled datasource if possible. Defaults to True.

      - loop – event loop. default=None will use `asyncio.get_running_loop()`

      - executor – `concurrent.futures.Executor` instance. default=None will use the default executor

    Returns

      The datasource container

    Return type

      coroutine<Union[`pandas.DataFrame`, `geopandas.GeoDataFrame`, `xarray.Dataset`]>

  query(query)

    Make a datamesh query

    Parameters

      query (Union[`oceanum.datamesh.Query`, dict]) – Datamesh query as a query object or a valid query dictionary

    Returns

      The datasource container

    Return type

      Union[`pandas.DataFrame`, `geopandas.GeoDataFrame`, `xarray.Dataset`]

  async query_async(query)

    Make a datamesh query asynchronously

    Parameters

      - query (Union[`oceanum.datamesh.Query`, dict]) – Datamesh query as a query object or a valid query dictionary

      - loop – event loop. default=None will use `asyncio.get_running_loop()`

      - executor – `concurrent.futures.Executor` instance. default=None will use the default executor

    Returns

      The datasource container

    Return type

      Coroutine<Union[`pandas.DataFrame`, `geopandas.GeoDataFrame`, `xarray.Dataset`]>

previous

API documentation

next

oceanum.datamesh.Datasource