Welcome to Intake DAL (data access layer) plugin — intake-dal 0.1 documentation

# Welcome to Intake DAL (data access layer) plugin

This Intake plugin helps abstract a dataset over heterogeneous storage systems. It also provides a helper method to specialize a hierarchical catalog to a default storage system.

    user_events:
      driver: dal
      args:
        default: 'local'
        storage:
          local: 'csv://{{ CATALOG_DIR }}/data/user_events.csv'
          serving: 'in-memory-kv://foo'
          batch: 'parquet://{{ CATALOG_DIR }}/data/user_events.parquet'

    # Specialize the catalog dal default storage mode datasources
    # to be "serving".
    cat = DalCatalog(path, storage_mode="serving")

    # reads from the serving storage system
    # using the in-memory-kv Intake plugin
    df = cat.user_events.read()

# Indices and tables

- Index

- Module Index

- Search Page