API — Recipe 0.14.0 documentation

# API

This part of the documentation covers all the interfaces of Recipe.

## Recipe

class `recipe.` `Recipe`(shelf=None, metrics=None, dimensions=None, filters=None, order_by=None, session=None, extension_classes=(), dynamic_extensions=None)

  A tool for getting data.

  Parameters

    - shelf (Shelf) – A shelf to use for shared metrics

    - metrics (`list` of `str`) – A list of metrics to use from the shelf. These can also be `Metric` objects.

    - dimensions (`list` of `str`) – A list of dimensions to use from the shelf. These can also be `Dimension` objects.

    - filters (`list` of `str`) – A list of filters to use from the shelf. These can also be `Filter` objects.

    - order_by (`list` of `str`) – A list of dimension or metric keys from the shelf to use for ordering. If prefixed by ‘-‘ the ordering will be descending.

    - session (`Session`)

    - extension_classes (`list` of `RecipeExtension`) – Extensions to apply to this recipe.

    - dynamic_extensions (`list` of `str`) – Dynamic extensions to apply to this recipe.

  Returns

    A Recipe object.

  `all`()

    Return a (potentially cached) list of result objects.

  `as_table`(name=None)

    Return an alias to a table

  `cache_prefix`(value)

    Set a cache prefix for recipe-caching to use

  `cache_region`(value)

    Set a cache region for recipe-caching to use

  `dimensions`(*dimensions)

    Add a list of Dimension ingredients to the query. These can either be Dimension objects or strings representing dimensions on the shelf.

    The Dimension expression will be added to the query’s select statement and to the group_by.

    Parameters

      dimensions (list) – Dimensions to add to the recipe. Dimensions can either be keys on the `shelf` or Dimension objects

  `filters`(*filters)

    Add a list of Filter ingredients to the query. These can either be Filter objects or strings representing filters on the service’s shelf. `.filters()` are additive, calling .filters() more than once will add to the list of filters being used by the recipe.

    The Filter expression will be added to the query’s where clause

    Parameters

      filters (list) – Filters to add to the recipe. Filters can either be keys on the `shelf` or Filter objects

  `first`()

    Return the first element on the result

  classmethod `from_config`(shelf, obj, **kwargs)

    Construct a Recipe from a plain Python dictionary.

    Most of the directives only support named ingredients, specified as strings, and looked up on the shelf. But filters can be specified as objects.

    Additionally, each RecipeExtension can extract and handle data from the configuration.

  `limit`(limit)

    Limit the number of rows returned from the database.

    Parameters

      limit (int) – The number of rows to return in the recipe. 0 will return all rows.

  `metrics`(*metrics)

    Add a list of Metric ingredients to the query. These can either be Metric objects or strings representing metrics on the shelf.

    The Metric expression will be added to the query’s select statement. The metric value is a property of each row of the result.

    Parameters

      metrics (list) – Metrics to add to the recipe. Metrics can either be keys on the `shelf` or Metric objects

  `offset`(offset)

    Offset a number of rows before returning rows from the database.

    Parameters

      offset (int) – The number of rows to offset in the recipe. 0 will return from the first available row

  `one`()

    Return the first element on the result

  `order_by`(*order_bys)

    Apply an ordering to the recipe results.

    Parameters

      order_bys (list(str)) – Order_bys to add to the recipe. Order_bys must be keys of ingredients already added to the recipe. If the key is prefixed by “-” the ordering will be descending.

  `query`()

    Generates a query using the ingredients supplied by the recipe.

    Returns

      A SQLAlchemy query

  `shelf`(shelf=None)

    Defines a shelf to use for this recipe

  `subquery`(name=None)

    The recipe’s query as a subquery suitable for use in joins or other queries.

  `to_sql`()

    A string representation of the SQL this recipe will generate.

  `total_count`(query=None)

    Return the number of rows that would be returned by this Recipe, ignoring any limit that has been applied.

    Parameters

      query – An optional SQLAlchemy query to calculate total_count for. If None, the recipe query will be used.

    Returns

      A count of the number of rows that are returned by this query.

  `use_cache`(value)

    If False, invalidate the cache before fetching data.

## Shelf

class `recipe.` `Shelf`(*args, **kwargs)

  Holds ingredients used by a recipe.

  Can be initialized with no arguments, but also accepts: - a dictionary of ingredients as a positional argument - ingredients as keyword arguments

  These keyword arguments have special meaning:

  Parameters

    - select_from – The SQLALchemy-compatible object which will be queried (usually a Table or ORM object).

    - table – Unused, but stored on the Meta attribute.

    - metadata – Unused, but stored on the Meta attribute.

  `brew_query_parts`(order_by_keys=[])

    Make columns, group_bys, filters, havings

  property `dimension_ids`

    Return the Dimensions on this shelf in the order in which they were used.

  `enchant`(data, cache_context=None)

    Add any calculated values to each row of a resultset generating a new namedtuple

    Parameters

      - data – a list of row results

      - cache_context – optional extra context for caching

    Returns

      a list with ingredient.cauldron_extras added for all ingredients

  property `filter_ids`

    Return the Metrics on this shelf in the order in which they were used.

  `find`(obj, filter_to_class=<class 'recipe.ingredients.Ingredient'>, constructor=None)

    Find an Ingredient, optionally using the shelf.

    Parameters

      - obj – A string or Ingredient

      - filter_to_class – The Ingredient subclass that obj must be an instance of

      - constructor – An optional callable for building Ingredients from obj

    Returns

      An Ingredient of subclass filter_to_class

  classmethod `from_config`(obj, selectable, ingredient_constructor=<function ingredient_from_validated_dict>, metadata=None)

    Create a shelf using a dict shelf definition.

    Parameters

      - obj – A Python dictionary describing a Shelf.

      - selectable – A SQLAlchemy Table, a Recipe, a table name, or a SQLAlchemy join to select from.

      - metadata – If selectable is passed as a table name, then in order to introspect its schema, we must have the SQLAlchemy MetaData object to associate it with.

    Returns

      A shelf that contains the ingredients defined in obj.

  classmethod `from_validated_yaml`(yaml_str, selectable, **kwargs)

    Create a shelf using a yaml shelf definition.

    Parameters

      - yaml_str – A string containing yaml ingredient definitions.

      - selectable – A SQLAlchemy Table, a Recipe, or a SQLAlchemy join to select from.

    Returns

      A shelf that contains the ingredients defined in yaml_str.

  classmethod `from_yaml`(yaml_str, selectable, **kwargs)

    Shim that calls from_validated_yaml.

    This used to call a different implementation of yaml parsing

  `ingredients`()

    Return the ingredients in this shelf in a deterministic order

  `items`()

    Return an iterator over the ingredient names and values.

  `keys`()

    Return an iterator over the ingredient keys.

  property `metric_ids`

    Return the Metrics on this shelf in the order in which they were used.

  `pop`(k, d=<object object>)

    Pop an ingredient off of this shelf.

  `values`()

    Return an iterator over the ingredients.

## Ingredients

class `recipe.` `Ingredient`(**kwargs)

  Ingredients combine to make a SQLAlchemy query.

  Any unknown keyword arguments provided to an Ingredient during initializatino are stored in a meta object.

      # icon is an unknown keyword argument
      m = Metric(func.sum(MyTable.sales), icon='cog')
      print(m.meta.icon)
      >>> 'cog'

  This meta storage can be used to add new capabilities to ingredients.

  Parameters

    - id (`str`) – An id to identify this Ingredient. If ingredients are added to a Shelf, the id is automatically set as the key in the shelf.

    - columns (`list` of `ColumnElement`) – A list of SQLAlchemy columns to use in a query select.

    - filters (`list` of `BinaryExpression`) – A list of SQLAlchemy BinaryExpressions to use in the .filter() clause of a query.

    - havings (`list` of `BinaryExpression`) – A list of SQLAlchemy BinaryExpressions to use in the .having() clause of a query.

    - columns (`list` of `ColumnElement`) – A list of SQLAlchemy columns to use in the group_by clause of a query.

    - formatters – (`list` of `callable`): A list of callables to apply to the result values. If formatters exist, property {ingredient.id}_raw will exist on each result row containing the unformatted value.

    - cache_context (`str`) – Extra context when caching this ingredient. DEPRECATED

    - ordering (string, ‘asc’ or ‘desc’) – One of ‘asc’ or ‘desc’. ‘asc’ is the default value. The default ordering of this ingredient if it is used in a `recipe.order_by`. This is added to the ingredient when the ingredient is used in a `recipe.order_by`.

    - group_by_strategy (`str`) – A strategy to use when preparing group_bys for the query “labels” is the default strategy which will use the labels assigned to each column. “direct” will use the column expression directly. This alternative is useful when there might be more than one column with the same label being used in the query.

    - quickselects (`list` of named filters) – A list of named filters that can be accessed through `build_filter`. Named filters are dictionaries with a `name` (:obj:str) property and a `condition` property (`BinaryExpression`)

  Returns

    An Ingredient object.

  `build_filter`(value, operator=None)

    Builds a filter based on a supplied value and optional operator. If no operator is supplied an `in` filter will be used for a list and a `eq` filter if we get a scalar value.

    `build_filter` is used by the AutomaticFilter extension.

    Parameters

      - value – A value or list of values to operate against

      - operator (`str`) – An operator that determines the type of comparison to do against value.

        The default operator is ‘in’ if value is a list and ‘eq’ if value is a string, number, boolean or None.

    Returns

      A Filter object

  property `cauldron_extras`

    Yield extra tuples containing a field name and a callable that takes a row.

  `describe`()

    A string representation of the ingredient.

  property `expression`

    An accessor for the SQLAlchemy expression representing this Ingredient.

  `make_column_suffixes`()

    Make sure we have the right column suffixes. These will be appended to id when generating the query.

    Developers note: These are generated when the query runs because the recipe may be run with anonymization on or off, which will inject a formatter.

  property `order_by_columns`

    Yield columns to be used in an order by using this ingredient. Column ordering is in reverse order of columns

  property `query_columns`

    Yield labeled columns to be used as a select in a query.

class `recipe.` `Dimension`(expression, **kwargs)

  A Dimension is an Ingredient that adds columns and groups by those columns. Columns should be non-aggregate SQLAlchemy expressions.

  The required expression supplies the dimension’s “value” role. Additional expressions can be provided in keyword arguments with keys that look like “{role}_expression”. The role is suffixed to the end of the SQL column name.

  For instance, the following

      Dimension(Hospitals.name,
                latitude_expression=Hospitals.lat
                longitude_expression=Hospitals.lng,
                id='hospital')

  would add columns named “hospital”, “hospital_latitude”, and “hospital_longitude” to the recipes results. All three of these expressions would be used as group bys.

  Two special roles that can be added are “id” and “order_by”. If a keyword argument “id_expression” is passed, this expression will appear first in the list of columns and group_bys. This “id” will be used if you call build_filter on the dimension.

  If the keyword argument “order_by_expression” is passed, this expression will appear last in the list of columns and group_bys.

  The following additional keyword parameters are also supported:

  Parameters

    - lookup (`dict`) – A dictionary that is used to map values to new values.

      Note: Lookup adds a `formatter` callable as the first item in the list of formatters.

    - lookup_default (`object`) – A default to show if the value can’t be found in the lookup dictionary.

  Returns

    A Filter object

  Parameters

    - lookup – dict A dictionary to translate values into

    - lookup_default – A default to show if the value can’t be found in the lookup dictionary.

  property `cauldron_extras`

    Yield extra tuples containing a field name and a callable that takes a row

  property `id_prop`

    The label of this dimensions id in the query columns

  `make_column_suffixes`()

    Make sure we have the right column suffixes. These will be appended to id when generating the query.

class `recipe.` `IdValueDimension`(id_expression, value_expression, **kwargs)

  DEPRECATED: A convenience class for creating a Dimension with a separate `id_expression`. The following are identical.

      d = Dimension(Student.student_name, id_expression=Student.student_id)

      d = IdValueDimension(Student.student_id, Student.student_name)

  The former approach is recommended.

  Parameters

    - id_expression (`ColumnElement`) – A column expression that is used to identify the id for a Dimension

    - value_expression (`ColumnElement`) – A column expression that is used to identify the value for a Dimension

class `recipe.` `Metric`(expression, **kwargs)

  A simple metric created from a single expression

  `build_filter`(value, operator=None)

    Building filters with Metric returns Having objects.

class `recipe.` `WtdAvgMetric`(expression, weight_expression, **kwargs)

  A metric that generates the weighted average of a metric by a weight.

class `recipe.` `DivideMetric`(numerator, denominator, **kwargs)

  A metric that divides a numerator by a denominator handling several possible error conditions

  The default strategy is to add an small value to the denominator Passing ifzero allows you to give a different value if the denominator is zero.

class `recipe.` `Filter`(expression, **kwargs)

  A simple filter created from a single expression.

  property `expression`

    An accessor for the SQLAlchemy expression representing this Ingredient.

class `recipe.` `Having`(expression, **kwargs)

  A Having that limits results based on an aggregate boolean clause

  property `expression`

    An accessor for the SQLAlchemy expression representing this Ingredient.

## Extensions

class `recipe.` `RecipeExtension`(recipe)

  Recipe extensions plug into the recipe builder pattern and can modify the generated query.

  recipe generates a query in the following way

  > (RECIPE) recipe checks if a query has been generated

  > (EXTENSIONS) all extension `add_ingredients` run to inject ingredients directly on the recipe

  > (RECIPE) recipe runs gather_all_ingredients_into_cauldron to build a global lookup for ingredients

  > (RECIPE) recipe runs cauldron.brew_query_parts to gather sqlalchemy columns, group_bys and filters

  > (EXTENSIONS) all extension `modify_recipe_parts(recipeparts)` run to directly modify the collected sqlalchemy columns, group_bys or filters

  > (RECIPE) recipe builds a preliminary query with columns

  > (EXTENSIONS) all extension `modify_prequery_parts(prequery_parts)` run to modify the query

  > (RECIPE) recipe builds a full query with group_bys, order_bys, and filters.

  > (RECIPE) recipe tests that this query only uses a single from

  > (EXTENSIONS) all extension `modify_postquery_parts( postquery_parts)` run to modify the query

  > (RECIPE) recipe applies limits and offsets on the query

  > (RECIPE) recipe caches completed query

  When the recipe fetches data the results will be `enchanted` to add fields to the result. `RecipeExtensions` can modify result rows with

  > enchant_add_fields: Return a tuple of field names to add to a result row

  > enchant_row(row): Return a tuple of field values for each row in results.

  `add_ingredients`()

    Add ingredients to the recipe

    This method should be overridden by subclasses

  `enchant_add_fields`()

    This method allows extensions to add fields to a result row. Return a tuple of the field names that are being added with this method

  `enchant_row`(row)

    This method adds the fields named in `enchant_add_fields` to each result row.

  `modify_postquery_parts`(postquery_parts)

    This method allows extensions to directly modify query, group_bys, filters, and order_bys generated from collected ingredients after a final query using columns has been created.

  `modify_prequery_parts`(prequery_parts)

    This method allows extensions to directly modify query, group_bys, filters, and order_bys generated from collected ingredients after a preliminary query using columns has been created.

  `modify_recipe_parts`(recipe_parts)

    Modify sqlalchemy components of the query

    This method allows extensions to directly modify columns, group_bys, filters, and order_bys generated from collected ingredients.

class `recipe.` `AutomaticFilters`(*args, **kwargs)

  Automatic generation and addition of Filters to a recipe.

  Automatic filters take a dictionary of keys and values. For each key in the dictionary, if the key is the id of a `Dimension` on the shelf, a filter will be added to the recipe containing the values.

  `apply_automatic_filters`(value)

    Toggles whether automatic filters are applied to a recipe. The following will disable automatic filters for this recipe:

        recipe.apply_automatic_filters(False)

  `automatic_filters`(value)

    Sets a dictionary of automatic filters to apply to this recipe. If your recipe uses a shelf that has dimensions ‘state’ and ‘gender’ you could filter the data to Men in California and New Hampshire with:

        shelf = Shelf({
            'state': Dimension(Census.state),
            'gender': Dimension(Census.gender),
            'population': Metric(func.sum(Census.population)),
        })
        recipe = Recipe(shelf=shelf)
        recipe.dimensions('state').metrics('population').automatic_filters({
            'state': ['California', 'New Hampshire'],
            'gender': 'M'
        })

    Automatic filter keys can optionally include an `operator`.

    List operators

    If the value provided in the automatic_filter dictionary is a list, the following operators are available. The default operator is `in`:

        in (default)
        notin
        quickselect (applies multiple conditions matching the
          named quickselect, quickselects are ORed together)
        between (requires a list of two items)

    Scalar operators

    If the value provided in the automatic_filter dictionary is a scalar (a string, integer, or number), the following operators are available. The default operator is `eq`:

        eq (equal) (the default)
        ne (not equal)
        lt (less than)
        lte (less than or equal)
        gt (greater than)
        gte (greater than or equal)
        like (SQL LIKE)
        ilike (Case insensitive LIKE)
        quickselect (applies the condition matching the named quickselect)

    An example using operators

    Here’s an example that filters to states that start with the letters A-C:

        shelf = Shelf({
            'state': Dimension(Census.state),
            'gender': Dimension(Census.gender),
            'population': Metric(func.sum(Census.population)),
        })
        recipe = Recipe(shelf=shelf)
        recipe.dimensions('state').metrics('population').automatic_filters({
            'state__lt': 'D'
        })

  `exclude_automatic_filter_keys`(*keys)

    A “blacklist” of automatic filter keys to exclude. The following will cause `'state'` to be ignored if it is present in the `automatic_filters` dictionary:

        recipe.exclude_automatic_filter_keys('state')

  `include_automatic_filter_keys`(*keys)

    A “whitelist” of automatic filter keys to use. The following will only use `'state'` for automatic filters regardless of what is provided in the automatic_filters dictionary:

        recipe.include_automatic_filter_keys('state')

class `recipe.` `BlendRecipe`(*args, **kwargs)

  Add blend recipes, used for joining data from another table to a base table

  Supply a second recipe with a different `from` Optionally supply join criteria, if no join criteria is provided join will be attempted using constraints. All ingredients from the blended recipe will be hoisted to the base recipe except for ingredients that are used for joins (they must be the same anyway).

  Supports blend (inner) and full_blend (outer) joins.

  `blend`(blend_recipe, join_base, join_blend)

    Blend a recipe into the base recipe. This performs an inner join of the blend_recipe to the base recipe’s SQL.

  `full_blend`(blend_recipe, join_base, join_blend)

    Blend a recipe into the base recipe preserving values from both recipes.

    This performs an outer join of the blend_recipe to the base recipe.

class `recipe.` `CompareRecipe`(*args, **kwargs)

  Add compare recipes, used for presenting comparative context vis-a-vis a base recipe.

  Supply a second recipe with the same ``from`. Metrics from the second recipe will be hoisted to the base recipe and suffixed with a string (the default is “_compare” Dimensions will be used to match the base recipe to the compare recipe. Ordering from the base recipe is maintained.

  `compare`(compare_recipe, suffix='_compare')

    Adds a comparison recipe to a base recipe.

class `recipe.` `SummarizeOver`(*args, **kwargs)

class `recipe.` `Anonymize`(*args, **kwargs)

  Allows recipes to be anonymized by adding an anonymize property. This flips the anonymize flag on all Ingredients used in the recipe.

  Injects an ingredient.meta._anonymize boolean property on each used ingredient.

  AnonymizeRecipe should occur last

  `add_ingredients`()

    Put the anonymizers in the last position of formatters

  `anonymize`(value)

    Should this recipe be anonymized

class `recipe.` `Paginate`(*args, **kwargs)

  Allows recipes to paginate results. Pagination also supports searching and sorting within paginated data.

  Using and controlling pagination

  Pagination returns pages of data using limit and offset.

  Pagination is enabled by setting a nonzero page size, like this:

      shelf = Shelf({
          'state': Dimension(Census.state),
          'gender': Dimension(Census.gender),
          'population': Metric(func.sum(Census.population)),
      })
      recipe = Recipe(shelf=shelf, extension_classes=[Paginate])            .dimensions('state')            .metrics('population')            .pagination_page_size(10)

  Pagination may be disabled by setting .apply_pagination(False).

  Searching

  pagination_q allows a recipe to be searched for a string. The default search fields are all dimensions used in the recipe. Search keys can be customized with pagination_search_keys. Search may be disabled by setting .apply_pagination_filters(False)

  Sorting

  Pagination can override ordering applied to a recipe by setting .pagination_order_by(…) to a list of ordering keys. If keys are preceded by a “-“, ordering is descending, otherwise ordering is ascending.

  An example using all features

  Here’s an example that searches for keys that start with “t”, showing the fifth page of results:

      shelf = Shelf({
          'state': Dimension(Census.state),
          'gender': Dimension(Census.gender),
          'age': Dimension(Census.age),
          'population': Metric(func.sum(Census.population)),
      })
      recipe = self.recipe()            .metrics("pop2000")            .dimensions("state", "sex", "age")            .pagination_page_size(10)            .pagination_page(5)            .pagination_q('t%')            .pagination_search_keys("state", "sex")

  This will generate SQL like:

      SELECT census.age AS age,
             census.sex AS sex,
             census.state AS state,
             sum(census.population) AS population
      FROM census
      WHERE lower(census.state) LIKE lower('t%')
        OR lower(census.sex) LIKE lower('t%')
      GROUP BY census.age,
               census.sex,
               census.state
      LIMIT 10
      OFFSET 40

  `add_ingredients`()

    Apply pagination ordering and search to this query if necessary.

  `apply_pagination`(value)

    Should this recipe be paginated.

    Parameters

      value (bool) – Enable or disable pagination for this recipe, default True

  `apply_pagination_filters`(value)

    Should this recipe apply the paginations query filtering.

    Should paginate_q be used to apply a search on paginate_search_keys or all dimensions used in the recipe.

    Parameters

      value (bool) – Enable or disable pagination filtering for this recipe, default True

  `modify_postquery_parts`(postquery_parts)

    Apply validated pagination limits and offset to a completed query.

  `pagination_order_by`(*value)

    Sort this pagination by these keys. Pagination ordering is applied before any other order_bys defined in the recipe.

    Parameters

      value (list(str)) – A list of keys to order the paginated recipe by

  `pagination_page`(value)

    Fetch this page.

    Parameters

      value (integer) – A positive integer page number to fetch

  `pagination_page_size`(value)

    Paginate recipe responses into pages of this size.

    A page size of zero disasbles pagination.

    Parameters

      value (integer) – A page size (zero or a positive integer)

  `pagination_q`(value)

    Search this recipe for this string. The search is an case insensitive like that ORs all dimensions in the recipe by default.

    To search for a substring, use a percentage sign for wildcard, like ‘%searchval%’.

    pagination_search_keys can be used to customize what keys are used for search.

    Parameters

      value (str) – A query string to search for this in this recipe. The query string is evaluated as a ilike on all dimensions in the recipe or pagination_search_keys if provided

  `pagination_search_keys`(*value)

    When querying this recipe with a pagination_q, search these keys

    pagination_search_keys do not have to be used in the recipe.

    Parameters

      value (list(str)) – A list of keys to search in the paginated recipe

  `validated_pagination`()

    Return pagination validated against the actual number of items in the response.

    Raises

      BadRecipe –

## Exceptions

exception `recipe.` `BadIngredient`

  Something is wrong with an ingredient

exception `recipe.` `BadRecipe`

  Something is wrong with a recipe

Now, go start some Recipe Development.