Schevo Database Usage Reference — libschevo 3.2.4 documentation

### Navigation

- index
- next |
- previous |
- libschevo 3.2.4 documentation »
- Schevo References »

# Schevo Database Usage Reference

## Overview

A Schevo database is the combination of a schema definition, stored data, and an engine that exposes the two. This document describes in detail the public API exposed by the Schevo engine.

Schema definitions are pure-Python modules that make use of Schevo’s syntax extensions. Schema definitions can be directly imported anywhere, but typically the Schevo database engine manages the import process in order to tie a schema to an open database.

Stored data is managed by the SchevoDurus package, based on Durus. The data is kept in a general structure used by all Schevo databases, described in the internal database structures reference.

The database engine is defined in the schevo.database module, and provides read/write access to stored data using the higher-level structures defined by the database’s schema definition.

## Doctest Setup

This document also functions as a doctest:

    .. sourcecode:: pycon

      >>> from schevo.test import DocTest

## sys Namespaces

`sys` (short for system) is a namespace that contains the public methods and properties of a Schevo object that are not fields, query methods, transaction methods, or view methods. These are kept in a separate namespace in order to reduce the amount of reserved words.

`sys` namespaces are accessed by getting the `sys` attribute from a Schevo object.

The following types of Schevo objects have a `sys` namespace:

- Entity instances
- Find query instances
- Param query instances
- Transaction instances
- View instances

## Database instances

### Database labels

By default, a database instance is labeled “Schevo Database”:

    .. sourcecode:: pycon

      >>> from schevo.label import label, relabel
      >>> t = DocTest("""
      ...     class Foo(E.Entity):
      ...         pass
      ...     """)
      >>> print label(t.db)
      Schevo Database

Override the label by using relabel:

    .. sourcecode:: pycon

      >>> relabel(t.db, 'My Database')
      >>> print label(t.db)
      My Database

## Extent instances

### Relaxing and enforcing key restrictions

Schevo has the unique ability to temporarily relax key restrictions while still ensuring that the database remains in a consistent state.

Key restrictions may only be relaxed within the execution of a transaction.

Given an extent db.Foo that has a key specification of _key(field1, field2), relax that key restriction by calling relax_index:

    .. sourcecode:: pycon

      db.Foo.relax_index('field1', 'field2')

Key restrictions that are relaxed within a transaction are relaxed for the entire execution of the transaction, or until the key is enforced within that transaction, or until that transaction ends, whichever comes first.

When a key restriction is relaxed by a transaction, it remains relaxed for all sub-transactions, regardless of any request by those sub-transactions to enforce the key restriction.

To re-enforce the key restriction before the end of a transaction’s execution, call enforce_index:

    .. sourcecode:: pycon

      db.Foo.enforce_index('field1', 'field2')

See the `tests/test_relax_index.py` test case for code examples.

### Extent labels

By default, an extent has a singular and plural label based on the entity class name associated with the extent:

    .. sourcecode:: pycon

      >>> from schevo.label import label, plural
      >>> t = DocTest("""
      ...     class GreatPerson(E.Entity):
      ...
      ...         name = f.string()
      ...     """)
      >>> print label(t.db.GreatPerson)
      Great Person
      >>> print plural(t.db.GreatPerson)
      Great Persons

To override the singular and/or plural labels of an extent, assign values to the _label and/or _plural attributes of the associated entity class:

    .. sourcecode:: pycon

      >>> from schevo.label import label, plural
      >>> t = DocTest("""
      ...     class GreatPerson(E.Entity):
      ...
      ...         name = f.string()
      ...
      ...         _label = 'Great PERSON'
      ...         _plural = 'Great PEOPLE'
      ...     """)
      >>> print label(t.db.GreatPerson)
      Great PERSON
      >>> print plural(t.db.GreatPerson)
      Great PEOPLE

## Entity instances

### Entity sys namespace

#### links: Return entities that link to this entity

Use `links` to find other entities that point to the calling entity.

To find all entities that link to the calling entity, use the form `links()`. The return value will be a dictionary of keys in the form `(other_extent_name, other_field_name)`, each mapping to a list of Entity instances in that extent where that field’s value is the calling entity:

    # Doctest to go here.

To find all entities in a specific extent that link to the calling entity, use the form `links(other_extent_name)`. The return value will be the same as that returned by a call to `links()`:

    # Doctest to go here.

To find all the entities in a specific extent where a specific field value links to the calling entity, use the form `links(other_extent_name, other_field_name)`. The return value will be a list of Entity instances in that extent where that field’s value is the calling entity:

    # Doctest to go here.

### Entity labels

The label of an entity instance is the same as the unicode representation of the entity instance.

By default, the label is determined by the default key defined in the entity class:

    .. sourcecode:: pycon

      >>> from schevo.label import label
      >>> t = DocTest("""
      ...     class State(E.Entity):
      ...
      ...         name = f.string()
      ...         abbreviation = f.string()
      ...
      ...         _key(name)
      ...         _key(abbreviation)
      ...
      ...         _initial = [
      ...             ('Missouri', 'MO'),
      ...             ]
      ...     """)
      >>> missouri = t.db.State.findone(name='Missouri')
      >>> print label(missouri)
      Missouri

If the key has more than one field, the values are separated by two colons:

    .. sourcecode:: pycon

      >>> t = DocTest("""
      ...     class City(E.Entity):
      ...
      ...         name = f.string()
      ...         state = f.entity('State')
      ...
      ...         _key(name, state)
      ...
      ...         _initial = [
      ...             ('Monett', ('Missouri',)),
      ...             ]
      ...
      ...     class State(E.Entity):
      ...
      ...         name = f.string()
      ...         abbreviation = f.string()
      ...
      ...         _key(name)
      ...         _key(abbreviation)
      ...
      ...         _initial = [
      ...             ('Missouri', 'MO'),
      ...             ]
      ...     """)
      >>> missouri = t.db.State.findone(name='Missouri')
      >>> monett = t.db.City.findone(name='Monett', state=missouri)
      >>> print label(monett)
      Monett :: Missouri

Override the __unicode__ method of an entity class to customize the label:

    .. sourcecode:: pycon

      >>> t = DocTest("""
      ...     class City(E.Entity):
      ...
      ...         name = f.string()
      ...         state = f.entity('State')
      ...
      ...         _key(name, state)
      ...
      ...         def __unicode__(self):
      ...             return u'%s, %s' % (self.name, self.state.abbreviation)
      ...
      ...         _initial = [
      ...             ('Monett', ('Missouri',)),
      ...             ]
      ...
      ...     class State(E.Entity):
      ...
      ...         name = f.string()
      ...         abbreviation = f.string()
      ...
      ...         _key(name)
      ...         _key(abbreviation)
      ...
      ...         _initial = [
      ...             ('Missouri', 'MO'),
      ...             ]
      ...     """)
      >>> missouri = t.db.State.findone(name='Missouri')
      >>> monett = t.db.City.findone(name='Monett', state=missouri)
      >>> print label(monett)
      Monett, MO

## Entity placeholders

schevo.placeholder.Placeholder instances store information about an entity in a database. Schevo databases keep Placeholder instances internally, and work with them instead of actual Entity instances since the latter cannot be directly persisted.

Do not worry about using Placeholder instances directly, but be aware of their existence. At times, you may run across an Exception whose message includes the repr of a Placeholder instance, such as a schevo.error.KeyCollision error.

## Field classes

## Field instances

## Query methods and objects

At their highest level, schevo.query.Query objects use criteria you specify to create sequences of results.

There are several types of queries, and some queries’ criteria consist of subqueries, from which it obtains results during its execution.

For the purposes of example, let us assume that we are working with the following schema for this section:

    >>> t = DocTest("""
    ... from schevo.schema import *
    ... schevo.schema.prep(locals())
    ...
    ... class Person(E.Entity):
    ...
    ...     name = f.string()
    ...     birthdate = f.date()
    ...
    ...     _key(name)
    ...
    ...     _plural = u'People'
    ...
    ...     def __unicode__(self):
    ...         return u'%s, born on %s' % (self.name, self.birthdate)
    ...
    ...     _sample_unittest = [
    ...         ('Robert Jones', '1965-02-03'),
    ...         ('Roberto Gonzales', '1976-04-05'),
    ...         ('Roberta Smith', '1987-06-07'),
    ...         ('Rance Thomas', '1954-12-01'),
    ...         ]
    ...
    ... class Location(E.Entity):
    ...
    ...     name = f.string()
    ...
    ...     _key(name)
    ...
    ...     _sample_unittest = [
    ...         ('Bank',),
    ...         ('Grocery Store',),
    ...         ('Library',),
    ...         ]
    ...
    ... class Sighting(E.Entity):
    ...
    ...     person = f.entity('Person')
    ...     location = f.entity('Location')
    ...     when = f.date()
    ...
    ...     def __unicode__(self):
    ...         return u'%s at %s on %s' % (
    ...             self.person.name,
    ...             self.location,
    ...             self.when,
    ...             )
    ...
    ...     _sample_unittest = [
    ...         (('Robert Jones',), ('Library',), '2004-01-01'),
    ...         (('Roberto Gonzales',), ('Library',), '2004-01-04'),
    ...         (('Robert Jones',), ('Grocery Store',), '2004-01-02'),
    ...         (('Roberto Gonzales',), ('Bank',), '2004-01-05'),
    ...         (('Robert Jones',), ('Bank',), '2004-01-03'),
    ...         (('Roberto Gonzales',), ('Grocery Store',), '2004-01-06'),
    ...         (('Roberta Smith',), ('Grocery Store',), '2004-01-07'),
    ...         (('Rance Thomas',), ('Grocery Store',), '2004-01-08'),
    ...         ]
    ... """)
    >>> db = t.db

Both schema modules and database engines expose all available Query classes in its `Q` namespace. For brevity, we will use an alias:

    >>> Q = db.Q

### Match queries

The simplest query is a match on a field. Use the `schevo.query.Match` class:

    >>> person_name = Q.Match(db.Person, 'name', 'startswith')

To set the value of the match query, set the query’s value:

    >>> person_name.value = u'Robert'

If you already have a search in mind, you can specify it in the query constructor:

    >>> person_name = Q.Match(db.Person, 'name', 'startswith', u'Robert')

It is worth noting that at any point, you can get a human language version of the query by converting it to a string:

    >>> print person_name
    People where Name starts with Robert

### Retrieving results

To retrieve the results of a query, call the Query instance to get an iterator over the results. For example, if we are using the `person_name` query defined above:

    >>> results = person_name()
    >>> for person in results:
    ...     print person
    Robert Jones, born on 1965-02-03
    Roberto Gonzales, born on 1976-04-05
    Roberta Smith, born on 1987-06-07

Query results, at the minimum, must support iteration:

    >>> results = person_name()
    >>> i = iter(results)
    >>> i
    <generator object ...>

To cache the query results, use whatever tools you like. For example, you can get a list of results in order to retrieve the length or perform slices:

    >>> results = list(person_name())
    >>> len(results)
    3
    >>> r0 = results[0]
    >>> r0
    <Person entity oid:1 rev:0>
    >>> results[1:3]
    [<Person entity oid:2 rev:0>, <Person entity oid:3 rev:0>]
    >>> results.index(r0)
    0

### Compound queries

Continuing the example, let us also search for persons who have a birthdate before 1980-01-01, and who were most recently sighted at the grocery store.

Create the birthdate query:

    >>> person_birthdate = Q.Match(db.Person, 'birthdate', '<', '1980-01-01')
    >>> print person_birthdate
    People where Birthdate < 1980-01-01
    >>> for person in sorted(person_birthdate()):
    ...     print person
    Rance Thomas, born on 1954-12-01
    Robert Jones, born on 1965-02-03
    Roberto Gonzales, born on 1976-04-05

Combine the results of these queries using an intersection, so that we get the Person entities that match both queries:

    >>> persons = Q.Intersection(person_name, person_birthdate)
    >>> print persons
    the intersection of (People where Name starts with Robert,
    People where Birthdate < 1980-01-01)
    >>> for person in sorted(persons()):
    ...     print person
    Robert Jones, born on 1965-02-03
    Roberto Gonzales, born on 1976-04-05

Get all the sightings for each person:

    >>> person_sightings = Q.Match(db.Sighting, 'person', 'in', persons)
    >>> print person_sightings
    Sightings where Person in the intersection of (People where Name
    starts with Robert, People where Birthdate < 1980-01-01)
    >>> for sighting in sorted(person_sightings()):
    ...     print sighting
    Robert Jones at Library on 2004-01-01
    Roberto Gonzales at Library on 2004-01-04
    Robert Jones at Grocery Store on 2004-01-02
    Roberto Gonzales at Bank on 2004-01-05
    Robert Jones at Bank on 2004-01-03
    Roberto Gonzales at Grocery Store on 2004-01-06

Group the sightings by person. Since we are grouping the results of a query, which may be heterogeneous, we also specify a field class:

    >>> by_person = Q.Group(person_sightings, 'person', db.Sighting.f.person)
    >>> print by_person
    Sightings where Person in the intersection of (People where Name
    starts with Robert, People where Birthdate < 1980-01-01), grouped
    by Person
    >>> for group in sorted(by_person()):
    ...     print '----'
    ...     for result in group:
    ...         print result
    ----
    Robert Jones at Library on 2004-01-01
    Robert Jones at Grocery Store on 2004-01-02
    Robert Jones at Bank on 2004-01-03
    ----
    Roberto Gonzales at Library on 2004-01-04
    Roberto Gonzales at Bank on 2004-01-05
    Roberto Gonzales at Grocery Store on 2004-01-06

The results of a Group query is a list of results. Reduce each group to the result that has the maximum value for the `when` field. Again, we specify a field class since the query source may be heterogeneous:

    >>> most_recent = Q.Max(by_person, 'when', db.Sighting.f.when)
    >>> print most_recent
    results that have the maximum When in each (Sightings where Person
    in the intersection of (People where Name starts with Robert, People
    where Birthdate < 1980-01-01), grouped by Person)
    >>> for sighting in sorted(most_recent()):
    ...     print sighting
    Robert Jones at Bank on 2004-01-03
    Roberto Gonzales at Grocery Store on 2004-01-06

Finally, filter `most_recent` by the grocery store Location:

    >>> grocery_store = db.Location.findone(name=u'Grocery Store')
    >>> last_seen_shopping = Q.Match(
    ...     most_recent, 'location', '==', grocery_store, db.Sighting.f.location)
    >>> print last_seen_shopping
    results that have the maximum When in each (Sightings where Person
    in the intersection of (People where Name starts with Robert, People
    where Birthdate < 1980-01-01), grouped by Person) where Location ==
    Grocery Store
    >>> for sighting in sorted(last_seen_shopping()):
    ...     print sighting
    Roberto Gonzales at Grocery Store on 2004-01-06

### Default queries

The default query for an extent is an Intersection of Match queries against all non-calculated fields of an extent, with a default operator of `any` (which means “is any value”), and a default value of `UNASSIGNED` (which is ignored when using the `any` operator).

This query is the same as would be returned when calling `db.Sighting.q.default()`:

    >>> query = Q.Intersection(
    ...     Q.Match(db.Sighting, 'person', 'any'),
    ...     Q.Match(db.Sighting, 'location', 'any'),
    ...     Q.Match(db.Sighting, 'when', 'any'),
    ...     )
    >>> print query
    all Sightings

Without changing any of the operators, iterating over the results of the default query for an extent yields the same thing as iterating over the extent itself.

The Intersection query optimizes its execution plan to use the extent’s `find` method when all of the `Match` queries match on the same extent, when there is only one of any given field name, when the only operators used are either `any` or `==`, and when all values are not queries.

As seen above, it also optimizes its label for brevity if all values have the `any` operator.

## Field Containers

The following types of objects are field containers:

- Entity instances
- Param query instances
- Transaction instances
- View instances

A field container `obj` implements the following:

- They have a namespace, `obj.f`, that contains field objects.

  - The value of `obj.f.field_name` is a Field instance associated with `obj`, with the name `field_name`.

  - The value of `obj.f['field_name']` is the same as `obj.f.field_name`.

  - Iterating over `obj.f` yields the names of the fields, in the order they were defined.

  - If allowed by the field container, you can remove a field named `field_name` from `obj` using the following:

        .. sourcecode:: pycon

          del obj.f.field_name

  - If allowed by the field container, you can add a new field named `field_name` and of type `FieldClass` to `obj` using either of the following:

        .. sourcecode:: pycon

          obj.f.field_name = existing_field_instance
          obj.f.field_name = f.field_class(...)

    The field is added to the end of the definition order. To re-order fields, you can get them from `obj.f`, delete them from `obj.f`, then reassign them to `obj.f`, as in this example:

        .. sourcecode:: pycon

          # Assume original declaration order was:
          #    foo = f.string()
          #    bar = f.string()
          #    baz = f.string()

          bar, baz = obj.f.bar, obj.f.baz
          del obj.f.bar, obj.f.baz
          obj.f.baz = baz
          obj.f.bar = bar

          # Now the order is the following:
          #    foo = f.string()
          #    baz = f.string()
          #    bar = f.string()

- They have a method, `obj.s.fields()`, that returns a schevo.fieldspec.FieldMap ordered dictionary for the fields in `obj`.

  - Certain types of field decorators accept additional arguments to the call to `obj.s.fields()`.

## Iterators

Many objects in a Schevo database provide iterators.

### Extents

Iterating over an extent results in a sequence of all entities in the extent, ordered by OID.

### Query, view, and transaction namespaces

Iterating over a `f`, `q`, `t`, or `v` namespace results in a sequence of the names in that namespace, in no defined order.

Example:

    >>> sorted(db.Person.q)
    ['by_example', 'exact']
    >>> sorted(db.Person[1].t)
    ['clone', 'delete', 'update']

### Entity sys namespaces

Iterating over an entity’s `sys` namespace has no defined result. It may become part of the Objects without iterators.

### Objects without iterators

- Database instances.
- Entity instances.
- Query instances.
- Transaction instances.
- View instances.
- `sys` namespaces.

## Icon plugin

### Navigation

- index
- next |
- previous |
- libschevo 3.2.4 documentation »
- Schevo References »