====================================
 Schevo Schema Definition Reference
====================================


Overview
========

A Schevo database is the combination of a schema definition, stored
data, and an engine that exposes the two.  This document describes in
detail the schema definition syntax used by Schevo.

Schema definitions are pure-Python modules that make use of Schevo's
syntax extensions.  Schema definitions can be directly imported
anywhere, but typically the import process is managed by the Schevo
database engine in order to tie a schema to an open database.

Stored data is managed by the `schevo.store` package, based on Durus_.
The data is kept in a general structure used by all Schevo databases,
described in the `internal database structures`_ reference.

The database engine is defined in the `schevo.database` module, and
provides read/write access to stored data using the higher-level
structures defined by the database's schema definition.

.. _Durus: http://www.mems-exchange.org/software/durus/

.. _internal database structures: ../design/structures.html


Doctest Setup
=============

This document also functions as a doctest:

.. doctest::

    >>> from schevo.test import DocTest


Schema definition syntax
========================

This section describes the syntax used for schema definitions.


Reserved Words
--------------

The following cannot be used for field names in Entity class
definitions:

- Any single-letter name.  These are reserved for `Namespaces`.

- Any name beginning with an underscore.  These are reserved for
  private methods defined by Schevo.

- Any name beginning with a single letter and an underscore, such as
  ``t_something``.  These are used for query, transaction, view, and
  extender methods.

- ``classmethod``.  This is a decorator used to designate a method as
  attached to a namespace on an entity class's extent, and passed the
  entity class as the first argument when called.

- ``db``.  This contains the currently-open database that the schema
  is associated with.

- ``extentmethod`` and ``extentclassmethod``.  These are decorators
  used to designate a method as attached to a namespace on an entity
  class's extent.

- ``schevo``.  This is imported into the global namespace.

- ``sys``.  This is used to expose standard public methods and
  properties of Schevo objects.


Preamble
--------

All schemata must begin with the following two lines::

  from schevo.schema import *
  schevo.schema.prep(locals())


Preamble Template
.................

A file is provided in the Schevo source distribution called
``SCHEMA-PREAMBLE.txt`` that contains the source code required for all
Schevo schemata, as well as some additional comments that assist in
starting a new database schema.


Icon Support
............

To include icon support in a database schema, add the following extent
definition::

  class SchevoIcon(E.Entity):

      _hidden = True

      name = f.string()
      data = f.image()

      _key(name)


Overall Structure
-----------------

A Schevo database schema consists of a series of the following types
of declarations:

- `Entity/extent definitions`_.

- `Field definitions`_.

- Key/index specifications.

- `Query definitions`_ and `query methods`_.

- `Transaction definitions`_ and `transaction methods`_.

- `View definitions`_ and `view methods`_.

- `Extension methods`_.

- Labels.

- `String representations`_.

- `Initial/sample data`_.

Code blocks defined in a schema make use of the Database API.


Schema globals
--------------

The following objects are available in the global Python namespace of
a schema module:

- ``d``: The `schevo.namespace.SchemaDefinition` associated with the
  schema module.

- ``db``: The currently-open `schevo.database.Database` that is using
  the schema module.


Global namespaces
.................

The following Schevo-managed namespaces are available in the global
Python namespace of a schema module.

Methods defined in the schema definition may use any part of the
`public API <reference-api.html>`__ exposed by the currently opened
database that is associated with the schema.

These namespaces cannot be directly modified.  Schevo manages them,
and automatically adds objects to them when it imports a schema
definition as a module.

- ``E``: `Entity <class-schevo.entity.Entity.html>`__ classes.  The
  standard Entity class is in this namespace by default. Each
  entity/extent definition in the schema is added to this namespace.
  See `entity/extent definitions`_.

- ``F``: `Field <class-schevo.field.Field.html>`__ classes.  The field
  classes defined in `schevo.field <module-schevo.field.html>`__ are
  in this namespace by default.  Custom field definitions in the
  schema are added to this namespace.  See `field definitions`_.

- ``f``: `FieldDefinition
  <class-schevo.fieldspec.FieldDefinition.html>`__ constructors.
  These correspond to the classes defined in the ``F`` namespace and
  are used to create lists of field definitions for entities, queries,
  transactions, and views. See `field definitions`_.

- ``Q``: `Query <class-schevo.query.Query.html>`__ classes.  The Query
  class, and all its subclasses defined in `schevo.query
  <module-schevo.query.html>`__, are in this namespace by default.
  Each Query class defined at the database level is added to this
  namespace.  See `query definitions`_.

- ``q``: Database-level query methods.  See `query methods`_.

- ``T``: `Transaction <class-schevo.transaction.Transaction.html>`__
  classes.  The Transaction class, and all its subclasses defined in
  `schevo.transaction <module-schevo.transaction.html>`__, are in this
  namespace by default. See `transaction definitions`_.

- ``t``: Database-level transaction methods.  See `transaction
  methods`_.

- ``V``: `View <class-schevo.view.View.html>`__ classes.  The View
  class, and all its subclasses defined in `schevo.view
  <module-schevo.view.html>`__, are in this namespace by default.
  Each View class defined at the database level is added to this
  namespace.  See `view definitions`_.


Entity/extent definitions
-------------------------

An entity/extent definition minimally consists of a subclass of
``E.Entity``::

    class Foo(E.Entity):
        """Description of Foo."""

The above example defines a ``Foo`` extent, which contains ``Foo``
entities that each have zero fields, as none were defined.

The entity/extent definition may further contain the following types
of declarations:

- `Field definitions`_.  Here is a ``Foo`` extent where each entity
  has a ``name`` field of ``string`` type, and a ``FooChild`` extent
  where each entity has a reference to a ``Foo`` entity and also has a
  ``bar`` field of ``integer`` type::

      class Foo(E.Entity):

          name = f.string()

      class FooChild(E.Entity):

          foo = f.entity('Foo')
          bar = f.integer()

- `Calculated field methods`_.  Here is a ``Gender`` extent whose
  entities have a ``count`` field of type ``integer`` that calculates
  the number of ``Person`` entities whose ``gender`` field references
  that ``Gender`` entity::

      class Gender(E.Entity):

          code = f.string()
          name = f.string()
          @f.integer()
          def count(self):
              return self.s.count('Person', 'gender')

      class Person(E.Entity):

          name = f.string()
          gender = f.entity('Gender', required=False)

- Key/index specifications.  Here is a ``Person`` extent whose
  entities must have unique values for their ``name`` field, and that
  is also indexed by ``age`` then ``name``::

      class Person(E.Entity):

          name = f.string()
          age = f.integer()

          _key(name)
          _index(age, name)

- Extent-specific `query definitions`_ and `query methods`_.

- Entity/extent-specific `transaction definitions`_ and `transaction
  methods`_.

- Entity-specific `view definitions`_ and `view methods`_.

- Entity/extent-specific `extension methods`_.

- Extent labels.  Schevo creates a default singular and plural
  label for extents based on the class name used to define the
  extent.  This may be overridden in the class definition.  Here is a
  ``Person`` extent whose plural label is "People"::

      class Person(E.Entity):
          _plural = u'People'

  Here is a ``TpsReport`` extent whose singular label is
  "T.P.S. Report".  The default plural label is based on the singular
  label, so the plural label for this extent is "T.P.S. Reports"::

      class TpsReport(E.Entity):
          _label = u'T.P.S. Report'

- Entity `string representations`_.  Schevo user interfaces make use
  of string representations of entities when presenting short
  summaries of them.  The default string representation of an entity
  is the value of its ``name`` field if it has one, or the result of
  calling ``repr()`` on the entity if not.  Here is an example of
  a ``FooChild`` extent that has a custom string representation::

      class Foo(E.Entity):

          name = f.string()

      class FooChild(E.Entity):

          foo = f.entity('Foo')
          bar = f.integer()

          def __unicode__(self):
              return u'%s :: %s' % (self.foo, self.bar)

  In the above example, if a ``FooChild`` entity's ``bar`` field value
  was ``12``, and its ``foo`` value referenced a ``Foo`` entity whose
  ``name`` was ``u'abc'``, the result of calling ``unicode()`` on the
  ``FooChild`` entity would be ``u'abc :: 12'``.

- Extent `initial/sample data`_.


Extension methods
-----------------


Initial/sample data
-------------------

Initial data is data that Schevo uses when creating a new database.
Initial data is *always* processed during new database creation.
Specify initial data for an extent by assigning an `_initial`
attribute to the entity class in your database schema.

Sample data is data that Schevo uses to populate a database after its
creation.  Sample data is only processed if the ``-p`` or ``--sample``
option is passed to the ``schevo db create`` command-line tool, or if
you call the `populate` method on an open database.  Specify sample
data for an extent by assigning a `_sample` attribute to the entity
class in your database schema.

Unit test sample data is data that Schevo populates a database with
when running a suite of unit tests against a database schema.  Specify
unit test sample data for an extent by assigning a
`_sample_unittest` attribute to the entity class in your database
schema.

You may also specify sample data with a custom attribute name, such as
`_sample_abc`. Populate a database with your custom sample data by
passing the suffix to the call to `populate` on your database.
For the example, to populate `db` with the `abc` sample data, call
``db.populate('abc')``.

Specify each collection of initial or sample data as a list of tuples,
where each tuple contains values for the fields that the extent's
`create` transaction expects, in the order that it expects
them. Normally, those fields are the same as the fields specified in
the entity class, but in complex database schemata the fields may
differ.

To specify the default value for a field, use the constant
`DEFAULT`. For example, the `Foo` entities that have the names
``'c'`` and ``'d'`` will have a `size` of ``5``::

    class Foo(E.Entity):

        name = f.string()
        size = f.integer(default=5)

        _key(name)

        _initial = [
            ('a', 1),
            ('b', 2),
            ('c', DEFAULT),
            ('d', DEFAULT),
            ]

To specify a value for an `Entity` field that allows only one entity
type, use a tuple containing the values of the fields of the first key
of that entity type.  For example::

    class Foo(E.Entity):

        name = f.string()
        for_rainy_days = f.boolean()

        _key(name, for_rainy_days)

        _initial = [
            ('One', False),
            ('Two', False),
            ('Buckle', False),
            ('Shoe', False),
            ('Shoe', True),
            ]

    class Bar(E.Entity):

        foo = f.entity('Foo')
        baz = f.string()

        _key(foo, bar)

        _initial = [
            (('One', False), 'This is how it starts.'),
            (('Shoe', False), 'This is how it ends.'),
            (('Shoe', False), 'Need to have two shoes.'),
            (('Shoe', True), 'And one for a rainy day.'),
            ]

To specify a value for an `Entity` field that allows more than one
entity type, use a two-tuple that first gives the entity type, then
gives a tuple of the values of the fields of the first key of that
entity type.  For example::

    class Foo(E.Entity):

        name = f.string()

        _key(name)

        _initial = [
            ('one',),
            ('two',),
            ]

    class Bar(E.Entity):

        number = f.integer()

        _key(number)

        _initial = [
            (1,),
            (2,),
            ]

    class Baz(E.Entity):

        foo_or_bar = f.entity('Foo', 'Bar')
        notes = f.string()

        _initial = [
            (('Foo', ('one',)), 'This is the Foo that is named one.'),
            (('Foo', ('two',)), 'This is the Foo that is named two.'),
            (('Bar', (1,)), 'This is the Bar that is numbered 1.'),
            (('Bar', (2,)), 'This is the Bar that is numbered 2.'),
            ]

To specify values for `EntityList` and `EntitySet` fields, simply
enclose the representations of entities within a list or a set,
respectively.


Field definitions
-----------------

Define a field for an Entity, Parameterized Query, Transaction, or
View by using a `field factory`.  Field factories are accessed using
the global `f` namespace.

The members of the `f` namespace are all of the available field types,
converted from their "CamelCase" names to "lowercase_with_underscores"
names, e.g. the `Boolean` field class becomes `f.boolean`, and the
`EntityList` field class becomes `f.entity_list`.

For example, this Entity subclass defines three fields::

    class Dog(E.Entity):

        name = f.string()                         # [1]
        birthdate = f.date(required=False)        # [2]
        disposition = f.entity('Disposition', on_delete=UNASSIGN,
                               required=False,
                               default=('Cheerful',))    # [3]

1. The `name` field is a required `String` field.  By default, all
   fields are required.

2. The `birthdate` field is an optional `Date` field.

3. The `disposition` field is an optional `Entity` field that can
   store a reference to a `Disposition` entity.

   When the referenced `Disposition` entity is deleted, this field's
   value is set to `UNASSIGNED`. See also `cascading delete rules for
   Entity fields`_.

   The default value of this field is the `Disposition` entity that
   matches ``('Cheerful',)``.  See `Default field values`_ below.


Calculated field methods
........................


Cascading delete rules for Entity fields
........................................

When defining an `Entity`, `EntityList`, `EntitySet`, or a custom
entity-storing field, you may specify cascade delete rules for that
field.

Upon receiving a request to delete an entity whose reference is stored
in one of these fields, Schevo will first check the rules of those
fields to see if the deletion is allowed and, if so, what should
happen to the field or entity that refers to the deleted entity.

The default rule of the `available rules`_ is `RESTRICT`, which is the
safest operation, since it prevents accidental deletion of an entity
if it is being referred to by another entity.


Default field values
....................

Default field values are assigned to fields in `Create` transactions.

Specify default field values in a field definition by giving either a
default value in the same notation as you would for initial or sample
data, or by specifying a callable that returns the actual value to be
used as the default value.

Default values are not assigned to a field if a value has been
supplied for that field in the keyword arguments specified when
creating the `Create` transaction.


Specifying type-specific rules
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Specify a rule for only a certain type of entity by giving the type
name and rule as a tuple to the field factory.

For example, to specify an `Entity` field that can refer to a `Cog` or
a `Sprocket` that allows cascade deletion when a `Cog` is deleted but
disallows deletion of a referred-to `Sprocket`, use this::

    part = f.entity(('Cog', CASCADE), ('Sprocket', RESTRICT))


Specifying default rules
~~~~~~~~~~~~~~~~~~~~~~~~

Specify a rule for all types of entities by giving a value to the
`on_delete` keyword argument to the field factory.

For example, to specify an `Entity` field that can refer to a `Cog`, a
`Sprocket`, or a `Widget`, but that requests field unassignment on
deletion of the referred entity, use this::

    part = f.entity('Cog', 'Sprocket', 'Widget', on_delete=UNASSIGN)


Available rules
~~~~~~~~~~~~~~~

* `CASCADE`: If entity `B` has a field that refers to entity `A`, and
  entity `A` is deleted, entity `B` will be deleted as well, provided
  that no other rules are restricting the deletion of entity `B`.

* `REMOVE`: If entity `B` has an `EntityList` or `EntitySet` field
  that contains a reference to entity `A`, and entity `A` is deleted,
  the reference to `A` will be removed from that field in `B`.

* `RESTRICT`: If entity `B` has a field that refers to entity `A`,
  Schevo will not allow deletion of entity `A`.

* `UNASSIGN`: If entity `B` has a field that refers to entity `A`, and
  entity `A` is deleted, the field containing the reference to `A`
  will be set to `UNASSIGNED`, if allowed.

  If the field definition in `B` is an `Entity` field, it must include
  ``required=False``.  If the field in `B` is a required field, then
  the unassignment will not work and the deletion will be restricted.

  If the field definition in `B` is an `EntityList` field, it must
  include ``allow_unassigned=True``. If the field in `B` does not
  allow `UNASSIGNED` to be a member of its collection, then the
  unassignment will not work and the deletion will be restricted.


Rule interaction
~~~~~~~~~~~~~~~~

There may be complex interactions between cascade deletion rules if
you have complex logic in your application.  Schevo is designed to
handle these in a consistent manner.  As of this writing, please see
the source for `schevo.transaction`, and the source for the
`test_on_delete` unit test, for further information.


Field types
-----------

Schevo contains many built-in field types typically used when building
a database schema.  You can also create `custom field types`_ when
your schema has unique requirements for data types.


String field types
..................

**String** fields are used to store Unicode strings. An application
may assume that a `String` field's value can be displayed to a user as
text.

String fields accept a `multiline` attribute that you can set to one
of the following:

* `None`: (default) The string field accepts newlines in its
  value. User interfaces are encouraged to render the field as a
  single-line widget.

* `True`: The string field accepts newlines in its value. User
  interfaces are encouraged to render the field as a multi-line
  widget.

* `False`: The string field does not accept newlines in its
  value. User interfaces are encouraged to render the field as a
  single-line widget.

**Path** fields are `String` fields used to store filesystem paths.


Bytes field types
.................

**Bytes** fields are used to store 8-bit byte sequences. An
application should not assume that a `Bytes` field can be displayed to
a user as text.

**Image** fields are `Bytes` fields that store binary data for an
image.


Numeric field types
...................

**Integer** fields store integer values.

**Float** fields store floating point values.

**Money** fields store fixed-point fractional values, and are commonly
used to store values representing monetary amounts.


Date and time field types
.........................

**Date** fields store `datetime.date` objects.

**Datetime** fields store `datetime.datetime` objects.


Boolean field types
...................

A **Boolean** field stores a boolean value.


Entity field types
..................

An **Entity** field stores a reference to another entity.

An **EntityList** field stores a list of references to other entities.

An **EntitySet** field stores a set of references to other entities.


HashedValue field types
.......................

A **HashedValue** field, upon having its value set, will store a
one-way hash of that value.

Use the `compare(value)` method of a `HashedValue` field instance to
see if the hash matches a value.

It is not possible to retrieve the original value from the one-way
hash.

**HashedPassword** fields are a convenience class to show that a field
specifically stores hashed values of passwords.

This is useful for storing information about a user's password in a
reasonably secure manner, preventing immediate discovery of passwords
if a proprietary Schevo database were stolen or otherwise accessed by
unauthorized users.


Custom field types
..................


Deprecated field types
......................

Deprecated field types are those field types that are available in
Schevo in order to support legacy schemata, but are no longer
recommended for use.

If you have a database schema that uses any of these field types, you
should use a different field type instead.

When you use a deprecated field type, Schevo will give a Python
`DeprecationWarning` about such use to encourage you to use a
different field type.

The following types of fields are deprecated:

- **Blob**: Use `Bytes field types`_ instead.

- **Memo**: Use `String field types`_ instead, setting ``multiline=True`` in your
  field definition.

- **Password**: Use one of the `HashedValue field types`_ instead.

  If you wish to store a plain-text password in your schema, you can
  do one of the following:

  * Use the `String` field type::

        class Foo(E.Entity):
            password = f.string()

  * Create a custom `PlaintextPassword` field type::

        class PlaintextPassword(F.String):
            pass

        class Foo(E.Entity):
            password = f.plaintext_password()

- **Unicode**: Use `String field types`_ to store text, or `Bytes field types`_
  to store 8-bit byte sequences.


Key and index specifications
----------------------------


Labels and plural labels
------------------------


String representations
----------------------


Query definitions
-----------------


Query methods
-------------

Add query methods to entity class definitions to provide access to
query classes.

For entity-level queries, a query method looks something like this,
where `factor` is an optional argument that can be given when the
query method is called, and `RelatedStuff` is a global query class
defined elsewhere::

    def q_related_stuff(self, factor=None):
        q = RelatedStuff()
        if factor is not None:
            q.factor = factor
        return q


Embedded simple queries
.......................

`Simple queries` are queries that do not take any parameters and are
not visibly composed of subqueries.

Because of these properties, it is easy to embed a function within a
query method to expose simple queries, without having to write a
separate query class to hold the code that performs the query.

An embedded simple query looks something like this::

    def q_related_stuff(self):
        def fn():
            # ... Do stuff to build "results" ...
            return results
        return Q.Simple(fn, 'Related Stuff')


Transaction definitions
-----------------------


Customizing standard Create transactions
........................................

See `Transaction Hook Methods <transaction-hook-methods.html>`__.


Customizing standard Delete transactions
........................................

See `Transaction Hook Methods <transaction-hook-methods.html>`__.


Customizing standard Update transactions
........................................

See `Transaction Hook Methods <transaction-hook-methods.html>`__.


Creating new transactions
.........................


Transaction methods
-------------------


View definitions
----------------


View methods
------------


..
     Local Variables:
     mode: rst
     End:
