yamdl
=====

Lets you store instances of Django models as flat files (simplified fixtures).
For when you want to store content in a Git repo, but still want to be able to
use the normal Django ORM methods and shortcut functions.

It works by loading the data into an in-memory SQLite database on startup, and
then serving queries from there. This means it adds a little time to your app's
boot, and slightly more RAM usage, but with the advantage of a much easier time
dealing with static files (rather than custom code to load them directly).

It does not persist changes to the models back into files - this is purely for
authoring content in a text editor and using it via Django.

Due to Python limitations yamdl currently only works on **Python 3.4** and up.


Why not use normal fixtures?
----------------------------

They're not only a little verbose, but they need to be loaded into a non-memory
database (slower) and you need lots of logic to work out if you should update
or delete existing entries. They're still a better solution for anything that
has a lot of data or which needs JOINs, though.


Installation
------------

First, install the package::

    pip install yamdl

Then, add it to ``INSTALLED_APPS``::

    INSTALLED_APPS = [
        ...
        'yamdl',
        ...
    ]

Then, add the in-memory database to ``DATABASES`` (note that you must have at
least **Python 3.4** to have a SQlite module that understands shared memory URIs)::

    DATABASES = {
        ...
        'yamdl': {
            'ENGINE': 'django.db.backends.sqlite3',
            'NAME': 'file:yamdl-db?mode=memory&cache=shared',
        }
    }

Then, add a ``YAMDL_DIRECTORIES`` setting which defines where your directories
of YAML files can be found (it's a list)::

    YAMDL_DIRECTORIES = [
        os.path.join(PROJECT_DIR, "content"),
    ]

Finally, add the database router::

    DATABASE_ROUTERS = [
        "yamdl.router.YamdlRouter",
    ]


Usage
-----

First, add the ``__yamdl__`` attribute to the models you want to use static
content. A model can only be static or dynamic, not both::

    class MyModel(models.Model):
        ...
        __yamdl__ = True

Then, start making static files under one of the directories you listed in the
``YAMDL_DIRECTORIES`` setting above. Within one of these, make a directory with
the format ``appname.modelname``, and then YAML files ending in ``.yaml``::

    andrew-site/
        content/
            speaking.Talk/
                2017.yaml
                2016.yaml

Within those YAML files, you can define either a list of model instances, like
this::

    - title: 'Alabama'
      section: us-states

    - title: 'Alaska'
      section: us-states
      done: 2016-11-18
      place_name: Fairbanks

    - title: 'Arizona'
      section: us-states
      done: 2016-05-20
      place_name: Flagstaff

Or a single model instance at the top level, like this::

    conference: DjangoCon AU
    title: Horrors of Distributed Systems
    when: 2017-08-04
    description: Stepping through some of the myriad ways in which systems can fail that programmers don't expect, and how this hostile environment affects the design of distributed systems.
    city: Melbourne
    country: AU
    slides_url: https://speakerdeck.com/andrewgodwin/horrors-of-distributed-systems
    video_url: https://www.youtube.com/watch?v=jx1Hkxe64Xs

When you start up Django, as either ``runserver`` or in production, it will read the
YAML files and load them into an in-memory database and then let you query them
using all the standard ORM stuff.


Todo
----

Here's a short list of things I'd like to get done before a 1.0:

* Maybe replace the ``__yamdl__`` attribute with something nicer.
* Support for Python versions before 3.4, either by using a global SQLite ``:memory:`` instance with thread locking or by supporting disk databases with a wipe phase.
* Include YAML files in the Django auto-reloader so editing them loads changes in development.
* Potentially load changes to flat files in production using mtime checking.
