.. _ref-topics-modeltranslation:

===================
 Model translation
===================

.. admonition:: About this document

    This document provides an introduction to the modeltranslation application.

.. currentmodule:: modeltranslation.models
.. moduleauthor:: P. Eschler <peschler@googlemail.com>

The modeltranslation application can be used to translate dynamic content of 
existing models to an arbitrary number of languages without having to change 
the original model classes. It uses a registration approach (comparable to 
Django's admin app) to be able to add translations to existing or new projects 
and is fully integrated into the Django admin backend.

The advantage of a registration approach is the ability to add translations to
models on a per-project basis. You can use the same app in different projects,
may they use translations or not, and you never have to touch the original 
model class.

.. contents::

Features
========

- Unlimited number of target languages
- Add translations without changing existing models
- Django admin support 
- ?Supports inherited models

Installation
============

To install the application please follow these steps. Each step is described 
in detail in the following sections:

1. Add the ``modeltranslation`` app to the ``INSTALLED_APPS`` variable of your
   project's ``settings.py``. 

2. Configure your languages in the ``settings.py``.

3. Create a ``translation.py`` in your project directory and register
   ``TranslationOptions`` for every model you want to translate.

4. Configure the ``TRANSLATION_REGISTRY`` variable in your ``settings.py``.

5. Sync the database using ``manage.py syncdb`` (note that this only applies 
   if the models registered in the ``translations.py`` did not have been 
   synced to the database before. If they did - read further down what to do
   in that case.

Configure the project's ``settings.py``
---------------------------------------
The following variables have to be added to or edited in the project's 
``settings.py``:

**settings.INSTALLED_APPS**

Make sure that the ``modeltranslation`` app is listed in your 
``INSTALLED_APPS`` variable::

    INSTALLED_APPS = (
        ...
        'modeltranslation',
        ....
    )

Also make sure that the app can be found on a path contained in your 
``PYTHONPATH`` environment variable.

**settings.LANGUAGES**

The LANGUAGES variable must contain all languages used for translation. The 
first language is treated as the *default language*.

The modeltranslation application uses the list of languages to add localized 
fields to the models registered for translation. To use the languages ``de`` and 
``en`` in your project, set the settings.LANGUAGES variable like this (where 
``de`` is the default language)::

    gettext = lambda s: s
    LANGUAGES = (
        ('de', gettext('German')),
        ('en', gettext('English')),
    )

Note that the ``gettext`` lambda function is not a feature of the 
modeltranslation app, but rather required for Django to be able to 
(statically) translate the verbose names of the languages using the standard
``i18n`` solution.

**settings.TRANSLATION_REGISTRY**

In order to be able to import the project's ``translation.py`` registration
file the ``TRANSLATION_REGISTRY`` must be set to a value in the form
``<PROJECT_MODULE>.translation``. E.g. if your project is located in a folder
named ``myproject`` the ``TRANSLATION_REGISTRY`` must be set like this::

    TRANSLATION_REGISTRY = "myproject.translation"


Registering models and their fields for translation
---------------------------------------------------
The ``modeltranslation`` app can translate ``CharField`` and ``TextField`` 
based fields of any model class. For each model to translate a translation
option class containg the fields to translate is registered with the  
``modeltranslation`` app.

Registering models and their fields for translation requires the following 
steps:

1. Create a ``translation.py`` in your project directory.
2. Create a translation option class for every model to translate.
3. Register the model and the translation option class at the 
   ``modeltranslation.translator.translator``

The ``modeltranslation`` application reads the ``translation.py`` file in your
project directory thereby triggering the registration of the translation 
options found in the file. 

A translation option is a class that declares which fields of a model to 
translate. The class must derive from ``modeltranslation.ModelTranslation`` 
and it must provide a ``fields`` attribute storing the list of fieldnames. The
option class must be registered with the 
``modeltranslation.translator.translator`` instance.

.. note:: In contrast to the Django admin application which looks for 
          ``admin.py`` files in the project **and** application directories, 
          the modeltranslation app looks only for one ``translation.py`` file in 
          the project directory. 

To illustrate this let's have a look at a simple example using a ``News`` model. 
The news in this example only contains a ``title`` and a ``text`` field. Instead 
of a news, this could be any Django model class::

    class News(models.Model):
        title = models.CharField(max_length=255)
        text = models.TextField()
    
In order to tell the ``modeltranslation`` app to translate the ``title`` and 
``text`` field, create a ``translation.py`` file in your project directory and 
add the following::
    
    from modeltranslation.translator import translator, TranslationOptions    
    from some.news.models import News
    
    class NewsTranslationOptions(TranslationOptions):
        fields = ('title', 'text',)
    
    translator.register(News, NewsTranslationOptions)

Note that this does not require to change the ``News`` model in any way, it's
only imported. The ``NewsTranslationOptions`` derives from
``TranslationOptions`` and provides the ``fields`` attribute. Finally the model
and it's translation options are registered at the ``translator`` object.

At this point you are mostly done and the model classes registered for 
translation will have been added some auto-magical fields. The next section 
explains how things are working under the hood.

Changes automatically applied to the model class
------------------------------------------------
After registering the ``News`` model for transaltion an SQL dump of the
News app will look like this::

    $ ./manage.py sqlall news
    BEGIN;
    CREATE TABLE `news_news` (
        `id` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
        `title` varchar(255) NOT NULL,
        `title_de` varchar(255) NULL,
        `title_en` varchar(255) NULL,
        `text` longtext NULL,
        `text_de` longtext NULL,
        `text_en` longtext NULL,
    )
    ;
    ALTER TABLE `news_news` ADD CONSTRAINT page_id_refs_id_3edd1f0d FOREIGN KEY (`page_id`) REFERENCES `page_page` (`id`);
    CREATE INDEX `news_news_page_id` ON `news_news` (`page_id`);
    COMMIT;
  
Note the ``title_de``, ``title_en``, ``text_de`` and ``text_en`` fields which 
are not declared in the original News model class but rather have been added by
the modeltranslation app. These are called *translation fields*. There will be 
one for every language in your project's ``settings.py``. 

The name of these additional fields is build using the original name of the
translated field and appending one of the language identifiers found in the
``settings.LANGUAGES``.

As these fields are added to the registered model class as fully valid Django
model fields, they will appear in the db schema for the model although it has
not been specified on the model explicitly. 

.. _set_language: http://docs.djangoproject.com/en/dev/topics/i18n/#the-set-language-redirect-view

If you are starting a fresh project and have considered your translation needs 
in the beginning then simply sync your database and you are ready to use 
the translated models.

In case you are translating an existing project and your models have already 
been synced to the database you will need to alter the tables in your database 
and add these additional translation fields. Note that all added fields are 
declared ``null=True`` not matter if the original field is required. In other 
words - all translations are optional. To populate the default translation 
fields added by the ``modeltranslation`` application you can use the 
``update_translation_fields`` command below. See the `The 
update_translation_fields command` section for more infos on this.


Accessing translated and translation fields
===========================================
The ``modeltranslation`` app changes the behaviour of the translated fields. To
explain this consider the News example again. The original ``News`` model 
looked like this::

    class News(models.Model):
        title = models.CharField(max_length=255)
        text = models.TextField()

Now that it is registered with the ``modeltranslation`` app the model looks 
like this - note the additional fields automatically added by the app::

    class News(models.Model):
        title = models.CharField(max_length=255)  # original/translated field
        title_de = models.CharField(null=True, blank=True, max_length=255)  # default translation field
        title_en = models.CharField(null=True, blank=True, max_length=255)  # translation field
        text = models.TextField() # original/translated field
        text_de = models.TextField(null=True, blank=True) # default translation field
        text_en = models.TextField(null=True, blank=True) # translation field
    
The example above assumes that the default language is ``de``, therefore the 
``title_de`` and ``text_de`` fields are marked as the *default translation 
fields*. If the default language is ``en``, the ``title_en`` and ``text_en`` 
fields would be the *default translation fields*.

Rules for translated field access
---------------------------------
So now when it comes to setting and getting the value of the original and the 
translation fields the following rules apply:

**Rule 1** 

Reading the value from the original field returns the value translated to the 
*current language*.

**Rule 2**

Assigning a value to the original field also updates the value in the 
associated default translation field.

**Rule 3**

Assigning a value to the default translation field also updates the original 
field - note that the value of the original field will not be updated until the 
model instance is saved.

**Rule 4** 

If both fields - the original and the default translation field - are updated 
at the same time, the default translation field wins.


Examples for translated field access
------------------------------------
Because the whole point of using the ``modeltranslation`` app is translating 
dynamic content, the fields marked for translation are somehow special when it 
comes to accessing them. The value returned by a translated field is depending 
on the current language setting. "Language setting" is referring to the Django
`set_language`_ view and the corresponding ``get_lang`` function.

Assuming the current language is ``de`` in the News example from above, the 
translated ``title`` field will return the value from the ``title_de`` field::
    
    # Assuming the current language is "de"
    n = News.objects.all()[0]
    t = n.title # returns german translation

    # Assuming the current language is "en"
    t = n.title # returns english translation 

This feature is implemented using Python descriptors making it happen without 
the need to touch the original model classes in any way. The descriptor uses
the ``django.utils.i18n.get_language`` function to determine the current 
language.


Django admin backend integration
================================
In order to be able to edit the translations via the admin backend you need to
register a special admin class for the translated models. The admin class must
derive from ``modeltranslation.admin.TranslationAdmin`` which does some funky
patching on all your models registered for translation::

    from django.contrib import admin
    from modeltranslation.admin import TranslationAdmin

    class NewsAdmin(TranslationAdmin):
        list_display = ('title',)

    admin.site.register(News, NewsAdmin)

Tweaks applied to the admin
---------------------------

The ``TranslationAdmin`` class does only implement one special method which is
``def formfield_for_dbfield(self, db_field, **kwargs)``. This method does the
following:

1. Removes the original field from every admin form by setting it 
   ``editable=False``.
2. Copies the widget of the original field to each of it's translation fields.
3. Checks if the - now removed - original field was required and if so makes the
   default translation field required instead. 

TranslationAdmin in combination with other admin classes
--------------------------------------------------------
If there already exists a custom admin class for a translated model and you 
don't want or can't edit that class directly there is another solution.

Taken the News example let's say there is a ``NewsAdmin`` class defined by the
News app itself. This app is not yours or you don't want to touch it at all, but
it has this nice admin class::

    class NewsAdmin(model.Admin):
        def formfield_for_dbfield(self, db_field, **kwargs):
            # does some funky stuff with the formfield here

So a first attempt might be to create your own admin class which subclasses 
``NewsAdmin`` and ``TranslationAdmin`` to combine stuff like so::

    class MyTranslatedNewsAdmin(NewsAdmin, TranslationAdmin):
        pass

Unfortunately this won't work because Python can only execute one of the 
``formfield_for_dbfield`` methods. Since both admin class implement this method
Python must make a decision and it chooses the first class ``NewsAdmin``. The 
functionality from ``TranslationAdmin`` will not be executed and translation in
the admin will not work for this class.

But don't panic, here's a solution::

    class MyTranslatedNewsAdmin(NewsAdmin, TranslationAdmin):
        def formfield_for_dbfield(self, db_field, **kwargs):
            field = super(MyTranslatedNewsAdmin, self).formfield_for_dbfield(db_field, **kwargs)
            self.patch_translation_field(db_field, field, **kwargs)
            return field
            
This implements the ``formfield_for_dbfield`` such that both functionalities 
will be executed. The first line calls the superclass method which in this case
will be the one of ``NewsAdmin`` because it is the first class inherited from.
The ``TranslationAdmin`` capsulates all it's functionality in the 
``patch_translation_field(db_field, field, **kwargs)`` method and the 
``formfield_for_dbfield`` implementation of the ``TranslationAdmin`` class 
simply calls it. You can copy this behaviour by calling it from a
custom admin class and that's done in the example above. After that the 
``field`` is fully patched for translation and finally returned.


The ``update_translation_fields`` command
=========================================
In case the modeltranslation app was installed on an existing project and you
have specified to translate fields of models which are already synced to the
database, you have to update your database schema manually.

Unfortunately the newly added translation fields on the model will be empty 
then, and your templates will show the translated value of the fields (see 
Rule 1 below) which will be empty in this case. To correctly initialize the 
default translation field you can use the ``update_translation_fields`` 
command::

    manage.py update_translation_fields 

Taken the News example from above this command will copy the value from the
news object's ``title`` field to the default translation field ``title_de``.
It only does so if the default translation field is empty otherwise nothing 
is copied.

.. note:: The command will examine your ``settings.LANGUAGES`` variable and the
          first language declared there will be used as the default language.

All translated models (as specified in the project's ``translation.py`` will be
populated with initial data. 

Caveats
=======
Consider the following example (assuming the default lanuage is ``de``)::
    
    >>> n = News.objects.create(title="foo")
    >>> n.title
    'foo'
    >>> n.title_de
    >>>    

Because the original field ``title`` was specified in the constructor it is 
directly passed into the instance's ``__dict__`` and the descriptor which 
normally updates the associated default translation field (``title_de``) is not 
called. Therefor the call to ``n.title_de`` returns an empty value.

Now assign the title, which triggers the descriptor and the default translation 
field is updated::

    >>> n.title = 'foo'
    >>> n.title_de
    'foo'
    >>>

Accessing translated fields outside views
-----------------------------------------
Since the ``modeltranslation`` mechanism relies on the current language as it 
is returned by the ``get_language`` function care must be taken when accessing
translated fields outside a view function.

Within a view function the language is set by Django based on a flexible model
described at `How Django discovers language preference`_ which is normally used
only by Django's static translation system. 

.. _How Django discovers language preference: http://docs.djangoproject.com/en/dev/topics/i18n/#id2

When a translated field is accessed in a view function or in a template, it 
uses the ``django.utils.translation.get_language`` function to determine the 
current language and return the appropriate value.

Outside a view (or a template), i.e. in normal Python code, a call to the
``get_language`` function still returns a value, but it might not what you 
expect. Since no request is involved, Django's machinery for discovering the
user's preferred language is not activated. *todo: explain more*

The unittests in ``tests.py`` use the ``django.utils.translation.trans_real``
functions to activate and deactive a specific language outside a view function.

Related projects
================

`django-multilingual`_
----------------------

    A library providing support for multilingual content in Django models. 

It is not possible to reuse existing models without modifying them.


`django-multilingual-model`_
----------------------------
A much simpler version of the above `django-multilingual`.
It works very similiar to the `django-multilingual` approach. 

`transdb`_
----------

    Django's field that stores labels in more than one language in database. 

This approach uses a specialized ``Field`` class, which means one has to change
existing models. 

`i18ndynamic`_
--------------
This approach is not developed any more.


`django-pluggable-model-i18n`_
------------------------------

    This app utilizes a new approach to multilingual models based on the same
    concept the new admin interface uses. A translation for an existing model can
    be added by registering a translation class for that model.
    
This is more or less what ``modeltranslation`` does, unfortunately it is far 
from being finished.

.. _django-multilingual: http://code.google.com/p/django-multilingual/
.. _django-multilingual-model: http://code.google.com/p/django-multilingual-model/
.. _django-transdb: http://code.google.com/p/transdb/
.. _i18ndynamic: http://code.google.com/p/i18ndynamic/
.. _django-pluggable-model-i18n: http://code.google.com/p/django-pluggable-model-i18n/
