Pootle 2.8 release notes — Pootle 2.9.0b2 documentation

# Pootle 2.8 release notes

Released on 16 August 2017

Welcome to Pootle 2.8!

If you want to try it, check one of the following:

- Installation instructions
- Upgrading instructions

## Changes in Requirements

- Django>=1.10.5,<1.11
- zlib1g-dev required for lxml
- MySQL support uses mysqlclient instead of MySQLdb
- django-transaction-hooks is no longer used

## Major Changes

- Undertook a security audit
- Improved editor performance
- Added pluggable search backend
- Pootle FS

Below we provide much more detail. These are by no means exhaustive, view the git log for complete information.

## Details of changes

- MySQL now uses the `mysqlclient` database driver instead of `MySQLdb`, this is Django’s preferred database driver.
- Undertook a security audit
- JavaScript fixes addressing performance and memory leaks in the editor
- Improved editor performance
- Extensive review and fixes to RTL layout
- Added pluggable search backend
- Pootle FS:
  - Added admin UI to set up projects configuration and language mapping
  - CLI
  - LanguageMapper
  - FileMapper
  - Store de/serialization
  - Removed the ability to add new TPs from the admin UI for Pootle FS projects
- Refactored code for syncing stores
- Changed the default `robots.txt`
  - It is now a static file
  - Now the site is `allow` by default
- Editor:
  - Suggestions:
    - Support rejecting/accepting suggestions with comments
      - Suggesters are emailed based on new `POOTLE_EMAIL_FEEDBACK_ENABLED` setting
    - Accept suggestion without “can review” permissions if submitted translation is 100% match.
    - Prevent sending suggestions matching a previously submitted suggestion or current translation.
  - Current unit is shown at the top of the unit list so it is easier to use the navigation bar and it doesn’t jump so much.
  - Faster terminology matching yielding more results by using stemming
  - Special characters are depicted by symbols from a custom-built font:
    - Properly display whitespace as special character
    - Technical details of escape sequences are omitted in the output displayed to end users
  - Buttons to add special characters display helpful information when hovering
  - Errors fixed:
    - Fixed bugs in muting/unmuting checks
    - Fixed filtering translations by month
    - Fixed `TypeError` error when filter in editor gets no units
    - Fixed bug that prevented translators from clicking on context rows to edit those context units
  - Translation memory:
    - Display original and translations side-by-side for TM results
    - AmaGama Translation Memory is now queried using CORS
    - AmaGama is only queried if the source language is supported by amaGama
  - Perform all highlighting in the client
  - Clearly present plurals to translator
  - Allow selecting similar translations’ text in editor
  - Force word wrapping on long strings with no spaces
  - Improvements on timeline
  - Cross-language translation is now restricted to admins as it heavily impacts performance and translators are unlikely to require it
  - Removed ability to clear language or project dropdown to prevent performance degradation.
  - Alternate source language translations are no longer displayed for anonymous users.
  - Editor is disabled for users without the required permissions. Reviewing suggestions is similarly disabled without required permissions.
  - Check categories can now be used in dropdown to filter units.
  - Auto-matched translations are now highlighted as such.
  - Incomplete plural translations may now be submitted.
  - String error reporting form is now clearer and prevents empty reports from being submitted.
- Added suggestion bulk management:
  - Provides filtering by user, etc.
  - Allows to review multiple suggestions at once and reject/accept them at once optionally providing a comment for the suggesters
- Configuration system
- Plugin framework
- Comment system
- Removed Plurr format checks
- Removed ENChecker
- Added `pootle_format` app
  - Added support to have several formats in the same project
- Browse pages:
  - Refactored stats backend:
    - Removed stats refresh notice
    - Faster stats retrieval
  - Hide disabled items and allow showing them to admin users
  - Altered order of columns to highlight latest activity and pending work
    - Last updated data is now only shown to admin users
  - Altered order in which some items are listed by default:
    - Projects and languages are sorted by most recent translators changes to highlight activity
    - Virtual folders are sorted by priority to highlight most important strings to translate
  - Hid most of the special templates language data as it is unnecessary
  - Got rid of fat cookies:
    - Increases responsiveness and removes security issue
    - Most data is now stored in user session instead
    - Sidebar is no longer automatically open for anonymous users when an announcement changes.
  - Show leaderboard on top panel and expanded stats panel
  - Numbers are rendered in a locale aware fashion
  - Search widget is disabled if user cannot translate
  - Improvements to the statistics table for overly long filenames and smaller screens
  - Files dropdown no longer keeps references to empty directories
  - Fixed issue where “Back” button not always worked
  - Fixed issue with project dropdown when there are projects without name
- Search:
  - No longer autocompletes
  - Old ‘Exact Match’ was separated into ‘Case-sensitive match’ and ‘Phrase match’ allowing finer-grained searches.
- Added team page:
  - Only for languages so far, and only available to language managers
  - Replaces permissions with roles
  - Provides direct access to suggestion bulk management
- Revamped user profile page
- Removed for performance reasons:
  - Removed statistics from user profiles. Will be brought back in the future.
  - Removed export view
  - Removed unperformant “More stats” in admin dashboard
- Removed reports feature
- Own Pootle localization changes
  - Updated translations. You can still contribute translation updates for your language.
  - Now compilejsi18n is used to compile JavaScript translations into assets, thus requiring django-statici18n app.
  - Password reset email is now localizable in Pootle
  - Multiple changes in localizable strings to ease translation
  - Now Pootle bundles Select2 localization.
- Upload and download:
  - Disabled upload for non-PO projects as conflict handling currently only works in PO
  - Admins can upload translations as other user
  - Fixed error for stores with no revision
  - Added ability to download TMX exports
- New Machine Translation providers:
  - Caighdeán - Irish
  - Welsh
- Refactoring of models to increase performance, including dropping unnecessary indices.
- User input is sanitized for outgoing emails
- Usernames using latin1 characters are now allowed
- Improved RQ usage and new management commands
- Changed Pootle logo and styling
- Added the ability to use a custom logo with `POOTLE_CUSTOM_LOGO`
- Documentation updates

### Pootle FS

Pootle FS enables synchronization of Pootle against a filesystem, or version control system, handling conflict resolution and other situations of two files being out of sync.

### Development changes

- Updated and pinned PyPI requirements:
  - From now on requirements will be pinned in order to simplify support and development
- Tests:
  - Massive improvement in test framework
  - Coverage increased from 55% to 94%
  - Moved to tox
  - Travis caching and optimisations
  - Added JavaScript testing
- Code sanity:
  - Python code cleanup/linting pep8/pyflakes/pep257 to increase code health
  - Javascript code linting and cleanups
  - CSS code linting and cleanups
- Code polishing:
  - Moved all commands to argparse
  - Moved shortcuts to Mousetrap
  - JS improvements, move to React components
- Triage meetings are now held on a weekly basis

### Command changes and additions

- Running Pootle commands using manage.py is no longer supported, use pootle instead.
- pootle warns if configuration is missing.
- `run_cherrypy` has been removed.
- start has been removed, use `runserver` instead.
- `verify_user` and `purge_user` now accept multiple usernames.
- `refresh_scores` now recalculates user scores and accepts multiple usernames. It can be run by projects and/or languages.
- `contributors` command has been refactored in order to return more accurate results and has new options `--since`, `--until` and `--mailmerge`. The `--from-revision` option has been removed.
- `list_serializers` has been added to view serializers and deserializers installed on your system.
- `flush_cache` flushes `default`, `redis` caches, accepts `--rqdata`, `--django-cache` options.
- `config` was added to get, set, list, append and clear configuration settings.
- `init_fs_project` was added.
- `set_filetype` was added.
- Removed refresh_stats and clear_stats commands.
- `export` is now able to export zipped TMX files per translation project with the `--tmx` option. `--rotate` option allows old files to be removed.
- Added `schema` command that allows to dump the database schema on MySQL which is useful for diagnosing differences in database schema.
- Added `update_data` command that allows to update the stats data.
- `init` now creates a development configuration with `--dev` option.

### Changes in settings

- MySQL database connections should now use `STRICT_TRANS_TABLES`.
- `POOTLE_SCORE_COEFFICENTS` has been removed and replaced with `POOTLE_SCORES`.
- `POOTLE_SCORES` accepts custom settings for user scores calculation.
- `POOTLE_TM_SERVER` no longer receives the `MIN_SCORE` parameter, as it was misleading and had questionable effects.
- `POOTLE_TM_SERVER` now accepts a `MIN_SIMILARITY` parameter, to filter out results which might be irrelevant. To learn more, check the documenation on `MIN_SIMILARITY`.
- `POOTLE_SEARCH_BACKEND` was added, to allow configuring the search backend to be used.
- Changed the default value for ACCOUNT_SESSION_REMEMBER so now sessions are always remembered.
- `POOTLE_EMAIL_FEEDBACK_ENABLED` was added, to allow disabling sending emails to suggesters when suggestions are accepted or rejected.
- Added new `POOTLE_CUSTOM_LOGO`, `POOTLE_FAVICONS_PATH`, `POOTLE_FS_WORKING_PATH` and `POOTLE_CANONICAL_URL` settings.
- Deprecated `POOTLE_QUALITY_CHECKER` setting.
- Added new `POOTLE_SQL_MIGRATIONS` setting.
- `POOTLE_MARKUP_FILTER` defaults to `'markdown'`, and `None`, `'html'`, `'textile'` and `'restructuredtext'` values have been deprecated. Deployments using any deprecated markup must migrate manually to Markdown. This setting will be removed in the future since Markdown will be the only available markup.
- Added new `AMAGAMA_SOURCE_LANGUAGES` setting.

## Credits

This release was made possible by the following people:

Ryan Northey, Dwayne Bailey, Julen Ruiz Aizpuru, Taras Semenenko, Leandro Regueiro, Igor Afanasyev, Claude Paroz, Safa Alfulaij, Rene Ladan, Kevin Scannell, Jason P. Pickering, Eamonn Lawlor, Alexander Lakhin, Robbie Cole, Rhoslyn Prys, Prasasto Adi, Nootan Ghimire, Mikhail Paulyshka, Mike Robinson, leonardcj, Henrik Feldt, Francesc Ortiz, Allan Nordhøy, Christian Lohmaier, Burhan Khalid, benbankes, Arash Mousavi, Andy Kittner, Adam Chainz.

And to all our bug finders, testers and translators, a Very BIG Thank You.