Architecture
============

This document describes the high-level architecture of `vidyut-lipi`. We have
written this doc for anyone who wants to better understand how we designed our
system.


Goals and values
----------------

Our goal is to create a standard transliterator for the Sanskrit ecosystem that
is easy to bind to new programming languages.

- *Correctness* and *coverage* of different schemes are our primary goals, and
  both are works in progress. We measure and prove correctness through our test
  suite, which is still growing.

- *Performance* is a secondary goal. While we care about runtime performance,
  it is not a major concern because transliteration is not usually a
  bottleneck. Instead, we care more about code size (for frontend use) and
  startup time (for backend use).

Our major design value is *pragmatism*: we focus on actions that measurably
improve our transliteration for real use cases.


Design influences
-----------------

We have been influenced by a variety of transliteration projects, most prominently
[Sanscript.js][sans], [Aksharamukha][amu], and [indic-transliteration][it].

- We are the original authors of Sanscript.js and follow its vocabulary and
  major design decisions: we store a list of *schemes*, we use these schemes to
  create a *mapping* between characters, and we use that mapping in one of two
  distinct transliteration functions (one for Brahmic schemes and one for Roman
  schemes).

- From indic-transliteration, we have reused the idea of keying scheme maps
  with Devanagari strings, which lets us provide very flexible mapping support.
  That said, Devanagari renders poorly in our terminal and editor, and many of
  its code points don't seem to render at all. Partly for these reasons, we are
  considering other options here.

- From Aksharamukha, we have embraced the spirit of supporting as many schemes
  as we can. We have often used Aksharamukha's output as a gold standard when
  implementing new schemes or verifying behavior.


[sans]: https://github.com/indic-transliteration/sanscript.js
[amu]: https://github.com/virtualvinodh/aksharamukha-python
[it]: https://github.com/indic-transliteration


Core data types
---------------

The three most important types are `Scheme`, `Mapping`, and `Lipika`.

- `Scheme` is a format that we can transliterate to or from. A `Scheme` is
  either an alphabet or an abugida, and it provides a list that maps Devanagari
  strings to its own format.

- `Mapping` defines how one scheme should change into another. Internally, it
  is a hash map with some extra metadata.

- `Lipika` is a thin wrapper over our lower-level code that manages some
  tedious bookkeeping on the user's behalf.


Core modules
------------

We start our description here with the high-level `lipika` module then work our
way into specific implementation details.


### `lipika`

This defines `Lipika`, which provides our main API. `Lipika` is a thin wrapper
over the `mapping` and `transliterate` modules, which we describe below.


### `detect`

This defines logic for detecting the scheme used by some input string. We
recommend using `detect` when building user interfaces.


### `mapping`

This defines the `Mapping` struct and how to create it.


### `transliterate`

This defines our core transliteration logic.


### `scheme`

This defines our list of `Scheme`s along with some helpful utility functions.


### `autogen_schemes`

This is an autogenerated module that defines the actual mapping data for each
scheme. We create this module with `scripts/create_schemes.py`, which loads
data from [`indic_transliteration/common_maps`][cm] then applies some small
changes.

[cm]: https://github.com/indic-transliteration/common_maps/
