<!--
SPDX-FileCopyrightText: 2025 Florent Rougon
SPDX-License-Identifier: GPL-2.0-or-later
-->
# Python Tooling for Flightgear Development

This package contains Python modules and scripts used for development of the
[FlightGear](https://www.flightgear.org/) flight simulator. It is built from
an official FlightGear repository called
[fgmeta-python](https://gitlab.com/flightgear/fgmeta-python), uploaded by
FlightGear developers but does not contain the flight simulator itself. If you
are interested in the latter, please visit its [home
page](https://www.flightgear.org/).

This package is mostly of interest to FlightGear developers and
contributors. For instance:

- internationalization scripts for aircraft and add-on developers who want to
  make their work translatable into other languages;
- `fg-build-catalog` for preparing aircraft catalogs (for hangars that can be
  added in the FlightGear built-in launcher);
- `fg-check-aircraft` for checking `-set.xml` and other PropertyList files
  inside aircraft directories;
- `fg-validate-PropertyList` for checking PropertyList files in other places
  like FGData or add-ons;
- `git-date.py` for easily finding commits close to a given date in
  several FlightGear repositories;
- the `terrasync.py` script used for maintaining scenery mirrors (please don't
  abuse it, as it downloads very large amounts of data which couldn't be
  afforded at higher scale);
- `create_dirindex.py`, used to add or update `.dirindex` files in a directory
  tree;
- etc.

There is nothing special about the packaging. The following instructions
are for people familiar with FlightGear but not necessarily with `pip` and
Python `venv`s.

[[_TOC_]]

## The `pipx` Way

[pip](https://pip.pypa.io/) is the standard package manager for the Python
ecosystem. The [pipx tool](https://pipx.pypa.io/) is a `pip` front-end that
provides an easy way to install Python packages in virtual environments
(“venvs”) and run programs from these virtual environments. If you are not
familiar with Python venvs and the [classic installation
method](#the-classic-way) seems complicated to you, you can install `pipx` and
use it to run programs from the `flightgear` Python package. The main
difference is that `pipx` creates and manages virtual environments for you,
whereas in what I call the “classic way”, you are responsible for these little
tasks.

`pipx` offers basically two ways for running programs shipped in a Python
package:

- user-managed installation (`pipx install`): `pipx` creates a venv dedicated
  to the package you want to install and its dependencies. It also arranges
  things so as to allow you to run scripts shipped in the package without
  specifying their full path (on Linux, by default, it creates symbolic links
  in `$HOME/.local/bin` that point to the scripts located inside the venv). In
  this mode, the package is not upgraded unless you explicitly ask for it.

- install-and-run mode (`pipx run`): `pipx` creates or reuses a temporary venv
  dedicated to the package you want to install and its dependencies, installs
  them in the venv if not already done, then runs the specified command from
  the venv. The venv is cached and reused for up to 14 days.

Although `pipx` supports installation as root for all users
(cf. `pipx install --global`), the instructions below are written assuming
you'll run them under a normal user account.

### User-managed installation

The [`pipx` install](https://pipx.pypa.io/stable/docs/#pipx-install) command
performs a permanent installation of the specified packages inside a dedicated
virtual environment (“venv”). Let's use it to install the latest version of
the `flightgear` Python package, which is the package contained in the
[fgmeta-python repository](https://gitlab.com/flightgear/fgmeta-python):

    pipx install flightgear

This may give a hint concerning your `PATH`, please pay attention to it. In
particular, you may want to add `$HOME/.local/bin` to your `PATH` on Linux if
not already there, or run `pipx ensurepath` (this command should work on any
operating system supported by `pipx`). Then you can run programs shipped by
the package, for instance:

    fg-check-aircraft --help
    fg-check-aircraft -rv -I/path/to/fgdata /path/to/fgaddon/Aircraft
    fg-extract-translatable-strings --help
    git-date.py --help

Upgrading the package can be done with `pipx upgrade flightgear`; removing it,
including the dedicated venv, with `pipx uninstall flightgear`.
[^footnote-pipx-shallow-upgrade] The `pipx list` command shows which packages
have been installed using `pipx` and which executables they provide. See the
[`pipx` documentation](https://pipx.pypa.io/stable/docs/) for more
information.

[^footnote-pipx-shallow-upgrade]: Note that by default, `pipx upgrade` only upgrades the specified package inside its venv, not its dependencies; see `--pip-args=--upgrade-strategy=eager` for changing this behavior and this [blog post](https://jamescooke.info/pipxs-upgrade-is-shallow-lets-go-deeper.html) for more details.

If, rather than the latest release of the `flightgear` Python package, you
want to install its latest state directly from the `fgmeta-python` Git
repository, you can use:

    pipx install git+https://gitlab.com/flightgear/fgmeta-python.git

Alternatively, you can pass a local directory to `pipx install` if you already
have a clone of the repository (both of these are `pip` features, actually).

Finally, like `pip`, `pipx` supports *editable installations* using
`pipx install -e`: if you have a local clone of
[fgmeta-python](https://gitlab.com/flightgear/fgmeta-python), you can manage
it yourself (update, modify Python files, etc.) and immediately see the
results in the package installed from it.

### Install-and-run mode

This section introduces an alternative to [`pipx`
install](https://pipx.pypa.io/stable/docs/#pipx-install). When using the
[`pipx run`](https://pipx.pypa.io/latest/docs/#pipx-run) command, `pipx`
downloads the latest version of a Python package to a temporary virtual
environment (or reuses a cached version if present), then runs a program from
it. The environment will be cached and reused for up to 14 days.

The following command, run as a normal user, downloads the latest release of
the `flightgear` Python package and runs `git-date.py --help` from it:

    pipx run --spec flightgear git-date.py --help

Want to extract translatable strings from your aircraft or add-on?

    pipx run --spec flightgear fg-extract-translatable-strings write /path/to/aircraft-or-addon

etc. For the final example, we'll run `fg-check-aircraft` which is also part
of the `flightgear` Python package. Assuming we want to use its very latest
code, we'll ask `pipx` to build the package from the latest state of the
default branch (`next`) of its Git repository. First, we ask
`fg-check-aircraft` to display its usage instructions:

    pipx run --spec git+https://gitlab.com/flightgear/fgmeta-python.git fg-check-aircraft --help

Then we use it to check an aircraft directory, for instance
`/path/to/fgaddon/Aircraft/707`:

    pipx run --spec git+https://gitlab.com/flightgear/fgmeta-python.git fg-check-aircraft --include=$FG_ROOT /path/to/fgaddon/Aircraft/707

That's it! If you're only a user of the scripts, this method may be enough to
suit your needs. On the other hand, if you want the scripts available as
normal commands or intend to modify them or the libraries in
[fgmeta-python](https://gitlab.com/flightgear/fgmeta-python) (develop, test,
file merge requests, etc.), `pipx install` as described
[above](#user-managed-installation) or the “classic way” explained below may
be more convenient. In particular, a `pip` [editable
installation](https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs)
is a must for development.

## The Classic Way

### Short Installation Instructions

#### Installing the latest release

Installing the latest release can be done with `pip install flightgear`.
However, unless you're confident that plain `pip` is going to install
where you want (see [below](#a-word-on-pip)), we suggest you create a
virtual environment and install the `flightgear` Python package inside
(you can replace `my-venv` with the name of your choice):

    python3 -m venv my-venv
    my-venv/bin/pip install flightgear

> [!note]
> On Windows, the `bin` directory is called `Scripts`.

This doesn't require any superuser privilege but you may need to first
install a package such as `python3-venv` using your operating system
package manager if it doesn't provide a complete Python installation by
default.

Once the above two commands have been run, installed scripts are located
in `my-venv/bin` (`my-venv\Scripts` on Windows) so, for instance, you can
run:

    my-venv/bin/git-date.py --help

to obtain the help text of `git-date.py` (adapt the path if running on
Windows). If you want to have some of these scripts in your `PATH` for
more convenience, see [here](#installation-inside-a-venv).

Assuming you've installed the `flightgear` Python package as above,
updating it can be performed with:

    my-venv/bin/pip install -U flightgear

(`-U` being a shorthand for `--upgrade`) and removing it with:

    my-venv/bin/pip uninstall flightgear

(alternatively, you can remove the whole `my-venv` directory).

For more information on the `pip` package manager, please refer to its
[website](https://pip.pypa.io/).

> [!note]
> If your Python installation is later updated (version change) or if you
> move the directories in the above commands elsewhere, delete the
> `my-venv` directory and redo the installation.

#### Installing from the Git repository

This method describes an *editable installation*; it is for people who are
comfortable using [Git](https://git-scm.com/) and want to use the latest
state of the code. The following commands clone the [fgmeta-python
repository](https://gitlab.com/flightgear/fgmeta-python), create a venv
and perform an editable installation of the `flightgear` Python package
inside the venv:

    git clone https://gitlab.com/flightgear/fgmeta-python.git
    cd fgmeta-python
    python3 -m venv .venv
    .venv/bin/pip install -e .

> [!note]
> On Windows, the `bin` directory is called `Scripts`.

If the last command shows an error, this is likely due to outdated
packages. In such a case, run the following:

    .venv/bin/pip install -U build pip setuptools

then redo the `pip install` step.

Scripts declared in `pyproject.toml` have been created by that step in the
`.venv/bin` directory. Updates are obtained by updating the
`fgmeta-python` clone (`git pull`). If your Python installation is updated
(version change) or if you move the directories in the above commands
elsewhere, delete the `.venv` directory, recreate the venv and redo the
`pip install` step.

In case you add, rename or remove installed scripts (those declared in
`pyproject.toml`), redo the `pip install` step to have the `.venv/bin`
directory updated accordingly.

### Long Installation Instructions

Normal installation of the package is done with the Python package
installer [pip](https://pip.pypa.io/).

#### A Word on `pip`

Telling people to just `pip install ...` is quite common but reality is a
bit more complex. If you installed Python without a package manager, this
should be just fine (on Windows, you may have to use something like
`py -m pip install ...` if `pip` is not in your `PATH` but the Python
Launcher for Windows `py` is).

The main case where straight `pip install ...` commands are not
appropriate is if your Python installation (at large, including Python
packages that don't belong to the Python standard library) is managed by a
package manager such as `dpkg` or `rpm`. This is typically the case on
Linux distributions (unless you are using a Python you compiled yourself).
In that case, permissions would require you to run `pip` as root, which
would modify files and directories in places that are normally under
complete control of the package manager, “behind its back”. That would
likely lead to problems with your Python installation.

For this reason, in such cases where the whole Python installation is
managed by a package manager, we advise you to perform the installation
inside a `venv`—which is very quick and easy, see below.

#### Installation Inside a Venv

According to the above, you may possibly skip this section.

A Python [venv](https://docs.python.org/3/library/venv.html) (a kind of
virtual environment) is a directory tree that is linked to a Python
installation and can be used to install packages in a way that doesn't
alter the Python installation itself nor other venvs. Venvs are quick and
easy to create. You can store them where you see fit. They are convenient
for installing Python packages without messing with the rest of the
system. Another strong point of venvs (probably the initial motivation) is
that using several ones, you can install Python packages that have
incompatible dependencies (each in its own venv).

Installation of Python packages inside a venv is very easy:

1. First, you create a venv in some some directory `⟨dir⟩` of your choice:

        python3 -m venv ⟨dir⟩

   When using per-project venvs, people often choose the `.venv`
   subdirectory of the project root.

2. Then you use the `pip` executable from the venv, for instance:

        ⟨dir⟩/bin/pip install flightgear

> [!note]
> On Windows, the `bin` directory is called `Scripts`.

After these commands, `⟨dir⟩/bin` contains all scripts declared as such in
the installed package. Besides, `⟨dir⟩/bin/python` is a Python interpreter
pretty much like the one from the base installation, except it has access
to all modules belonging to packages installed in the venv (not to
third-party modules from the underlying Python installation, unless the
venv was created with option `--system-site-packages`).

Uninstalling the package would be done with
`⟨dir⟩/bin/pip uninstall flightgear`, etc. If you have many commands to run
from the venv and don't want to write `⟨dir⟩/bin/` over and over, you may
want to read about `activate` in the
[venv manual](https://docs.python.org/3/library/venv.html#how-venvs-work).

In the rest of this documentation, assuming you've chosen to work in a
venv whose base directory is `⟨dir⟩`, whenever you see a `pip` command,
you can use it as is if you've *activated* the venv; otherwise, you should
read it as `⟨dir⟩/bin/pip` (`⟨dir⟩\Scripts\pip` on Windows).

After installing a package in a venv, if you want to have the installed
scripts in your `PATH`, there are basically three ways:

- create symbolic links from a directory that is in your `PATH`;

- add `⟨dir⟩/bin` to your `PATH`; this can be convenient if you use a
  “main venv” but before doing so, please note that `⟨dir⟩/bin` normally
  contains `python`, `python3` and `pip` executables that would then also
  be in your `PATH`—your call;

- in Bash or Zsh, you can run `. ⟨dir⟩/bin/activate` to add `⟨dir⟩/bin` to
  the `PATH` *only in the current shell;* do whatever you want in this
  shell, then run `deactivate` to undo the setting, or simply terminate
  the shell session. This method is also applicable to other systems using
  appropriate commands: see the [venv
  documentation](https://docs.python.org/3/library/venv.html#how-venvs-work)
  for more information.

If the underlying Python installation for a venv is updated (as in,
upgraded from Python X.Y to Python X.Y+1) or if the venv directory is
moved elsewhere (which includes one of its parents being moved or
renamed), you'll have to recreate the venv: simply delete the `⟨dir⟩`
directory and redo the preceding steps (if you use a “main venv” with a
bunch of packages, having a script to recreate it is convenient).

#### Installing Using `pip`

The Python distribution package this document belongs to is called
`flightgear` (case doesn't matter) and normal installations should be done
with [pip](https://pip.pypa.io/), for instance:

    pip install flightgear

This would install the latest release. It is also possible to install
using the URL of a Git repository:

    pip install git+https://gitlab.com/flightgear/fgmeta-python.git@next

(here, `@next` could be removed as `next` is the default branch). Another
interesting possibility is to install from a local clone of the
repository. Assuming you have one in directory `/path/to/fgmeta-python`,
you could install the package with:

    pip install /path/to/fgmeta-python

(This may show an error due to outdated packages; in this case, run
`pip install -U build pip setuptools` and retry.)

Finally, given the target audience of this package and since releases may
not be that frequent, an interesting variation is to add option `-e`, like
so:

    pip install -e /path/to/fgmeta-python

This performs the installation in *editable mode*. This has the
consequence that updates to Python files that belong to the package under
`/path/to/fgmeta-python` are immediately visible to the Python
installation or venv in which you installed the package. This is perfect
for developing the package, or if you want to get the latest state of the
code by simply updating the `fgmeta-python` repository.

Installations done with `pip` perform two things:

- they install Python modules so that Python can import them (here, the
  `flightgear` import package would be in Python's `sys.path`);

- they create scripts in the `bin` subdirectory (`Scripts` on Windows) of
  the Python installation or venv: this is the case for scripts declared
  as such in the `pyproject.toml` file.

Another possibility that doesn't use `pip`, explained below in more
detail, is to modify `PYTHONPATH` yourself; however, this only covers the
first item of the previous list.

#### Upgrading Using `pip`

If you installed a release of the package, upgrading it can be done with
the following command, where option `-U` is a shorthand for `--upgrade`:

    pip install -U flightgear

If, on the other hand, you installed in editable mode from a clone of the
`fgmeta-python` repository, simply update the repository (for changes like
installed scripts or dependencies being added to or removed from
`pyproject.toml`, you'll need to redo the `pip install -e` step too).

#### Uninstalling Using `pip`

Uninstallation of the package using `pip` can be done with:

    pip uninstall flightgear

(yes, even if you installed with `pip install -e /path/to/fgmeta-python`).

#### Partial Installation Without `pip`

This section describes a partial installation method. It may be helpful in
some cases but isn't equivalent to normal installation with `pip`.

This method consists in modifying the `sys.path` value seen by the Python
interpreter, either by adding an element (directory) to the `PYTHONPATH`
environment variable or by creating a `.pth` file. This has a few
drawbacks as compared to the installation with `pip`:

- it won't warn you if your Python version is unsuitable for running the
  code contained in the package;

- it won't warn you if you don't have required dependencies (whereas `pip`
  would automatically install them for you);

- it won't create the scripts declared in the `pyproject.toml` file
  (however, they can currently be invoked as Python modules).

So, how does it work? For instance, you can use something like the
following in your shell setup:

    export PYTHONPATH="/path/to/fgmeta-python/src"

This example uses Bourne-style shell syntax; adjust for your particular
shell. Several directories may be added this way using a colon separator
on Unix or macOS (`:`), and a semicolon on Windows (`;`).

An alternative to setting `PYTHONPATH` is to add `.pth` files in special
directories of your Python installation(s). For instance, you could create
a file, say, `fgmeta-python.pth`, containing a single line (with no space
at the beginning):

    /path/to/fgmeta-python/src

If you want the modules present in `/path/to/fgmeta-python/src` to be
accessible to a particular Python interpreter (say, a Python 3.13), simply
put the `.pth` file in
`/path/to/python-install-dir/lib/python3.13/site-packages/`. For the
system Python interpreters on Debian, you can put the `.pth` file in, e.g,
`/usr/local/lib/python3.13/dist-packages/`. You may put several lines in a
`.pth` file in case you want to add several paths to the Python
interpreter's `sys.path`.

Note that if you use this method, you won't have the scripts declared in
the `pyproject.toml` file that `pip` would create in normal installations.

### The Scripts

Regardless of the method chosen for installation, the Python modules from
`fgmeta-python/src` should be available to the chosen Python interpreter
(if you installed in a venv whose base directory is `⟨dir⟩`, the
interpreter is `⟨dir⟩/bin/python`). What about scripts that rely on these
modules?

Normally, scripts are declared in the `pyproject.toml` file and
automatically created by the `pip install` command in the `bin`
subdirectory of the Python installation or venv. These scripts work out of
the box, can be invoked directly.

There are a few other scripts which exist as files in the repository rather
than being declared in `pyproject.toml`. As long as the required modules are
available to the Python interpreter in use (i.e., accessible via its
`sys.path`), such scripts will work too. For instance, if you installed the
package in a `⟨dir⟩` venv, `⟨dir⟩/bin/python` is a Python interpreter that can
see the modules, therefore `⟨dir⟩/bin/python some-script` would be a suitable
command for running `some-script`.

The only remaining problem is therefore the following: if you used the
partial installation method, you don't have the scripts that `pip install`
would have normally created for you. However, these scripts are currently
available as Python modules. For instance, the file
`src/flightgear/meta/scripts/i18n/fg_extract_translatable_strings.py`
can be imported as a Python module. Example: the command

    python3 -m flightgear.meta.scripts.i18n.fg_extract_translatable_strings --help

is equivalent to

    fg-extract-translatable-strings --help

This might be useful for one-off uses of the scripts where one doesn't
necessarily want to run `pip`: one could set `PYTHONPATH` and run the
desired module(s) as shown above (the obvious alternative being to create
a temporary venv and use the normal installation method).

### Running the Unit Tests

Once the modules from this package are visible to Python (regardless of
the method used), the unit tests can be run with the following command
from the root of the `fgmeta-python` repository:

    python3 -m unittest

(this is equivalent to `python3 -m unittest discover`; for more details,
see `tests/README.md`).

### Building the Package

In case you want to build the package, you can run the following command
from the root of the `fgmeta-python` repository:

    python3 -m build

This requires the Python `build` tool as explained in the
[Python Packaging User Guide](https://packaging.python.org/en/latest/guides/distributing-packages-using-setuptools/#packaging-your-project).
