# Migration to Python 3.12, Pydantic v2, and uv tooling

This document describes the migration steps to perform on this project according to past experience. Please apply all these changes if they make sense.

## 0. Initial setup
- git pull the current master branch
- create a py312 git branch and switch to it
- change pyenv local to pymultirole312
- reset version in __init__.py to 0.6.0

## 1. Build system: flit → hatchling/uv

The build system was migrated from **flit** to **hatchling** with **uv** as the package manager.

- `pyproject.toml`: rewritten from flit legacy format (`[tool.flit.metadata]`) to PEP 621
  with `hatchling` backend and `[project]` table
- Version read dynamically from `src/pyconverters_pyword/__init__.py` via `[tool.hatch.version]`
- Entry point `pyconverters.plugins` moved from `[tool.flit.entrypoints]` to `[project.entry-points]`
- Replaced `flit` commands with `uv build` and `uv publish`
- Jenkinsfile updated to use `uv sync` and `uv build`/`uv publish`
- Dockerfile updated to install `uv` via `COPY --from=ghcr.io/astral-sh/uv:latest`

## 2. Python version: 3.8+ → 3.12

- `requires-python` set to `>=3.12` in `pyproject.toml`
- `Dockerfile` base image changed from `python:3.8-slim-bookworm` to `python:3.12-slim-bookworm`
- `FLIT_ROOT_INSTALL=1` env var removed from Dockerfile (no longer needed)
- Classifiers updated from `Python :: 3.8` to `Python :: 3.12`

## 3. Pydantic v1 → v2

Replaced deprecated Pydantic v1 APIs in `pyword.py` and `tests/test_pyword.py`:

- `.json(exclude_none=True, exclude_unset=True, indent=2)` → `.model_dump_json(...)` in tests
- `DocumentList(__root__=docs)` → `DocumentList(root=docs)` (Pydantic v2 `RootModel` API)
- `Field(extra="internal"/"advanced")` → `Field(json_schema_extra={"extra": "..."})` in `pyword.py`
- `params.dict()` → `params.model_dump()` in `pyword.py`

## 3bis. Spacy
For everything related to spacy look at what has been done in ../pysegmenters_rules_segmenter

## 4. Python 3.12 type modernization

Leveraged Python 3.12 builtins and modern syntax (enforced by ruff `UP` rules):

- `typing.List[Document]` → `list[Document]` (method signatures in `pyword.py` and tests)
- `typing.Type[BaseModel]` → `type[BaseModel]` (`get_model` return type in `pyword.py`)
- Removed unused `typing` imports: `List`, `Type`

## 5. Linter: black + flake8 → ruff

- Removed `black`, `flake8`, `pytest-flake8`, `pytest-black`, `flakehell` dependencies
- Added `ruff` to `[project.optional-dependencies].test`
- Removed `[tool.flakehell]` configuration block from `pyproject.toml`
- Configured `[tool.ruff]` in `pyproject.toml`:
  - `line-length = 120`
  - `target-version = "py312"`
  - Lint rules: `E`, `W`, `F`, `I`, `B`, `C4`, `UP`, `ARG`, `SIM`
  - Format: double quotes, space indent
- Updated `.pre-commit-config.yaml`: replaced `black` and `flake8` hooks with `ruff` and `ruff-format`

### Running the linter

```
uv run ruff check .
uv run ruff format --check .
```

To auto-fix formatting:

```
uv run ruff format .
```

## 6. Test runner: tox → uv

**tox** was removed entirely. The project now uses `uv run` to execute tests and linting directly.

### What changed

- Deleted `tox.ini`
- Removed `tox`, `flake8`, `pytest-flake8`, `pytest-black` from test dependencies
- Moved pytest configuration from `tox.ini` to `pyproject.toml`:

```toml
[tool.pytest.ini_options]
addopts = "--durations=5"
norecursedirs = ["docs"]
```

### Running tests

```
uv run pytest
```

### Why tox was unnecessary

- The project targets a single Python version (3.12)
- `uv run` provides virtual environment isolation without the extra layer

## 7. Documentation generation

Sphinx is used for documentation with the `docs` optional dependency group.

### Fix: added `lxml_html_clean` dependency

The `jupyter_sphinx` extension requires `lxml_html_clean` (split from `lxml` in recent versions).
This was added to `[project.optional-dependencies].docs`.

### Generating docs

```
uv run --extra docs sphinx-build docs docs/_build
```

## 8. Jenkinsfile updates

- `__init__.py` generation now includes a blank line between the docstring and `__version__`
  to comply with `ruff format`
- Set `__version__ = "0.6.0"`
- In Jenkinsfile, set `MINOR_VERSION = "6"`
- Install step changed from `flit install` to `uv sync --extra test`
- Lint step added: `uv run ruff check .` + `uv run ruff format --check .`
- Test step changed from `tox` to `uv run pytest --junit-xml=...`
- Build/publish pipeline uses `uv build && uv publish` instead of `flit publish`
- Credentials: Jenkinsfile helper functions still read `FLIT_USERNAME` and `FLIT_PASSWORD`
  from `.passwd-pypi`, but expose them as `UV_PUBLISH_USERNAME` and `UV_PUBLISH_PASSWORD`
  for `uv publish`
- Removed `.tox` cleanup steps

## 9. .gitignore updates

Added the following entry:
- `uv.lock` — generated lock file, not committed

(`docs/_build/` was already present.)

## 10. Dependency cleanup and upgrades

### Removed unused dependencies

- None

### Upgraded pinned dependencies

- `pymultirole-plugins>=0.5.0,<0.6.0` → `pymultirole-plugins>=0.6.0,<0.7.0`
- update dependencies to latest versions if possible

## 11. Starlette `UploadFile` API change

The `UploadFile` constructor signature changed in recent Starlette versions:

**Before (old API):**
```python
UploadFile(filename, file, content_type)  # positional args
```

**After (new API):**
```python
UploadFile(file=..., filename=..., headers=Headers({"content-type": "..."}))
```

Updated in `tests/test_pyword.py` and added `Headers` import from `starlette.datastructures`.

## 12. Additional unit tests

- Analyze src directory and extended `tests/` with tests covering previously untested behaviour

## 13. Update README.md


## 14. Better documentation
Propose better and more informative description for Field parameters and component docstring (Annotator, Converter, etc...)

