# `0.2.1` Release Plan

## Summary

`v0.2.1` is a focused LLM-stabilization release for `paralabelgen`. The main
goal is to make the opt-in LLM extraction path more reliable, diagnosable, and
safer to use in iterative evaluation workflows, without redesigning the rest of
the labeling pipeline.

This release keeps the public `fit()`, `transform()`, and `fit_transform()`
methods synchronous and preserves `ConceptMention` as the downstream extraction
contract. The LLM path remains additive and explicit rather than becoming the
default extractor.

## Key Decisions

- The target release is `0.2.1`.
- The release priority is LLM stability first.
- Extraction remains hybrid:
  - `spacy`
  - `heuristic`
  - `llm`
- The LLM path remains opt-in through top-level configuration.
- Provider support remains:
  - `openai`
  - `mistral`
  - `qwen`
- The provider layer stays unified around one OpenAI-compatible client in
  `0.2.1`.
- Public pipeline methods remain synchronous in `0.2.1`.
- Cache and artifact behavior remain available for experimental workflows, but
  gain stronger output-safety controls.

## Implementation Shape

### Structured LLM Output

- Upgrade the provider-backed extraction path from prompt-shaped JSON only to
  provider-enforced structured output where supported.
- Keep the current prompt guidance, but treat it as a secondary guard rather
  than the primary extraction contract.
- Preserve the existing `ConceptMention` downstream shape.
- Fail clearly when provider output is invalid or does not match the expected
  paragraph-aligned schema.
- If an OpenAI-compatible endpoint explicitly rejects JSON-schema response
  formatting, allow one fallback retry on the same `llm` path without
  `response_format`, rather than silently switching extractor modes.
- Do not silently fall back from `llm` to `spacy` or `heuristic`.

### Provider Diagnostics

- Improve provider error reporting so request failures are actionable instead of
  collapsing into generic runtime errors.
- Include provider name in surfaced request failures.
- Preserve HTTP status information when available.
- Include a compact response-body summary when it is safe to surface.
- Distinguish configuration failures, transport failures, retry exhaustion, and
  parse failures.
- Make retry-related diagnostics explicit enough for iterative evaluation and
  debugging.

### Cache And Artifact Safety

- Add explicit config controls for whether raw response text is written to
  artifacts.
- Add explicit config controls for whether paragraph text and paragraph metadata
  are written to artifacts.
- Keep cache support for parsed concept outputs, but document its intended use
  in local evaluation and experiment replay.
- Clarify that artifact recording remains optional and should not be enabled by
  default for routine usage.
- Keep the current cache/artifact directories configurable and make the safer
  defaults more visible in documentation and examples.

### Cleanup And Documentation

- Remove clearly unused runtime dependencies such as `networkx`.
- Treat `networkx` as a dead dependency in the released implementation:
  - the current graph builder already emits the project-local `ConceptGraph`
  - the default community-detection path uses `igraph` and `leidenalg`
  - the fallback connected-components path already uses project-local graph
    traversal logic
  - `networkx` is not currently required for graph construction, Leiden
    detection, or fallback connected-components detection
- Update README to explain:
  - the current unified OpenAI-compatible provider model
  - structured-output expectations for the LLM path
  - recommended low-risk LLM configuration for evaluation workflows
  - recommended debugging-oriented LLM configuration when artifact capture is
    needed
- Update `docs/public_api.md` to describe new safety and diagnostics-related LLM
  config fields.

## Testing Goals

- Verify structured-output request shaping in the provider client.
- Verify extraction still returns deterministic `ConceptMention` objects under
  the enforced output path.
- Verify provider failures preserve useful diagnostic context:
  - provider
  - retry exhaustion
  - status code when available
  - compact response summary
- Verify cache hits still avoid repeated provider calls after structured-output
  changes.
- Verify artifact safety controls suppress the intended fields without breaking
  artifact generation.
- Verify serialization preserves new LLM safety and diagnostics config.
- Verify extractor mode selection remains backward compatible.
- Add provider-response fixtures for OpenAI-compatible success and failure
  shapes.
- Keep live-provider smoke tests out of the default test suite, but document a
  small optional smoke-test path for manual verification.

## Notes

- This release does not yet split provider handling into vendor-specific
  adapters.
- This release does not yet add async public APIs.
- This release does not yet redesign downstream graph or label result schemas.
- This release does not make `llm` the default extractor mode.
- This document is a planning artifact for repository work and not part of the
  published package API.
