# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

---

## [1.2.0] — 2026-03-25

### Added

- **GLiNER2 entity extraction (NER)** — inline named entity annotations on every element and chunk record
  - `EntityAnnotation` schema model: `entity_type`, `text`, `start`, `end`, `score`, `extraction_method`, `source_element_id`
  - `entities: list[EntityAnnotation] | absent` on `BaseElement` and `ChunkRecord`
  - `chunk_metadata.entity_types: list[str] | absent` — sorted deduplicated label list for fast vector store payload filtering
  - Local inference: `fastino/gliner2-base-v1` via `[ner]` extras (no external API required)
  - Element type gating: code blocks, equations, media, page headers/footers, form fields skipped automatically
  - Default label set: `organization`, `person`, `date`, `location`, `product`, `technology`, `event`, `quantity`
  - Graceful fallback: if `[ner]` extras not installed, extraction continues with `W_NER_MODEL_UNAVAILABLE` manifest warning
  - New `[ner]` extras: `pip install "coresdk-extractor[ner]"`
- **CLI NER flags**: `--entities/--no-entities`, `--entities-model`, `--entities-types`, `--entities-threshold`
- **LangChain adapter** (`extractor.integrations.langchain`): `entity_types`, `entities`, `section_path_str` added to Document metadata
- **LlamaIndex adapter** (`extractor.integrations.llamaindex`): `entity_types`, `entities` added to TextNode metadata; `excluded_llm_metadata_keys` set to keep entity JSON out of LLM synthesis prompts
- **`extract()` API**: `tokenizer` and `inject_context_prefix` parameters now correctly forwarded to `pipeline.run()` (were silently dropped before)
- **`OutputWriter` context manager**: `__enter__`/`__exit__` support — temp file cleaned up automatically on exception
- **Schema backward compatibility**: `schema_version` validator now accepts `1.0.0` and `1.1.0` (previously rejected any version other than current)

### Changed

- **Schema version**: `1.0.0` → `1.1.0` (MINOR bump — all new fields are optional, fully additive)
- **JSONL serialization**: switched to `exclude_none=True` — absent optional fields are omitted from output instead of emitted as `null`
- **Manifest write**: atomic write via `.tmp` → rename, matching JSONL output pattern
- **`_collect_files`**: directory scan now excludes the output directory and `.jsonl`/`.manifest.json`/`.tmp` files to prevent re-parsing previously extracted output

### Fixed

- `aextract()` async generator: executor future was not awaited — exceptions from the worker thread were silently swallowed and queue had no backpressure (`put_nowait` → `await queue.put`)
- `_reassemble_table_continuations`: `table:continuation` elements were silently dropped when their parent table was absent or filtered — orphaned continuations are now emitted as-is
- `chunk_elements`: overlap tail token count not included in size guard — chunks could silently exceed `max_tokens` when overlap was enabled
- `quarantine.check_zip_bomb`: streaming decompression guard used `>` instead of `>=` (exact-size bombs bypassed); members with `compressed==0` and `uncompressed>0` now raise `QuarantineError`
- NER model singleton: API-tier model was cached under the local model ID instead of `"__api__"` key — could cause wrong model to be returned on subsequent calls
- Integration adapters: accessing `chunk.chunk_metadata` on a `BaseElement` (when `mode="elements"` is mistakenly passed) now raises a clear `TypeError` instead of `AttributeError`

### Security

- Zip bomb detection hardened: exact-ceiling bypass fixed (`>` → `>=`); zero-compressed-size members now rejected

---

## [0.1.0] — 2026-03-24

### Added

- Initial release
- PDF, DOCX, XLSX, PPTX, HTML, Markdown, EPUB, LaTeX, JSON, XML, CSV, plain text parsers
- Section-aware chunker with atomic element protection
- Content-addressed element and chunk IDs
- JSONL streaming output with manifest
- `extractor run`, `view`, `validate`, `info`, `schema` CLI commands
- LangChain and LlamaIndex integration adapters
- Quarantine layer (zip bomb, XXE, SSRF protection)
- Schema v1.0.0
