# Lodum Roadmap

This document outlines the planned future enhancements for `lodum`. Our goal is to maintain the library's high performance while expanding its utility in the Python ecosystem.

## 0. Completed Milestones

### v0.2.0 — First Public Release
- **AST Bytecode Compiler**: Replaced string-based `exec` with a robust and maintainable AST generation engine using a specialized DSL.
- **Modular Architecture**: Refactored the core engine into decoupled `compiler/` and `handlers/` subpackages.
- **Thread Safety**: Implemented thread-local `Context` management with a lock-free fast path for cache performance.
- **Centralized Schema API**: Promoted `lodum.schema()` as the primary entry point for JSON Schema generation.
- **Extended Type Support**: Added native support for `numpy`, `pandas`, `polars`, and numerous standard library types.
- **Community & Governance**: Established Code of Conduct, Security policy, and standard issue/PR templates.

### v0.3.0 — WASM & Concurrency Support
- **Robust Concurrency Detection**: Implemented active probing for native threading capability in Pyodide/WASM environments.
- **WASM Shims**: Introduced `SequentialThread`, `DummyLock`, and `DummyLocal` to ensure graceful degradation in restricted environments.
- **Unified Threading API**: Refactored core internals to use a cross-platform concurrency abstraction, enabling the same code to run on high-concurrency servers and in single-threaded browsers.

## 1. Current Milestone (v0.4.0+)

### Streaming Support ([Plan](plans/streaming_support.md))

* **Goal**: Enable processing of datasets larger than memory.
* **Implementation**: Add `dump_stream(obj, stream)` and `load_stream(cls, stream)` to the IDumper/ILoader protocols.
* **Target Formats**: JSON (via `ijson` or similar), MsgPack, CBOR.

### Partial Serialization ([Plan](plans/partial_serialization.md))

* **Goal**: Allow users to serialize only a subset of fields or only fields that have changed.
* **API Idea**: `dumps(obj, include={'id', 'name'})` or `field(serialize_if_changed=True)`.

## 2. Short-Term Goals (v0.5.0+)

### Data Science & ML Optimization ([Plan](plans/memory_aware_serialization.md))

* **Goal**: Enable efficient handling of multi-gigabyte quantized datasets in memory-constrained environments (Pyodide/WASM).
* **Key Components**:
    * **Lazy Loading**: Two-tier on-demand materialization (Tensor proxies and Complex Object proxies).
    * **Quantization Handlers**: Native AST-generated loaders for 4-bit, 8-bit, and bit-packed data.
    * **Zero-Copy Architecture**: Integrated `mmap` and `memoryview` views to prevent memory expansion.

### Framework Integrations

* **FastAPI** ([Plan](plans/fastapi_integration.md)): Create a `LodumResponse` class and a `Depends` provider for automatic parsing.
* **OpenAPI Support** ([Plan](plans/openapi_support.md)): Utilities to convert `lodum` schemas directly into OpenAPI/Swagger specifications.

## 3. Medium-Term Goals (v0.7.0+)

### Advanced Schema Features

* **Schema-to-Code** ([Plan](plans/schema_to_code_cli_tool.md)): A CLI tool to generate `@lodum` classes from an existing JSON Schema.

### Compiler Enhancements

* **Native Handlers** ([Plan](plans/compiler_enhancements_cython_mypyc.md)): Explore compiling the generated bytecode handlers into C via Cython or MyPyC for even greater speed.

## 4. Long-Term Vision (v1.0.0)

### Ecosystem & Tooling

* **Protocol Buffers** ([Plan](plans/protocol_buffers_support.md)): Native Protobuf support via the AST compiler.
* **IDE Plugin** ([Plan](plans/ide_plugin.md)): A VS Code / PyCharm plugin to auto-generate `field()` definitions and validate schema compliance in real-time.

## Contributing

We welcome contributions on any of these items! Please open an issue to discuss your implementation plan before submitting a PR.
