# Wasot Dependency Analysis Report

## Overview

This report analyzes the 30 non-dev dependencies in the `wasot-compiler` crate, explaining the purpose and necessity of each in the final binary. Dependencies are grouped by functional area.

---

## CLI & User Interface

### clap (4.4) with `derive` feature
Clap is the command-line argument parsing library that powers Wasot's CLI interface. It provides the declarative derive macros used to define subcommands like `xml-listener`, `to-sqlite`, `to-jsonl`, and `select`. The derive feature enables the ergonomic `#[derive(Parser)]` syntax for defining CLI structures, automatic help generation, and argument validation. Without clap, users would have no way to interact with the tool from the command line.

### clap_complete (4.5.65)
This companion crate to clap generates shell completion scripts for bash, zsh, fish, and other shells. It enables Wasot to provide tab-completion for commands and arguments, significantly improving the developer experience when using the CLI frequently. Users can generate completions for their preferred shell to auto-complete subcommands and file paths.

---

## Parsing & Grammar

### pest (2.7)
Pest is a Parsing Expression Grammar (PEG) parser generator that forms the backbone of Wasot's selector system. The CSS-like selector syntax (e.g., `.paragraph`, `.block > .nested`) is defined in `selectors.pest` grammar files and parsed using pest's efficient parsing algorithms. Pest provides zero-copy parsing and excellent error messages, making it ideal for the selector DSL.

### pest_derive (2.7)
This crate provides the `#[derive(Parser)]` macro for pest grammars. It reads `.pest` grammar files at compile time and generates the Rust parsing code, eliminating runtime grammar compilation overhead. The selector parser in `src/selectors/parser.rs` uses this to convert the grammar into efficient Rust code.

### regex (1.10)
Regular expressions are used throughout Wasot for pattern matching in text processing, selector matching, and input validation. The regex crate provides a fast, safe implementation with excellent Unicode support. It's used in the parser for identifying block types, extracting arguments, and matching text patterns in selectors.

---

## Serialization & Output Formats

### serde (1.0) with `derive` feature
Serde is Rust's universal serialization framework that provides the `Serialize` and `Deserialize` traits. Wasot uses serde to serialize events, metadata, and document structures for output in multiple formats. The derive feature enables automatic implementation generation with `#[derive(Serialize, Deserialize)]`, used extensively on event types and configuration structures.

### serde_json (1.0)
This crate enables JSON serialization through serde. It powers the `to-jsonl` command that converts Wasot documents to JSON Lines format, where each event becomes a JSON object. JSON output is essential for interoperability with other tools, data pipelines, and analysis systems that consume JSON natively.

### serde_yaml (0.9)
YAML serialization support for configuration files and human-readable output. While JSON Lines is preferred for machine processing, YAML provides a more readable format for configuration and debugging scenarios where developers need to inspect the document structure manually.

### xml-rs (0.8)
A pure-Rust XML library used by the `xml-listener` command to generate well-formed XML output from Wasot documents. This enables conversion from Wasot's markup format to standard XML, allowing integration with XML-based toolchains, XSLT transformations, and systems that require XML input.

### hex (0.4)
Provides hexadecimal encoding and decoding utilities. Used in Wasot's ID generation and base36 encoding systems for converting binary data to readable string representations. This is essential for the document-id system that generates unique, sortable identifiers for blocks.

---

## Async Runtime & Concurrency

### tokio (1.0) with `full` feature
Tokio is the async runtime that powers Wasot's concurrent event processing. The streaming architecture uses Tokio's channels for bounded event queues between emitters and consumers, ensuring constant memory usage regardless of document size. The `full` feature includes the multi-threaded scheduler, I/O drivers, timers, and synchronization primitives needed for the SQLite consumer and gRPC services.

### async-trait (0.1)
Rust doesn't natively support async methods in traits, so async-trait provides a proc-macro workaround. This is used to define the `Emitter` and `Consumer` traits with async methods, allowing different implementations (WasotParser, SqliteEmitter, XmlListener) to share a common interface while supporting async operations.

### bus (2.4)
A broadcast channel implementation that allows multiple consumers to receive the same events. In Wasot's event-driven architecture, this enables fan-out patterns where a single emitter can broadcast events to multiple consumers simultaneously—for example, writing to both SQLite and JSONL outputs from the same parse.

### dashmap (6.1)
A concurrent, lock-free hashmap implementation. Used in the tool registry and event routing systems where multiple threads may read and write simultaneously. DashMap provides better performance than `Mutex<HashMap>` for read-heavy workloads, which is common in the streaming event pipeline.

---

## Database

### sqlx (0.7) with `runtime-tokio-rustls`, `sqlite`, `json`, `chrono` features (optional)
SQLx is an async, compile-time checked SQL toolkit used for the SQLite storage backend. The `to-sqlite` command uses SQLx to persist Wasot documents with full query support. The `chrono` feature enables timestamp handling, `json` enables JSON column types, and `runtime-tokio-rustls` integrates with Tokio and provides TLS support. This dependency is optional and only included with the `sqlite` feature flag.

---

## gRPC & Protocol Buffers

### tonic (0.11)
Tonic is a gRPC implementation built on Tokio. Wasot uses it for the calculator-server/client demos and potentially for distributed processing scenarios. gRPC enables efficient binary RPC communication between services, useful for building wasot-based document processing pipelines across networked services.

### prost (0.12)
The Protocol Buffers implementation for Rust. Prost generates Rust structs from `.proto` files and handles serialization/deserialization of protobuf messages. This is the underlying serialization format for gRPC communication in Wasot's distributed components.

### prost-types (0.12)
Provides Rust types for Protocol Buffers well-known types like `Timestamp`, `Duration`, and `Any`. These standard types are commonly used in gRPC service definitions for consistent handling of dates, times, and polymorphic message types.

### tower (0.4)
Tower is a middleware framework for building robust clients and servers. It provides abstractions for timeouts, rate limiting, load balancing, and retry logic. Wasot's gRPC services use Tower to compose middleware stacks that handle cross-cutting concerns without polluting business logic.

### tower-service (0.3)
The core `Service` trait from the Tower ecosystem, separated into its own crate for minimal dependencies. This trait represents an async request/response handler and is fundamental to how Tonic and Tower compose services. It enables Wasot's services to be wrapped with middleware.

---

## Error Handling & Logging

### thiserror (1.0)
A derive macro for implementing `std::error::Error` with minimal boilerplate. Wasot defines custom error types for parsing failures, I/O errors, selector errors, and database errors. Thiserror generates the `Display` and `Error` trait implementations automatically, producing consistent, informative error messages throughout the codebase.

### tracing (0.1)
A structured logging and diagnostics framework. Unlike traditional logging, tracing captures spans (execution scopes) and events with structured fields, enabling detailed performance analysis and debugging. Wasot uses tracing to instrument the event pipeline, making it possible to trace an event from parsing through to output.

### tracing-subscriber (0.3)
Collects and formats tracing data for output. This subscriber implementation connects the tracing instrumentation to actual log output, supporting JSON formatting, filtering by level/target, and integration with log aggregation systems. Essential for observability in production deployments.

### env_logger (0.11)
A simple logger configured via environment variables. Provides an alternative to tracing for simpler logging needs, controlled by the `RUST_LOG` environment variable. Useful during development for quick debugging without full tracing infrastructure.

---

## Date/Time & Utilities

### chrono (0.4) with `serde` feature
Comprehensive date and time handling library. Wasot uses chrono for timestamping events, tracking document modification times, and handling date fields in SQLite storage. The `serde` feature enables automatic serialization of date/time types in JSON and other formats.

### tempfile (3.0)
Creates temporary files and directories with automatic cleanup. Used in testing and for intermediate processing steps where temporary storage is needed. Important for the SQLite roundtrip operations and file conversion pipelines that need scratch space.

### rand (0.8)
Random number generation for unique ID creation and test data generation. Wasot's ID system uses randomness to ensure uniqueness across distributed systems. Also used in fuzzing tests and generating sample documents for performance testing.

### rlimit (0.10)
Allows querying and setting resource limits (ulimits) on Unix systems. Used to manage memory and file descriptor limits during processing of large documents, preventing runaway resource consumption and providing graceful degradation when limits are approached.

---

## Optional: Memory Optimization

### mimalloc (0.1) with `optimized-alloc` feature (optional)
A high-performance memory allocator from Microsoft that can replace the system allocator. When enabled via the `optimized-alloc` feature, mimalloc provides faster allocation/deallocation and reduced memory fragmentation for long-running processes. Particularly beneficial when processing millions of events where allocation overhead becomes significant.

---

## Optional: Python Bindings

### pyo3 (0.22) with `extension-module` feature (optional)
PyO3 enables building Python extension modules in Rust. When the `python` feature is enabled, Wasot compiles as a Python module (`pywasot`) that can be imported directly into Python code. This enables Python scripts to parse Wasot documents, register tools, and process events using Python callbacks while benefiting from Rust's performance.

---

## Summary

| Category | Dependencies | Purpose |
|----------|-------------|---------|
| CLI | clap, clap_complete | User interface |
| Parsing | pest, pest_derive, regex | Grammar & pattern matching |
| Serialization | serde, serde_json, serde_yaml, xml-rs, hex | Output formats |
| Async | tokio, async-trait, bus, dashmap | Concurrent streaming |
| Database | sqlx | SQLite persistence |
| gRPC | tonic, prost, prost-types, tower, tower-service | Distributed processing |
| Errors/Logging | thiserror, tracing, tracing-subscriber, env_logger | Diagnostics |
| Utilities | chrono, tempfile, rand, rlimit | Core utilities |
| Optional | mimalloc, pyo3 | Performance & Python |

The dependency set reflects Wasot's architecture as a high-performance, event-driven streaming processor with multiple output formats and extensibility through Python bindings.
