# CaDaR Agent Guidelines

CaDaR is a high-performance bidirectional transliteration library for Darija (Moroccan Arabic) with Rust core and Python bindings via PyO3. Uses a 6-stage pipeline with Intermediate Canonical Representation (ICR).

## Essential Commands

```bash
# Build
./build.sh dev      # Development (fast)
./build.sh release  # Release (optimized)
maturin develop     # Direct development build

# Test
cargo test                                    # All tests (41 tests)
cargo test test_name -- --nocapture          # Single test with output
cargo test script_detection                  # Module tests
RUST_LOG=debug cargo test -- --nocapture     # Debug tests
python -c "import cadar; print(cadar.ara2bizi('سلام', darija='Ma'))"

# Lint/Format
cargo fmt           # Format code
cargo clippy        # Lint (run before commits)

# Documentation
mkdocs serve        # Serve docs locally
```

## Code Style Guidelines

### Project Structure
- Rust core in `src/` with modules: `lib.rs`, `types/`, `stages/`, `linguistic/`
- Python bindings in `src/python_bindings.rs`
- Python package in `python/cadar/`
- Tests in `#[cfg(test)]` modules at file bottom
- Dataset processing scripts in `scripts/`

### Imports
- External crates first, then internal modules
- Group by category with blank lines between
```rust
use serde::{Deserialize, Serialize};
use std::fmt;
use std::str::FromStr;

use crate::types::Script;
```

### Naming Conventions
- Structs/Enums: PascalCase (CaDaR, Normalizer, Script, Dialect)
- Functions/Methods: snake_case (ara2bizi, transliterate, normalize)
- Module files: snake_case (script_detection.rs, normalization.rs)
- Private methods: snake_case with leading underscore if internal
- Constants: SCREAMING_SNAKE_CASE

### Types
- Derive Debug, Clone, Copy for enums/simple structs
- Derive PartialEq, Eq, Hash for comparison types
- Use serde Serialize/Deserialize for data types
```rust
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub enum Script { Arabic, Latin, Mixed, Unknown }
```

### Error Handling
- Use `StageResult<T>` = `Result<T, StageError>` for pipeline operations
- Define errors in `stages/mod.rs` StageError enum
- Wrap external errors with context:
```rust
Err(StageError::Normalization(format!("Failed: {}", e)))
```

### Documentation
- Use `///` for public APIs with brief descriptions
- Document pipeline stages with comments
- Add examples for complex functions

### Testing
- Place tests in `#[cfg(test)]` modules at file bottom
- Test both success and error paths
- Use descriptive test names: `test_ara2bizi`, `test_round_trip`
```rust
#[test]
fn test_ara2bizi() {
    let cadar = CaDaR::new(Dialect::Moroccan);
    let result = cadar.ara2bizi("سلام").unwrap();
    assert!(!result.is_empty());
}
```

### Python Bindings
- All public Rust functions need Python wrappers in `python_bindings.rs`
- Use `#[pyfunction]` and return `PyResult<T>`
- Convert errors consistently:
```rust
#[pyfunction]
fn ara2bizi(text: &str, darija: Option<String>) -> PyResult<String> {
    let dialect = parse_dialect(darija)?;
    let cadar = CaDaR::new(dialect);
    cadar.ara2bizi(text).map_err(|e| PyValueError::new_err(e.to_string()))
}
```

## Architecture Pipeline

6-stage pipeline: Script Detection → Normalization → Tokenization → ICR Building → Script Generation → Validation

- Stage errors bubble up via `?` operator
- Dialect passed through all stages
- Use `StageResult<T>` for return types

## Important Gotchas

- Always test: `cargo test && python -c "import cadar; print('OK')"`
- Use `maturin develop` for iterative development (faster than release builds)
- Unicode normalization: `text.nfc().collect::<String>()`
- Handle zero-width characters in Arabic text
- Version numbers in both Cargo.toml and pyproject.toml
- No rustfmt.toml - use standard rustfmt defaults
- Empty string inputs should return Err(StageError::InvalidInput)
- Releases automatically trigger dataset processing workflow that:
  * Loads arbml/darija dataset from HuggingFace
  * Applies bizi2ara transformation
  * Pushes to OiTe/CaDaR dataset with versioning
  * Dataset processing failures will block the release

## Key Dependencies

- PyO3 for Python bindings
- Serde for serialization
- Unicode libs for text processing
- Thiserror for error handling
- Regex for pattern matching
- Python 3.8+ support required
- Crate type: ["cdylib", "rlib"] for Python bindings
