# aifastdb

Python bindings for **AiFastDb** — a high-performance AI-friendly database engine written in Rust.

[![PyPI](https://img.shields.io/pypi/v/aifastdb.svg)](https://pypi.org/project/aifastdb/)
[![Python](https://img.shields.io/pypi/pyversions/aifastdb.svg)](https://pypi.org/project/aifastdb/)
[![License](https://img.shields.io/badge/license-AiFastDB-blue.svg)](LICENSE)

## Features

- **Core Database** — Key-value store with HNSW vector search, batch operations, and hybrid (text + vector) recall
- **SocialGraphV2** — Social network graph with path finding, network analysis, family trees, and concurrent sharding
- **DocumentStore** — JSONL-based document store with tag/type indexing and async flush
- **Federation** — Heterogeneous storage federation (Graph + Document + Vector) with cross-database queries and 2PC transactions
- **Embedding** — Built-in embedding generation (dummy / MiniLM / BGE / Gemma) with numpy output
- **Reasoning** — LLM-based reasoning via Ollama or OpenAI (intent, logic, summarize)
- **pandas integration** — One-line DataFrame conversion for any query result

## Installation

```bash
pip install aifastdb

# With optional extras:
pip install aifastdb[pandas]    # pandas DataFrame support
pip install aifastdb[numpy]     # numpy array support
pip install aifastdb[all]       # everything
```

## Requirements

- Python >= 3.8
- No additional dependencies for core functionality (self-contained Rust binary)

## Quick Start

### Core Database

```python
from aifastdb import AiFastDb

with AiFastDb("./my_db") as db:
    # Store a memory
    db.remember("doc1", "The quick brown fox", tags=["animal", "classic"])

    # Retrieve by ID
    doc = db.get("doc1")
    print(doc["content"])  # "The quick brown fox"

    # Hybrid search (text + vector)
    results = db.recall(query="brown fox")
    for r in results:
        print(f"  {r['node']['id']}  score={r['score']:.3f}")

    # Delete
    db.forget("doc1")
```

### Social Graph

```python
from aifastdb import SocialGraphV2

with SocialGraphV2("./social_data") as sg:
    # Create people
    alice = sg.add_person({"name": "Alice", "age": 30})
    bob   = sg.add_person({"name": "Bob",   "age": 28})
    carol = sg.add_person({"name": "Carol", "age": 32})

    # Add friendships
    sg.add_friend(alice["id"], bob["id"])
    sg.add_friend(bob["id"], carol["id"])

    # Find shortest path
    path = sg.find_path(alice["id"], carol["id"])
    print(f"Path hops: {path['hops']}")

    # Mutual friends
    mutual = sg.get_mutual_friends(alice["id"], carol["id"])
    print(f"Mutual friends: {[m['name'] for m in mutual]}")

    # Network stats
    stats = sg.get_network_stats(bob["id"])
    print(f"Bob's degree centrality: {stats['degree_centrality']:.2f}")
```

### Document Store

```python
from aifastdb import DocumentStore

with DocumentStore("./docs.jsonl") as store:
    store.put("note-1", "Meeting at 3pm", doc_type="note", tags=["work"])
    store.put("note-2", "Buy groceries",  doc_type="note", tags=["personal"])

    # Query by tag
    work_notes = store.find_by_tag("work")
    print(f"Work notes: {len(work_notes)}")

    # All documents
    all_docs = store.all()
    print(f"Total documents: {len(all_docs)}")
```

### Federation (Multi-Store)

```python
from aifastdb import Federation

with Federation() as fed:
    fed.register_graph("social", "./social_data")
    fed.register_document("docs", "./docs.jsonl")

    # Query across stores
    result = fed.query("social", {"target": "person", "limit": 10})
    print(f"Found {result['total']} persons")

    # Write to document store
    fed.write("docs", {
        "op": "CreateDocument",
        "id": "fed-doc-1",
        "content": "Created via federation",
        "doc_type": "note",
    })

    # Cross-store transaction
    tx = fed.begin_transaction(["social", "docs"])
    tx.write("social", {"op": "CreateEntity", "name": "Eve", "entity_type": "person"})
    tx.write("docs",   {"op": "CreateDocument", "id": "eve-doc", "content": "Eve's note"})
    tx.commit()
```

### Embedding Engine

```python
from aifastdb import EmbeddingEngine
import numpy as np

engine = EmbeddingEngine()  # dummy 384d engine

# Single embedding
vec = engine.embed("Hello, world!")
print(f"Dimension: {len(vec)}")

# Numpy output
arr = engine.embed_numpy("Hello, world!")
print(f"Shape: {arr.shape}, dtype: {arr.dtype}")

# Batch
vecs = engine.embed_batch(["Hello", "World", "Foo"])
print(f"Batch size: {len(vecs)}")

# Cosine similarity
sim = EmbeddingEngine.cosine_similarity(
    engine.embed("cat"),
    engine.embed("dog"),
)
print(f"Similarity: {sim:.4f}")
```

### Perception Pipeline (Embed + Remember)

```python
from aifastdb import PerceptionPipeline, DocumentStore

pipe = PerceptionPipeline()

# Auto-embed content
result = pipe.remember_with_embedding(
    "The mitochondria is the powerhouse of the cell",
    "bio-001",
    tags=["biology"],
)
print(f"Embedding dim: {result['dimension']}")

# Store the result in a DocumentStore
with DocumentStore("./knowledge.jsonl") as store:
    store.put(
        result["id"],
        result["content"],
        tags=result["tags"],
        doc_type=result["doc_type"],
    )
```

### pandas Integration

```python
from aifastdb import SocialGraphV2, DocumentStore
from aifastdb.pandas_ext import to_dataframe, graph_to_dataframes

# SocialGraph → DataFrame
with SocialGraphV2("./social_data") as sg:
    persons_df = to_dataframe(sg.list_persons())
    print(persons_df[["id", "name", "age"]])

    entities_df, relations_df = graph_to_dataframes(sg.export_graph())
    print(f"Entities: {len(entities_df)}, Relations: {len(relations_df)}")

# DocumentStore → DataFrame
with DocumentStore("./docs.jsonl") as store:
    docs_df = to_dataframe(store.all())
    print(docs_df.head())
```

## API Reference

### Core Classes

| Class | Description |
|-------|-------------|
| `AiFastDb` | Core database with collections, CRUD, and hybrid search |
| `Collection` | Named collection within AiFastDb |
| `SocialGraphV2` | Social graph with path finding and network analysis |
| `DocumentStore` | JSONL document store with tag/type indexing |
| `Federation` | Multi-store federation with unified query/write |
| `FederatedTransaction` | Cross-store 2PC transaction |
| `EmbeddingEngine` | Vector embedding generation |
| `ReasoningEngine` | LLM-based reasoning (Ollama/OpenAI) |
| `PerceptionPipeline` | Combined embedding + reasoning pipeline |

### Exception Hierarchy

```
RuntimeError
└── AiFastDbError
    ├── StoreNotFoundError
    ├── RecordNotFoundError
    ├── QueryError
    ├── WriteError
    ├── TransactionError
    ├── StorageError
    ├── CircuitBreakerError
    └── RateLimitError
```

## Development

```bash
# Install Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Clone and build
git clone https://github.com/aifastdb/aifastdb.git
cd aifastdb/packages/python

# Create venv and install in dev mode
python -m venv .venv
source .venv/bin/activate  # or .venv\Scripts\activate on Windows
pip install maturin numpy pandas pytest
maturin develop --release

# Run tests
python -m pytest tests/ -v
```

## License

See [LICENSE](LICENSE) for details. Free for personal, educational, and non-commercial use. Commercial use requires a separate license.
