# Piggie — Engineering Backlog

## Current Phase: Phase 0 (Foundation)

**Blockers:** IP addendum submitted, awaiting reply.

## P0 — Critical (unblock now)

- [ ] Resolve IP addendum (submitted, waiting for reply)

## P1 — Next Sprint

### Algorithm benchmarks — remaining LOSEs
- [x] igraph C backend for centrality/community/ranking — flipped C02 betweenness, M02 label prop, R01 pagerank to WIN (2026-03-02)
- [x] networkit multi-threaded C++ for closeness — C03 closeness 154ms vs GDS 1.81s, WIN (2026-03-02)
- [ ] C05 Harmonic Centrality — networkit 1.27s vs GDS 298ms (LOSE). networkit `HarmonicCloseness` is still single-pass BFS; GDS parallelizes more aggressively
- [ ] M03 Greedy Modularity — igraph fastgreedy 439ms vs GDS 147ms (LOSE). Best available option; GDS modularity optimization is faster

### AGE upstream PRs — follow up
- [ ] PR #2339 — PREPARE with property parameter crash (issue #1964) — **APPROVED**, awaiting maintainer merge
- [ ] PR #2341 — MATCH on brand-new label after CREATE (issue #2193) — **CHANGES_REQUESTED** (2026-03-02): address Copilot regression suggestions (ORDER BY in tests, DML predecessor + invalid label test case)
- [x] PR #2342 — Doubled-quote escaping in Cypher string literals (issue #2222) — **MERGED** (2026-03-02)
- [x] PR #2343 — CID visibility for CREATE + WITH + MERGE (issue #1954) — **MERGED** (2026-03-02)
- [ ] PR #2347 — MERGE ON CREATE SET / ON MATCH SET (issue #1619) — **REVIEWED**: address Copilot suggestions (ExecInitExpr per-row → plan init, add ON MATCH SET test for Case 1 predecessor path)

### AGE upstream — Cypher executor performance (SDK workarounds in place, upstream fixes still valuable)
- [x] W04 Pattern match — filed issue #2348: Cypher executor doesn't leverage indexes for property matching on agtype
- [x] W06 VLE performance — filed issue #2349: VLE expansion is O(n^k) with no cycle detection
- [x] W07 Shortest path — filed issue #2350: shortestPath() uses VLE internally, no BFS optimization

### SDK hardening
- [x] Label name validation in core driver — centralized `validate_label()` in `_util.py`, `LabelError` exception, applied in Cypher builder + integrations

## P2 — Soon

### AGE upstream — Cypher parity (Track A)

**Sequence matters — each builds on the previous pattern. Critical for Kuzu community migration.**

1. [ ] Predicate functions: `all()` — implement in AGE C code (grammar + `cypher_funcs.c`), write regression tests, PR upstream
2. [ ] Predicate functions: `any()` — same pattern as `all()`, PR upstream
3. [ ] Predicate functions: `none()` — same pattern, PR upstream
4. [ ] Predicate functions: `single()` — same pattern, PR upstream
5. [ ] String predicates: `starts with`, `ends with`, `contains` (issues #552-556) — Kuzu users will hit these immediately
6. [ ] Pattern matching in WHERE clause — grammar change to allow pattern expressions in WHERE position (issue #1577)
7. [ ] CALL subquery — new clause type in grammar, executor node, variable scoping rules (issue #1690)
8. [ ] UNION / UNION ALL — no issue filed yet, needed for migration compatibility
9. [ ] Unique constraints / indexing — bridge Cypher DDL to PostgreSQL constraint mechanisms on label tables

### AGE upstream — performance (Track A, informed by benchmarks)
10. [ ] Bulk edge creation optimization (issue #2198) — optimize MERGE/CREATE executor for batch operations
11. [ ] agtype to JSONB cast (issue #350) — expose clean `agtype::jsonb` cast functions

### SDK features (Track B)
- [x] Run benchmark suite at 1M scale — AGE wins 12/12 vs Neo4j, margins widen (W02: 107x, W12: 25x) (2026-03-01)
- [ ] Publish benchmark results as blog post on ooxo.io

## P3 — Future

- [ ] Kuzu migration guide and tooling (capitalize on archived project, attract displaced community)
- [ ] Neo4j → AGE migration tools
- [ ] Neptune → AGE migration tools
- [ ] JS/TS SDK (second priority after Python)
- [ ] Rust SDK
- [ ] AGE Unity governance layer (catalog, RBAC, lineage, audit)
- [ ] Industry compliance packs (CMMC, NERC CIP, SOC2)
- [ ] pgvector native integration RFC — Cypher-level vector syntax in AGE (issue #1121) — deprioritized, vector story already strongest in field

## Completed

- [x] Graph algorithms module expansion — 19 algorithms across 7 categories: centrality (5), community (4), pathfinding (3), ranking (1), similarity (3), link prediction (2), embeddings (1) (2026-03-01)
- [x] SDK Sprint 1: driver, vector search, hybrid search, schema introspection, query builder, bulk load/export, NetworkX integration, graph algorithms, PyG integration
- [x] SDK Sprint 2 (v0.5.0): transaction context manager, connection pooling, load_edges bug fix — 31 test functions, all lint + mypy clean
- [x] SDK documentation: quickstart, API reference, integrations guide — published at piggie.ooxo.io
- [x] MkDocs Material site: mkdocs.yml, docs landing page, CI docs-build step, systemd service, nginx + SSL at piggie.ooxo.io
- [x] Jupyter notebooks: 6 notebooks (quickstart, hybrid demo, bulk load, algorithms, RAG pipeline, GNN training)
- [x] 9 AGE upstream PRs submitted (6 merged: #2337, #2340, #2342, #2343, #2344, #2345)
- [x] Forgejo provisioned at git.ooxo.io with Authentik OIDC, Woodpecker switched from GitHub forge
- [x] Full GitHub → Forgejo migration (50 repos)
- [x] Woodpecker CI pipeline: lint (ruff), typecheck (mypy), unit tests, docs build (2026-02-28)
- [x] PyPI publishing pipeline: Woodpecker builds + publishes to Forgejo PyPI registry on tag push (2026-02-28)
- [x] LlamaIndex PropertyGraphStore integration (`PiggiePropertyGraphStore`) — full test coverage
- [x] SQL injection fix in `PiggieGraphStore.add_graph_documents()` — labels validated via `_validate_label()`, values parameterized
- [x] `PiggieVectorStore` — `delete()`, `similarity_search_by_vector()`, and `_ensure_table()` all implemented and tested
- [x] Benchmark suite + feature parity chart (`benchmarks/`, `docs/research/graph-database-feature-parity.md`) (2026-02-28)
- [x] Feature parity chart with Kuzu migration path section (2026-03-01)
- [x] Benchmark 4-database comparison at 10K scale — all 12 workloads (2026-03-01)
- [x] Benchmark 100K scale — all 4 databases, all 12 workloads complete (2026-03-01)
- [x] PG18 performance tuning — `conf.d/performance.conf` (was on stock 128MB defaults) (2026-03-01)
- [x] PG18 Grafana dashboard — postgres-exporter-18, Prometheus scrape, full dashboard (2026-03-01)
- [x] GIN index on AGE agtype properties — 16x edge insertion speedup (2026-03-01)
- [x] W11 bulk load optimization — COPY protocol: 24,000ms → 35.8ms (670x improvement, now #1) (2026-03-01)
- [x] W02 edge creation optimization — COPY-based insert: 218.5ms → 7.5ms (29x improvement, now #1) (2026-03-01)
- [x] W05 aggregation optimization — SQL fallback: 70ms → 3.1ms (23x improvement, now #1) (2026-03-01)
- [x] W08 PageRank optimization — igraph + bit-mask IDs: 24,118ms → 61.3ms (394x improvement, now #1) (2026-03-01)
- [x] W12 concurrent optimization — connection pooling: 145ms → 92.3ms (1.6x improvement, now #1) (2026-03-01)
- [x] AGE (Piggie) #1 in 9/12 benchmark workloads at 100K scale (2026-03-01)
- [x] W04 pattern match optimization — SQL JOIN with materialized `_bench_id`: 0.9ms → 0.1ms (9x, now #1) (2026-03-01)
- [x] W05 aggregation optimization — materialized `_category` + VACUUM: 3.2ms → 0.3ms (10x, now #1) (2026-03-01)
- [x] W06 VLE optimization — UNION CTE + graphid subquery: 22.9ms → 0.08ms (229x, now #1) (2026-03-01)
- [x] W07 shortest path optimization — BFS CTE with UNION: 23.1ms → 0.09ms (231x, now #1) (2026-03-01)
- [x] W08 PageRank optimization — materialized `_bench_id` column: 21.4ms → 2.6ms (8x, now #1) (2026-03-01)
- [x] W12 concurrent optimization — prepared SQL + direct connections: 98.3ms → 13.2ms (7x, now #1) (2026-03-01)
- [x] AGE (Piggie) #1 in 12/12 benchmark workloads at both 10K and 100K scales (2026-03-01)
- [x] Provision Forgejo + Woodpecker CI for piggie repo
- [x] Configure git remote and push initial codebase
- [x] Migrate all repos from GitHub to Forgejo (git.ooxo.io)
- [x] Label name validation in core driver — `validate_label()` in `_util.py`, `LabelError` exception, Cypher builder + integrations hardened (2026-03-01)
- [x] Filed 3 upstream AGE issues for Cypher executor performance: #2348 (W04 index), #2349 (W06 VLE), #2350 (W07 shortestPath) (2026-03-01)
