---
status: proposed
date: 2024-08-20
consulted: alex.kiefer@verses.ai, mo.kalby@verses.ai
informed: richard.petty@verses.ai, mo.kalby@verses.ai
decision-makers: jasmine.moore@verses.ai
---
# Factor Graph Storage

## Context and Problem Statement

Factor graphs are mostly-sparse matrices connected by edges, forming a machine learning graph. Common wild formats are BIF and dict. Alex Kiefer is working on a native human representation format, VFG. These graphs must be stored in a fashion that allows for efficient retrieval of nodes or subgraph components.

## Decision Drivers

* Highly performant
  * Speed of interactions with the graph
  * Writing output
* Stability of library
* Size of output
* Size of library, including total dependencies
* Zero-copy storage
* Usability
* Bindings available for Python
* `#[no_std]` support for wasm

## Considered Options

We're going to consider storing options directly, and then fill out the matrix. Binary formats will be serialized using
efficient rust formats, and direct formats will be stored as json. The archive formats are efficient rust JSON items.

* Store VFG (Verses Factor Graph) Directly in JSON
	* [Schema of VFG](https://github.com/VersesTech/genius-samples/blob/feat/vfg-schema/scripts/python/vfg_schema.json)
	* [Example of VFG](https://github.com/VersesTech/genius-samples/blob/feat/vfg-schema/data/processed/sprinkler_factor_graph_vfg.json)
* Store BIF directly in JSON
	* [Example of BIF](https://github.com/VersesTech/genius-samples/blob/feat/vfg-schema/data/processed/sprinkler.bif)
* Standard rust serialization formats that are as fast or faster at serialization and deserialization than storing the factor graph using JSON:
  * bilrost
  * prost
  * bitcode
  * abombination
  * savefile
  * alkahest
  * nanoserde
  * speedy
  * databuf
  * rkyv
  * bincode
  * postcard
  * capnp
  * borsh
  * bincode
  * partiy-scale-codec
  * serde_bar
  * cbor4ii
  * dlhn
  * msgpacker
  * rmp-serde
  * serde_cbor
  * simd-json
  * pot
  * bson
  * flatbuffers
  * flexbuffers
* Zero-copy rust serialization formats
  * abombination
  * alkahest
  * capnp
  * flatbuffers
  * rkyv
* Apache Fury

## Decision Outcome

I would position ourselves to use rkyv internally, with protocol buffers as our wire format.

Where cross-platform native bindings are necessary, it is advised to use Protocol Buffers.
Where zero-copy is necessary, it is advised to use rkyv.
Where cross-platform bindings are not necessary, it is advised to use rkyv for higher performance.

### Consequences

* We would be able to use the high speed and performace of rkyv for internal storage
* The same data types can be annotated with prost for wire format
* This would allow cross-platform usage with native bindings for other languages
* rkyv and prost both have `#[no_std]` support

### Confirmation

* There are no complaints from python users about choosing a bad format
* We are able to maintain the speed improvement we see here with graph format
* We do not see greater than a 10% overhead on serialization for wire format

## Pros and Cons of the Options
These are listed in approximate order of best-option to worst-option.

### rkyv
- Good, due to high performance of zero copy
- Good, due to robust interface and ease of use
- Good, due to extremely fast serialization and deserialization (3x as fast as protobuf, 10x as fast as json; see `arch_tests/adr1`)
- Bad, due to rust-only bindings

### Protocol Buffers (prost)
- Good, due to standardized format
- Good, due to cross-platform language binding generation
- Good, because it is *very* easy to use, similar in difficulty to json
- Bad, due to *slightly* worse deserialization performance (check prototype: `arch_tests/adr1`)

### Flatbuffers
- Good, due to high performance of zero-copy
- Good, due to cross-platform language binding generation
- Bad, due to user experience being a little clunky (check prototype: `arch_tests/adr1`)
- Bad, due to lack of deserialization support

### Flexbuffers
- Good, due to high performance of zero-copy
- Good, due to cross-platform language generation
- Bad, due to *slow* deserialization performance (check prototype: `arch_tests/adr1`)
  - Performance is within 5% of JSON

### VFG Format (direct)
Directly storing the json format.
- Good, because it is human-readable
- Good, because there are existing graphs in this format
- Bad, due to performance of reading json
- Bad, due to the representation of numbers

### Non-zero-copy Rust Serialization Formats (bilrost, bitcode, savefile, ...)
- Good, because they're faster than json
- Good, because they represent numbers accurately
- Bad, because deserialization takes *milliseconds* per structured record, apples to apples. Deserialization is thousands of times slower than serialization across all formats
- Bad, because increased memory usage of copy will interfere with graph storage space

### Zero-copy rust serialization frameworks without a python binding (alkahest, abombination, rkyv)
- Good, because they're faster than json
- Bad, because they cannot be used in python

### Formats slower than JSON (not enumerated)
These are formats where we should just use json or any other better alternative. There exist formats that serialize and
deserialize faster than json, while maintaining number accuracy.
- Good, because they represent numbers more accurately
- Bad, because they are slower than JSON
- Bad, because they are harder to use than JSON

### BIF Formats (binary or direct)
This applies to *all* BIF formats.
- Good, because this is an industry-standard graph format.
- Bad, because it does not contain all of the information needed to store any factor graph that is not a bayes net
- Bad, because it needs to be translated for consumer code

### CapnProto
- Good, due to high performance of zero-copy
- Good, due to cross-platform language binding generation
- Bad, because of lack of tooling support
- Bad, because of difficulty of constructing and manipulating objects

Note: Disqualified due to issues with usability

### Apache Fury
- Bad, because no standard crate is available (we would have to vendor)
- Bad, because it is [described by the authors as not yet stable](https://github.com/apache/fury/issues/1649) (as of 2024-05-29)
- Good, because of cross-platform support

Note: Comparison benchmarks were not evaluated for Apache Fury due to the authors themselves saying it is not yet stable.

## More Information

If the scope increases to use WASM bindings across all projects, we might be able to easier deploy rkyv to all consumers.

If we are able to ship a rust library to all clients, we might also be able to use rkyv.

The exact struct definition might need to be adjusted as Alex continues to work on his variant.

We might reconsider build-vs-buy if we run into limitations in protobuf.

If Apache Fury stabilizes its rust crate, it should be reconsidered.

## Benchmark results from prototype
```
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.10s
     Running `target/debug/adr1_archive_benchmarks`
json (baseline) (5.0s) ...            60_392.838 ns/iter (0.996 R²)
flexbuffers (5.0s) ...                57_242.352 ns/iter (0.959 R²)
protobuf (prost) (5.0s) ...           16_756.363 ns/iter (1.000 R²)
flatbuffers (5.0s) ...                10_926.516 ns/iter (1.000 R²)
rkyv (5.0s) ...                        8_166.203 ns/iter (0.996 R²)
```
