# FlatBench

This is based on the Google FlatBuffer benchmark schema, but the
benchmark itself is independent and not directly comparable, although
roughly the same operations are being executed.

The `benchflatc` folder contains C++ headers and code generated by Googles
`flatc` compiler while `benchflatcc` contains material from this project.

The `benchraw` folder contains structs similar to those used in Googles
benchmark, but again the benchmark isn't directly comparable.

It should be noted that allocation strategies differ. The C++ builder
is constructed on each build iteration whereas theh C version resets
instead. The benchmark is designed such that the C++ version could do
the same if the builder supports it.

## Execution

Build and run each benchmark individually:

    benchmark/benchflatc/run.sh
    benchmark/benchflatcc/run.sh
    benchmark/benchraw/run.sh
    benchmark/benchflatccjson/run.sh

Note that each benchmark runs in both debug and optimized versions!


# Environment

The the benchmark are designed for a `*nix environmen.

- A C compiler named `cc` supporting -std=c11 is required for flatcc.
- A C++ compiler named `c++` supporting -std=c++11 is requried for
  flatc.
- A C compiler named `cc` supporting <stdint.h> is required for raw benchmark.
- Test is driven by a shell script.

The time measurements in `elapsed.h` ought to work with Windows, but it
has not been tested. The tests could be compiled for Windows with a
separate set of `.bat` files that adapt to the relevant compiler settings
(not provided).


## Output

The source and generated files and compiled binaries are placed in a
dedicated folder under:

    build/tmp/test/benchmark/

Only flatcc includes files from the containing project - other
benchmarks copy any relevant files into place.

The optimized flatc C++ benchmark is 24K vs flatcc for C using 35K.


## JSON numeric conversion

The Json printer benchmark is significantly impacted by floating point
conversion performance. By using the grisu3 algorithm instead of the
printing speed more than doubles compared to sprintf "%.17g" method with
clang glibc.  The parsing, on the other hand, parsing slows down
slightly because floats are always printed as double which increases the
json text from 700 to 722 bytes. For comparision, RapidJSON also only
supports double precision because the JSON spec does not specifically
mention preicision.
