
# Output specification

The table below contains information about all files generated by AEGIS. Here we provide an explanation of various table columns.
- `path` gives the location of the stored files. Paths are relative, all files are located in the same folder as the configuration file.
- `filetype` specifies the types of output files. AEGIS uses simple text formats (e.g. .csv, .txt, .json, .yml) and common python filetypes (.pickle and .feather).
- `category` specifies the kind of information stored – information about `genotype`, `phenotype`, `demography`, `population genetics` or `log` (logging information).
- `description` contains short descriptions of the files.
- `trait granularity` specifies the level of trait aggregation: `individual` means that data has not been aggregated across individuals, `popultion` means that the data has been aggregated across individuals, either by computing the count, mean or median of the target trait. Note that when individual-level data is recorded, all individual are included in the recording; not just a sample. In the current version, AEGIS does not keep track of individual IDs, thus individuals are not identifiable across recordings.
- `time granularity` specifies the level of time aggregation: `snapshot` means that no data has been aggregated over multiple simulation steps, while `interval` means that an aggregated metric has been computed across multiple simulation steps. Aggregated information has a smaller storage footprint, allowing it to be stored more frequently. 
- `frequency` specifies the parameter that sets the frequency of recording. Recording of all files is customizable, apart from files that are recorded once (marked by `once`). Higher frequency allows more granular inspection of the simulation but requires more storage space.
- `structure` gives more information about the internal format of the files.

|    | path                                 | filetype   | category            | description                                                                                                                                                                                                                      | trait granularity   | time granularity   | frequency parameter   | structure                                                                                                                                                                                       |
|---:|:-------------------------------------|:-----------|:--------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------|:-------------------|:----------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|  0 | /snapshots/genotypes/{step}.feather  | feather    | genotype            | A snapshot of complete binary genomes of all individuals at a certain simulation step.                                                                                                                                           | individual          | snapshot           | SNAPSHOT_RATE         | A bool matrix                                                                                                                                                                                   |
|  1 | /snapshots/phenotypes/{step}.feather | feather    | phenotype           | A snapshot of complete intrinsic phenotypes of all individuals at a certain simulation step.                                                                                                                                     | individual          | snapshot           | SNAPSHOT_RATE         | A float matrix                                                                                                                                                                                  |
|  2 | /snapshots/demography/{step}.feather | feather    | demography          | A recording of life history metrics (age, number of births given, step at which born, current size, sex) of all individuals until a certain simulation step.                                                                     | individual          | snapshot           | SNAPSHOT_RATE         | A matrix of ints and floats                                                                                                                                                                     |
|  3 | /gui/spectra/age_at_{cause}.csv      | csv        | demography          | Total number of deaths by age and cause of death, within a simulation interval.                                                                                                                                                  | population count    | interval           | INTERVAL_RATE         | An int matrix.                                                                                                                                                                                  |
|  4 | /phenomap.csv                        | csv        | genotype            | A static list of phenotypic effects of each genomic site.                                                                                                                                                                        | N/A                 | N/A                | once                  | A table with four columns: effector site index, affected trait, affected age, effect magnitude. Each row represents an effect of a single site on a specific trait expressed at a specific age. |
|  5 | /pickles/{step}                      | pickle     | log                 | A file that records the Population class instance which can be used to seed a future simulation.                                                                                                                                 | individual          | snapshot           | PICKLE_RATE           | Binary python file.                                                                                                                                                                             |
|  6 | /popgen/simple.csv                   | csv        | population genetics | Simple population metrics including population size, effective population size, mu, segregating sites, segregating sites using a genomic sample, theta, theta_w, theta_pi, tajimas_d, theta_h, and fayandwu_h.                   |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
|  7 | /popgen/allele_frequencies.csv       | csv        | population genetics | 1-allele population-frequencies of every genomic site.                                                                                                                                                                           |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
|  8 | /popgen/genotype_frequencies.csv     | csv        | population genetics | Genotype frequencies at site resolution (e.g. for a diploid genome, number of 00, 01 and 11 for each site).                                                                                                                      |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
|  9 | /popgen/sfs.csv                      | csv        | population genetics | A site frequency spectrum.                                                                                                                                                                                                       |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
| 10 | /popgen/mean_h_per_bit_expected.csv  | csv        | population genetics | Heterozygosity per bit.                                                                                                                                                                                                          |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
| 11 | /popgen/mean_h_per_bit.csv           | csv        | population genetics | Expected mean heterozygosity per bit under Hardy-Weinberg-Equilibrium.                                                                                                                                                           |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
| 12 | /popgen/mean_h_per_locus.csv         | csv        | population genetics | Mean bit heterozygosity per locus.                                                                                                                                                                                               |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
| 13 | /popgen/reference_genome.csv         | csv        | population genetics | Reference genome based on which allele is most common at each position.                                                                                                                                                          |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
| 14 | /popgen/reference_genome_gsample.csv | csv        | population genetics | Reference genome based on which allele is most common at each position in a sample of genomes.                                                                                                                                   |                     |                    | POPGENSTATS_RATE      | A float matrix.                                                                                                                                                                                 |
| 15 | /progress.log                        | txt        | log                 | A table documenting the estimated time of simulation completion (ETA), time to run one million steps (t1M), time since simulation start (runtime), number of simulated steps per minute (stg/min) and population size (popsize). | N/A                 | N/A                | LOGGING_RATE          | A str table with custom separator (` | `).                                                                                                                                                      |
| 16 | /input_summary.json                  | json       | genotype            | A json dictionary documenting a number of simulation traits at the start of the simulation; e.g. random seed, time at start.                                                                                                     |                     |                    | once                  | A json dictionary.                                                                                                                                                                              |
| 17 | /output_summary.json                 | json       | log                 | A json dictionary documenting a number of simulation traits at the end of the simulation; e.g. extinction status, random seed, time at start, total runtime, median memory use, total storage use.                               | N/A                 | N/A                | once                  | A json dictionary.                                                                                                                                                                              |
| 18 | /te/{te_number}.csv                  | csv        | demography          | Data for survival analysis; time until event (age at death or current viable age) and the event type (1 if death, 0 if still alive).                                                                                             | population          | interval           | TE_RATE               | An int matrix.                                                                                                                                                                                  |
| 19 | /ticker.txt                          | txt        | log                 | A live file useful for determining whether the simulation is still running. It gets updated every TICKER_RATE seconds; if it is not updated, the simulation is not running.                                                      | N/A                 | N/A                | TICKER_RATE           | A txt file with datetime stamp (%Y-%m-%d %H:%M:%S) in one line)                                                                                                                                 |
| 20 | /gui/genotypes.csv                   | csv        | genotype            | A table of allele frequencies (frequency of 1's in the population for each site) across simulation intervals. Columns are sites, rows are simulation intervals (spanning INTERVAL_RATE steps).                                   | population mean     | snapshot           | INTERVAL_RATE         | A float matrix.                                                                                                                                                                                 |
| 21 | /gui/genotypes.csv                   | csv        | phenotype           | A table of median intrinsic phenotypes (median phenotype rate for each trait at each age) across simulation intervals. Columns are traits, rows are simulation intervals (spanning INTERVAL_RATE steps).                         | population median   | snapshot           | INTERVAL_RATE         | A float matrix                                                                                                                                                                                  |