## Eval Outputs Typescript Types

### What This Is

This folder contains autogenerated Typescript types for the `eval_output_*.py` JSON schemas in each eval subfolder.

### What It's For

Neuronpedia runs on Typescript. For Neuronpedia to properly parse eval outputs, we generate Typescript types from the JSON schemas, and set it in Neuronpedia.

> ⚠️ Warning: Do not manually modify these files. Instead, update the respective `eval_output_*.py` file and follow "How To Update" below.

### How To Update

You will likely never need to do this yourself (a Neuronpedia maintainer will do it), but if you ever do need to update these types, here's how:

1. Ensure you have [NPM installed](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm).
2. Install the tool globally:
   ```
   npm install json-schema-to-typescript --global
   ```
3. Make your changes to the `eval_output_*.py` file(s)
4. Re-generate the JSON schema files

   ```
   # this brings you to the ./sae_bench/evals subdirectory of this repository
   cd sae_bench/evals

   python generate_json_schemas.py
   ```

5. Re-generate the Typescript types

   ```
   # this brings you to the ./sae_bench subdirectory of this repository (the parent directory of evals)
   cd sae_bench

   # run the tool
   json2ts -i 'evals/*/eval_output_*.json' -o 'evals_outputs_typescript_types'
   ```

6. The new types are under `evals_outputs_typescript_types`. Inform someone at Neuronpedia to update the server side. 🥳
