# Architecture club 4/25/23

Inaugural architecture club meeting

## Topic: Config

### How do we set config

Two scenarios for config:
- Library adds base config file
- User supplies overrides
Examples
  - Jupyter notebooks
  - Runtime supplies config yml

We want to handle these slightly differently because user-supplied overrides should take absolute precedence

This:
```python
import caikit

caikit.configure(overrides={"service_port": 8000})
```
should set the port to 8000 no matter if the `SERVICE_PORT` environment variable is set, or some other 
`caikit.configure` call happens later that pulls in another config file with a different value.

Open question: Should the same apply to values in some `my_mounted_configmap.yml` that is pulled in with
```shell
export CONFIG_FILES=my_mounted_configmap.yml
```
Or should environment variables continue to take precedence over that?

Some options:
1. Keep the same single `caikit.configure` function with a kwargs-only `overrides` param and good docstrings explaining that these take precedence over anything else
2. Split config out into a `caikit.register_base_config_file()` and `caikit.configure()` (or `caikit.override_config()`)

### What should the config keys look like?

Question: Do we want to break the configuration API?

Consensus: This is the best opportunity we'll have to do it before we pick up new users. Use this as a forcing function 
for existing users to come pick up upgrades

Logging:
- Super fun one: We only ever want to apply a logging config from a `main` context which is generally just "when we run a runtime"
- But we don't want users to need to use non-standard logging config keys. People like `export LOG_LEVEL=info`
- Probably keep it the same for now, punting further discussion

For everything else, should we keep keys flat or nest them (in some way, did not have time to talk through how)?

Consensus: We should nest things

Model train and model mesh:
- These have a whole lot of configs
- But these two are concrete implementations of a "training architecture plugin" and "serving architecture plugin"

We want something like this to let us spin up a factory for these plugins:
```yaml
serving_plugin:
  model_mesh:
    default_model_size: ...
training_plugin:
  model_train:
    ...
```
But we could consider having some common configs for each type of plugin that concrete implementations can pull
(Discussion cut for time)
