# Takeoff Environment Settings

The `TakeoffEnvSetting` class is a data model used in the Takeoff SDK for setting environment variables.

## TakeoffEnvSetting

| Variable Name (SDK)           | Environment Variable Name             | Default Value                              | Explanation                                                                                                                                  | Internal/External |
| ----------------------------- | ------------------------------------- | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- |
| model_name                    | TAKEOFF_MODEL_NAME                    | required                                   | The model name to use                                                                                                                        | External          |
| device                        | TAKEOFF_DEVICE                        | required                                   | The device model runs on                                                                                                                     | External          |
| consumer_group                | TAKEOFF_CONSUMER_GROUP                | primary, required in manifest mode         | Used to set the name of a consumer group a model should belong to when Takeoff is first spun up                                              | External          |
| max_batch_size                | TAKEOFF_MAX_BATCH_SIZE                | 8                                          | The maximum batch size the model can use                                                                                                     | External          |
| batch_duration_millis         | TAKEOFF_BATCH_DURATION_MILLIS         | 100                                        | The wait interval for dynamic batching.                                                                                                      | External          |
| access_token                  | TAKEOFF_ACCESS_TOKEN                  | /                                          | Access token for private HF repos                                                                                                            | External          |
| cuda_visible_devices          | TAKEOFF_CUDA_VISIBLE_DEVICES          | / (all gpus)                               | Which GPUs are visible to that reader. List must be of a length which is a power of 2.                                                       | External          |
| disable_continuous_generation | TAKEOFF_DISABLE_CONTINUOUS_GENERATION | FALSE                                      | Whether we should disable continuous generation                                                                                              | Internal          |
| log_level                     | TAKEOFF_LOG_LEVEL                     | INFO                                       | What log level reader messages are exported at                                                                                               | Internal          |
| backend                       | TAKEOFF_BACKEND                       | /                                          |                                                                                                                                              | Internal          |
| redis_host                    | TAKEOFF_REDIS_HOST                    | localhost                                  | Where the reader should look for redis.                                                                                                      | Internal          |
| redis_port                    | TAKEOFF_REDIS_PORT                    | 6379                                       | What port redis is serving on (on TAKEOFF_REDIS_HOST)                                                                                        | internal          |
| heartbeat_wait_interval       | TAKEOFF_HEARTBEAT_WAIT_INTERVAL       | 10                                         | How long the model should wait for heartbeats from the reader                                                                                | Internal          |
| tensor_parallel               | TAKEOFF_TENSOR_PARALLEL               | 1                                          | How many GPUs to split the model across. Will greedily select the lowest n gpus. Control which gpus to use with TAKEOFF_CUDA_VISIBLE_DEVICES | External          |
| disable_cuda_graph            | TAKEOFF_DISABLE_CUDA_GRAPH            | 0                                          | Turn off cuda graphs                                                                                                                         | Internal          |
| quant_type                    | TAKEOFF_QUANT_TYPE                    | auto                                       | Set to awq to force AWQ. Otherwise it is automatically detected.                                                                             | External          |
| max_seq_len                   | TAKEOFF_MAX_SEQUENCE_LENGTH           | 128                                        | The size of the sequence dim buffer in the KV Cache                                                                                          | External          |
| disable_kernel_injection      | TAKEOFF_DISABLE_KERNEL_INJECTION      | 0                                          | Turn off kernel injection                                                                                                                    | Internal          |
| disable_static                | TAKEOFF_DISABLE_STATIC                | 0                                          | Turn off static optimizations                                                                                                                | Internal          |
| nvlink_unavailable            | TAKEOFF_NVLINK_UNAVAILABLE            | 0                                          | Set to 1 if you are on a system without NVLINK (e.g. L4s, 4090s) to use multi-gpu                                                            | External          |
| license_key                   | LICENSE_KEY                           | / (required but it won’t kill the takeoff) | Takeoff licence key for key validation                                                                                                       | External          |
| rust_log_level                | RUST_LOG                              | INFO                                       | What level of log messages to expose from rust                                                                                               | internal          |
| run_name                      | TAKEOFF_RUN_NAME                      | takeoff_default                            | Used to set the file name of log messages                                                                                                    | internal          |
| export_traces                 | TAKEOFF_EXPORT_TRACES                 | FALSE                                      | Whether to export tracing information (in rust + python)                                                                                     | internal          |
| traces_host                   | TAKEOFF_TRACES_HOST                   | http://localhost:4317                      | Where to send tracing information in rust and python.                                                                                        | internal          |
| is_echo                       | TAKEOFF_ECHO                          | FALSE                                      | Adds a rust reader to the stream that echoes all the messages. For debugging rust.                                                           | internal          |
