S3 Backend
==========

Example Configurations
----------------------

Basic S3 Configuration
~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: yaml

   chunk_size: 256
   local_cpu: False
   save_unfull_chunk: False
   remote_url: "s3://your-bucket-name"
   remote_serde: "naive"
   blocking_timeout_secs: 10
   extra_config:
     s3_region: "us-east-1"
     s3_num_io_threads: 64
     save_chunk_meta: False

S3 Express One Zone
~~~~~~~~~~~~~~~~~~~

.. code-block:: yaml

   chunk_size: 256
   local_cpu: False
   save_unfull_chunk: False
   remote_url: "s3://{BUCKET_NAME}.s3express-{AZ_ID}.{REGION}.amazonaws.com"
   remote_serde: "naive"
   blocking_timeout_secs: 10
   extra_config:
     save_chunk_meta: False
     s3_num_io_threads: 64
     s3_prefer_http2: True
     s3_region: "{REGION}"
     s3_enable_s3express: True

CoreWeave (S3-compatible)
~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: yaml

   chunk_size: 256
   local_cpu: False
   max_local_cpu_size: 50
   save_unfull_chunk: False
   enable_async_loading: True
   remote_url: "s3://test-127.cwlota.com"
   remote_serde: "naive"
   blocking_timeout_secs: 10
   extra_config:
     s3_num_io_threads: 320
     s3_prefer_http2: False
     s3_region: "US-WEST-04A"
     s3_enable_s3express: False
     save_chunk_meta: False
     disable_tls: True
     aws_access_key_id: "your-access-key-id"
     aws_secret_access_key: "your-secret-access-key"

**Note**: `cwlota.com` is CoreWeave's S3-compatible Cloud Storage that caches for GPU locality. You can set `disable_tls: True` for non-AWS services.

Check out the blog post between LMCache, Cohere, and CoreWeave: https://blog.lmcache.ai/en/2025/10/29/breaking-the-memory-barrier-how-lmcache-and-coreweave-power-efficient-llm-inference-for-cohere/


Configuration Parameters
------------------------

* **remote_url**: S3 bucket URL (`s3://bucket-name`)
* **save_unfull_chunk**: Save partial chunks (default: False, **must be False for S3**)
* **enable_async_loading**: Async loading (default: False)
* **blocking_timeout_secs**: Timeout seconds (default: 10)

S3-Specific (in extra_config)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* **s3_region**: AWS region for S3 client (required)
* **s3_num_io_threads**: Number of IO threads for the AWS CRT client to spawn. Benefits taper out after exceeding the number of CPU cores. This is also a way to restrict the number of outgoing requests in case your S3-compatible object store has a rate-limiting gateway.
* **s3_prefer_http2**: Enable HTTP/2 with ALPN negotiation (["h2", "http/1.1"])
* **s3_enable_s3express**: Enable S3 Express One Zone support in AWS CRT client
* **save_chunk_meta**: Whether to save chunk metadata in the object store along with your data (False required for S3)
* **aws_access_key_id**: AWS access key ID (or log in with `aws configure` in your environment)
* **aws_secret_access_key**: AWS secret access key (or log in with `aws configure` in your environment)

**Tips:**::

- Use same region for compute and S3

- Consider S3 Express One Zone for less redundancy but better performance