.. _standalone_starter:

Standalone Starter
==================

The LMCache Standalone Starter allows you to run LMCacheEngine as a standalone service without vLLM or GPU dependencies. This is particularly useful for:

- Testing and development environments
- CPU-only or P2P backend deployments

Quick Start
-----------

Basic Usage
~~~~~~~~~~~

.. code-block:: bash

   # Start with default configuration
   python -m lmcache.v1.standalone

   # Start with custom configuration file
   python -m lmcache.v1.standalone --config examples/cache_with_configs/example.yaml

   # Start with environment variables
   export LMCACHE_CONFIG_FILE=examples/cache_with_configs/example.yaml
   python -m lmcache.v1.standalone

CPU-Only Mode
~~~~~~~~~~~~~

.. code-block:: bash

   python -m lmcache.v1.standalone \
       --config examples/cache_with_configs/example.yaml \
       --model_name my_model \
       --worker_id 0 \
       --world_size 1

Remote P2P Mode
~~~~~~~~~~~~~
TO be added

Configuration Section
---------------------

The standalone starter supports multiple configuration sources with the following priority order:

1. **Command-line arguments** (highest priority)
2. **Configuration file** (specified by ``--config`` or ``LMCACHE_CONFIG_FILE``)
3. **Environment variables** (e.g., ``LMCACHE_CHUNK_SIZE=512``)
4. **Default values** (lowest priority)

Parameter Details
~~~~~~~~~~~~~~~~~

**KV Cache Shape Specification**

The ``--kvcache_shape_spec`` parameter supports multi-layer group configurations:

- Format: ``(shape_string):dtype:layer_count;[...]``
- shape_string: comma-separated shape (e.g., '2,2,256,4,16')
- Examples:
  - Single group: ``(2,2,256,4,16):float16:2``
  - Multiple groups: ``(2,2,256,4,16):float16:2;(3,2,256,4,4):float32:3``

**Device Support**

- ``--device=cpu``: CPU-only mode (default)
- ``--device=cuda``: CUDA GPU acceleration
- ``--device=xpu``: XPU GPU acceleration

**MLA (Multi-Level Attention)**

- ``--use_mla``: Enable MLA for improved attention performance
- Requires compatible model and configuration

**Cache Formats**

- ``--fmt=vllm``: vLLM-compatible format (default)
- Supports other formats for different inference engines

Command-Line Parameters
-----------------------

Basic Parameters
~~~~~~~~~~~~~~~~

.. code-block:: bash

   --config CONFIG_FILE             # Path to configuration file
   --model_name MODEL_NAME          # Model name for cache identification
   --worker_id WORKER_ID            # Worker ID (default: 0)
   --world_size WORLD_SIZE          # Total workers (default: 1)
   --kv_dtype {float16,float32,bfloat16,uint8}  # KV cache data type
   --kv_shape KV_SHAPE              # KV cache shape (default: "2,2,256,4,16")
   --kvcache_shape_spec SPEC       # Multi-group KV shape specification
   --device {cpu,cuda,xpu}          # Device to run on (default: cpu)
   --fmt FORMAT                     # Cache format (default: vllm)
   --use_mla                        # Enable MLA (Multi-Level Attention)


Usage Examples
--------------

Custom Configuration
~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   python -m lmcache.v1.standalone \
       --config examples/cache_with_configs/example.yaml \
       --chunk_size=512 \
       --max_local_cpu_size=4.0 \
       --model_name=custom_model

Multi-Layer Group Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   python -m lmcache.v1.standalone \
       --config examples/cache_with_configs/example.yaml \
       --kvcache_shape_spec="(2,2,256,4,16):float16:2" \
       --kv_shape="2,2,256,4,16" \
       --model_name=multi_group_model \
       --device=cpu

GPU device
~~~~~~~~~~~~~~~~

.. code-block:: bash

   python -m lmcache.v1.standalone \
       --config examples/cache_with_configs/example.yaml \
       --kvcache_shape_spec="(2,2,256,4,16):float16:2" \
       --kv_shape="2,2,256,4,16" \
       --kv_dtype=float16 \
       --device=cuda \
       --use_mla \
       --model_name=gpu_model

MLA Configuration
~~~~~~~~~~~~~~~~~

.. code-block:: bash

   python -m lmcache.v1.standalone \
       --config examples/cache_with_configs/example.yaml \
       --kv_shape="16,2,512,16,64" \
       --kv_dtype=bfloat16 \
       --use_mla \
       --fmt=vllm \
       --model_name=mla_model

Internal API Server
-------------------

The standalone starter includes an internal API server for monitoring and management:

.. code-block:: bash

   python -m lmcache.v1.standalone \
       --config examples/cache_with_configs/example.yaml \
       --chunk_size=512 \
       --max_local_cpu_size=4.0 \
       --model_name=custom_model \
       --internal_api_server_enabled=True


Troubleshooting
----------------

Common Issues
~~~~~~~~~~~~~

**Issue**: "No config file specified"
**Solution**: Set ``LMCACHE_CONFIG_FILE`` or use ``--config`` parameter

**Issue**: "Failed to connect to controller"
**Solution**: Start controller first: ``python -m lmcache.v1.api_server``

**Issue**: "Invalid KV shape specification"
**Solution**: Check format: ``(shape):dtype:layer_count``, e.g., ``(2,2,256,4,16):float16:2``

**Issue**: "Device not available"
**Solution**: Verify device support: use ``--device=cpu`` if GPU not available

**Issue**: "MLA configuration error"
**Solution**: Ensure compatible model and check ``--use_mla`` parameter

Debug Mode
~~~~~~~~~~

Enable debug logging for troubleshooting:

.. code-block:: bash

   export LMCACHE_LOG_LEVEL=DEBUG
   python -m lmcache.v1.standalone

Advanced Debugging
~~~~~~~~~~~~~~~~~~

For detailed layer group information:

.. code-block:: bash

   export LMCACHE_LOG_LEVEL=DEBUG
   python -m lmcache.v1.standalone \
       --kvcache_shape_spec="(2,2,256,4,16):float16:2;(3,2,256,4,4):float32:3" \
       --device=cpu

Performance Tuning
------------------

Memory Configuration
~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   # For systems with large memory
   --max_local_cpu_size=8.0

   # For memory-constrained systems
   --max_local_cpu_size=1.0

Layer Group Optimization
~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: bash

   # Optimize for mixed precision models
   --kvcache_shape_spec="(2,2,256,4,16):float16:2;(3,2,256,4,4):bfloat16:3"

   # Optimize for different layer configurations
   --kvcache_shape_spec="(2,2,512,8,32):float16:4;(4,2,256,16,16):float32:2"

GPU Acceleration
~~~~~~~~~~~~~~~~~

.. code-block:: bash

   # GPU-optimized configuration
   --device=cuda --kv_dtype=float16 --use_mla

   # Large model on GPU
   --device=cuda --kv_shape="64,2,512,64,128" --max_local_cpu_size=16.0

MLA Performance
~~~~~~~~~~~~~~~

.. code-block:: bash

   # Enable MLA for attention optimization
   --use_mla --kv_dtype=bfloat16 --device=cuda

   # MLA with custom shape
   --use_mla --kv_shape="32,2,1024,32,64" --fmt=vllm


Best Practices
--------------

1. **Use configuration files** for production deployments
2. **Set appropriate cache sizes** based on available memory
3. **Enable internal API** for monitoring and management
4. **Monitor logs** for performance and error tracking
5. **Use multi-layer group configurations** for complex model architectures
6. **Enable MLA** for improved attention performance on supported hardware
7. **Choose appropriate device** based on available resources (CPU/GPU/XPU)
8. **Validate KV shape specifications** before deployment
9. **Test with debug logging** when configuring new layer groups
10. **Optimize chunk sizes** for specific hardware configurations

Multi-Layer Group Best Practices
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- Use consistent chunk sizes across layer groups for optimal performance
- Group layers with similar precision requirements together
- Validate shape specifications in development environment first
- Monitor memory usage when using multiple layer groups

MLA Configuration Tips
~~~~~~~~~~~~~~~~~~~~~~~

- Enable MLA only on supported hardware configurations
- Use bfloat16 or float16 precision for best MLA performance
- Test MLA performance impact before production deployment
- Monitor attention performance metrics with MLA enabled

Related Documentation
---------------------

- :doc:`../quickstart`
- :doc:`../../api_reference/configurations`
- :doc:`../../kv_cache/storage_backends/index`
- :doc:`../../kv_cache_management/index`