# Migrating from Akumuli to VictoriaMetrics

This document describes how to migrate your data from Akumuli to VictoriaMetrics.

## Prerequisites

1. Both `asyncakumuli` and `asyncvictoria` packages installed:
   ```bash
   pip install asyncakumuli
   pip install asyncvictoria
   ```

2. Access to your running Akumuli instance

3. A running VictoriaMetrics instance with OpenTSDB protocol enabled

## Starting VictoriaMetrics

VictoriaMetrics needs to be started with OpenTSDB protocol support:

```bash
victoria-metrics \
  -storageDataPath=/path/to/data \
  -httpListenAddr=:8428 \
  -opentsdbListenAddr=:4242
```

Where:
- `-httpListenAddr` specifies the port for HTTP queries (default: 8428)
- `-opentsdbListenAddr` specifies the port for OpenTSDB telnet protocol data ingestion (default: 4242)

## Migration Methods

### Method 1: Using the Migration Script

The `migrate_from_akumuli.py` script automates the migration process.

#### Option A: Auto-discover series from Akumuli database

The script can automatically discover all series by reading Akumuli's SQLite database:

```bash
python migrate_from_akumuli.py \
  --akumuli-http http://localhost:8181 \
  --akumuli-db /path/to/akumuli/db.akumuli \
  --victoria-tcp localhost:4242 \
  --victoria-http http://localhost:8428 \
  --start-time 1500000000 \
  --end-time 1600000000
```

#### Option B: Specify series manually

Alternatively, you can specify which series to migrate:

```bash
python migrate_from_akumuli.py \
  --akumuli-http http://localhost:8181 \
  --victoria-tcp localhost:4242 \
  --victoria-http http://localhost:8428 \
  --series "cpu_usage{host=server1}" \
  --series "memory_usage{host=server1}" \
  --start-time 1500000000 \
  --end-time 1600000000
```

#### Options:
- `--akumuli-http`: Akumuli HTTP endpoint (default: http://localhost:8181)
- `--akumuli-db`: Path to Akumuli's SQLite database (db.akumuli) for auto-discovering series
- `--victoria-tcp`: VictoriaMetrics OpenTSDB TCP endpoint (default: localhost:4242)
- `--victoria-http`: VictoriaMetrics HTTP endpoint (default: http://localhost:8428)
- `--series`: Series to migrate (can be specified multiple times)
  - Format: `"series_name"` or `"series_name{tag1=val1,tag2=val2}"`
- `--start-time`: Optional start timestamp (Unix timestamp in seconds)
- `--end-time`: Optional end timestamp (Unix timestamp in seconds)

**Note:** Either `--series` or `--akumuli-db` must be specified.

### Method 2: Programmatic Migration

You can also write your own migration script:

```python
import anyio
import httpx
from asyncakumuli import get_data as akumuli_get_data
from asyncvictoria import connect as victoria_connect, Entry as VEntry, DS

async def migrate():
    async with httpx.AsyncClient(timeout=600) as akumuli_session:
        async with victoria_connect(host="localhost", port=4242) as victoria:
            # Read from Akumuli
            data = akumuli_get_data(
                akumuli_session,
                series="cpu_usage",
                tags={"host": "server1"},
                t_start=1500000000,
                t_end=1600000000,
                url="http://localhost:8181/api/query"
            )
            
            # Write to VictoriaMetrics
            count = 0
            async for entry in data:
                ve = VEntry(
                    series=entry.series,
                    time=entry.time,
                    value=entry.value,
                    tags=entry.tags,
                    mode=DS.gauge
                )
                await victoria.put(ve)
                count += 1
                
                if count % 1000 == 0:
                    await victoria.flush()
                    print(f"Migrated {count} points...")
            
            await victoria.flush()
            print(f"Migration complete: {count} points")

anyio.run(migrate)
```

## Key Differences Between Akumuli and VictoriaMetrics

### Data Format

**Akumuli** uses a custom RESP (Redis Serialization Protocol) format:
```
series_name tag1=value1 tag2=value2
timestamp
value
```

**VictoriaMetrics** uses OpenTSDB telnet protocol format:
```
put series_name timestamp value tag1=value1 tag2=value2
```

### Querying

**Akumuli** uses a JSON-based query API:
```json
{
  "select": "cpu_usage",
  "where": {"host": "server1"},
  "range": {"from": "20210101T000000", "to": "20210201T000000"}
}
```

**VictoriaMetrics** uses PromQL-compatible queries:
```
cpu_usage{host="server1"}
```

### API Endpoints

**Akumuli:**
- Write: TCP port (default 8282)
- Read: HTTP `/api/query` (default port 8181)

**VictoriaMetrics:**
- Write: OpenTSDB telnet protocol (default port 4242)
- Read: HTTP `/api/v1/query` and `/api/v1/query_range` (default port 8428)

## Testing Migration

A test suite is provided to verify migration correctness:

```bash
pytest tests/test_migration.py
```

This test:
1. Starts an Akumuli instance
2. Writes test data
3. Starts a VictoriaMetrics instance
4. Migrates the data
5. Verifies all data was migrated correctly

## Post-Migration

After successful migration:

1. Update your application code to use `asyncvictoria` instead of `asyncakumuli`
2. Change connection parameters:
   - Write port: 8282 → 4242 (or your configured OpenTSDB port)
   - Read URL: `http://host:8181/api/query` → `http://host:8428/api/v1/query`
3. Update your queries to use VictoriaMetrics query format
4. Consider VictoriaMetrics' additional features like:
   - Prometheus-compatible remote write
   - Native InfluxDB line protocol support
   - Built-in deduplication
   - Downsampling

## Troubleshooting

### "Connection refused" errors

Ensure VictoriaMetrics is running with OpenTSDB protocol enabled:
```bash
victoria-metrics -opentsdbListenAddr=:4242
```

### Data not appearing in VictoriaMetrics

1. Check VictoriaMetrics logs for errors
2. Verify the data format is correct
3. Try querying with a wider time range
4. Force flush data to disk: `curl -X GET http://localhost:8428/internal/force_flush`

### Slow migration

- Increase batch size in the migration script
- Use multiple parallel migration processes for different series
- Ensure sufficient network bandwidth between Akumuli and VictoriaMetrics

## Support

For issues with:
- `asyncakumuli`: See https://github.com/smurfix/asyncakumuli
- `asyncvictoria`: See https://github.com/smurfix/asyncvictoria
- VictoriaMetrics: See https://docs.victoriametrics.com/
