<!-- cd04e3ec-916f-4418-922a-390f05ece83b -->
---
todos:
  - id: "1-create-project"
    content: "Create Flutter project: flutter create flutter_app"
    status: pending
  - id: "2-add-deps"
    content: "Add dependencies: ffmpeg_kit_flutter_new_min, file_picker, gallery_saver_plus, path_provider, permission_handler"
    status: pending
  - id: "3-ffmpeg-service"
    content: "Create lib/services/ffmpeg_service.dart - PCM extraction, duration, fps, merge"
    status: pending
  - id: "4-correlation"
    content: "Create lib/utils/correlation.dart - waveform cross-correlation"
    status: pending
  - id: "5-sync-service"
    content: "Create lib/services/sync_service.dart - orchestrate extract, correlate, merge, save"
    status: pending
  - id: "6-ui"
    content: "Create lib/pages/home_page.dart - file pickers, progress, sync button"
    status: pending
  - id: "7-permissions"
    content: "Configure Android permissions in AndroidManifest.xml"
    status: pending
  - id: "8-build"
    content: "Build APK: flutter build apk --release"
    status: pending
isProject: false
---
# Flutter Audio-Video Sync App

## Overview

A Flutter app in `flutter_app/` that replicates the Python CLI functionality using `ffmpeg_kit_flutter_new` - the only actively maintained FFmpeg library for mobile.

## Architecture

```mermaid
flowchart TD
    subgraph ui [UI Layer]
        HomePage[HomePage Widget]
        FilePickers[Video/Audio Pickers]
        ProgressBar[Progress Indicator]
    end
    
    subgraph services [Service Layer]
        SyncService[sync_service.dart]
        FFmpegService[ffmpeg_service.dart]
    end
    
    subgraph core [Core Logic]
        ExtractPCM[Extract PCM Audio]
        Correlate[Cross-correlate]
        Merge[FFmpeg Merge]
    end
    
    HomePage --> FilePickers
    HomePage --> SyncService
    SyncService --> FFmpegService
    FFmpegService --> ExtractPCM
    SyncService --> Correlate
    FFmpegService --> Merge
    Merge --> GallerySave[Save to Gallery]
```

## File Structure

```
flutter_app/
├── lib/
│   ├── main.dart                 # App entry, MaterialApp
│   ├── pages/
│   │   └── home_page.dart        # Main UI
│   ├── services/
│   │   ├── sync_service.dart     # Orchestrates sync flow
│   │   └── ffmpeg_service.dart   # FFmpeg commands
│   └── utils/
│       └── correlation.dart      # Cross-correlation algorithm
├── pubspec.yaml                  # Dependencies
└── android/app/build.gradle      # Android config
```

## Dependencies (pubspec.yaml)

```yaml
dependencies:
  flutter:
    sdk: flutter
  ffmpeg_kit_flutter_new_min: ^4.1.0  # Minimal FFmpeg (smallest size)
  file_picker: ^8.0.0                  # Pick video/audio files
  gallery_saver_plus: ^3.2.9           # Save to gallery
  path_provider: ^2.1.0                # Temp directories
  permission_handler: ^11.0.0          # Request permissions
```

## Algorithm (Dart port from [sync.py](src/audio_video_sync/sync.py))

**1. Extract PCM** - Same FFmpeg command as Python:
```dart
// lib/services/ffmpeg_service.dart
Future<Float32List> extractPCM(String inputPath) async {
  final tempDir = await getTemporaryDirectory();
  final pcmPath = '${tempDir.path}/pcm_${DateTime.now().millisecondsSinceEpoch}.raw';
  
  await FFmpegKit.execute(
    '-y -i "$inputPath" -t 30 -ac 1 -ar 8000 -f f32le "$pcmPath"'
  );
  
  final bytes = await File(pcmPath).readAsBytes();
  await File(pcmPath).delete();
  return bytes.buffer.asFloat32List();
}
```

**2. Cross-correlate** - Waveform only (simplified):
```dart
// lib/utils/correlation.dart
double correlate(Float32List audio1, Float32List audio2, int sampleRate) {
  // Limited lag range: +/- 10 seconds
  final maxLag = sampleRate * 10;
  double maxCorr = double.negativeInfinity;
  int maxIdx = 0;
  
  for (int lag = -maxLag; lag < maxLag; lag++) {
    double sum = 0;
    for (int i = 0; i < audio1.length; i++) {
      final j = i - lag;
      if (j >= 0 && j < audio2.length) {
        sum += audio1[i] * audio2[j];
      }
    }
    if (sum.abs() > maxCorr) {
      maxCorr = sum.abs();
      maxIdx = lag;
    }
  }
  
  return maxIdx / sampleRate; // offset in seconds
}
```

**3. Merge** - Same FFmpeg command as [ffmpeg.py](src/audio_video_sync/ffmpeg.py):
```dart
// lib/services/ffmpeg_service.dart
Future<String> merge(String videoPath, String audioPath, double offset) async {
  final tempDir = await getTemporaryDirectory();
  final outputPath = '${tempDir.path}/synced_${DateTime.now().millisecondsSinceEpoch}.mp4';
  final audioDuration = await getDuration(audioPath);
  final fps = await getFrameRate(videoPath);
  
  String cmd;
  if (offset >= 0) {
    cmd = '-y -ss $offset -i "$videoPath" -i "$audioPath" '
          '-map 0:v:0 -map 1:a:0 -fps_mode cfr -r ${fps.round()} '
          '-c:v libx264 -preset fast -crf 18 -c:a aac -b:a 192k '
          '-t $audioDuration -movflags +faststart "$outputPath"';
  } else {
    final trim = offset.abs();
    final effectiveDuration = audioDuration - trim;
    cmd = '-y -i "$videoPath" -ss $trim -i "$audioPath" '
          '-map 0:v:0 -map 1:a:0 -fps_mode cfr -r ${fps.round()} '
          '-c:v libx264 -preset fast -crf 18 -c:a aac -b:a 192k '
          '-t $effectiveDuration -movflags +faststart "$outputPath"';
  }
  
  await FFmpegKit.execute(cmd);
  return outputPath;
}
```

## UI (home_page.dart)

Simple Material Design UI:
- Two cards for video/audio selection (file_picker)
- "Sync" button (disabled until both selected)
- LinearProgressIndicator during processing
- Status text showing current step
- Success dialog with "Open Gallery" option

## Build Steps

1. Create Flutter project: `flutter create flutter_app`
2. Add dependencies to pubspec.yaml
3. Implement services and UI
4. Configure Android permissions in AndroidManifest.xml
5. Build APK: `flutter build apk --release`

## Simplifications from Python

- Waveform correlation only (skip chromagram - requires complex DSP)
- 8kHz sample rate, 30s duration (faster on mobile)
- Software encoding only (libx264 - more reliable cross-device)
