.. _example_cli_workflows:

================
CLI Workflows
================

This example demonstrates common workflows using the MedTDA command-line interface for batch processing, configuration management, and automation.

.. contents:: Contents
   :local:
   :depth: 2

Basic CLI Usage
===============

Single Image Processing
-----------------------

**Extract features from one image:**

.. code-block:: bash

    medtda extract image.nii.gz

**With mask:**

.. code-block:: bash

    medtda extract image.nii.gz --mask roi_mask.nii.gz

**Specify output:**

.. code-block:: bash

    medtda extract image.nii.gz --output features.npy

**Choose vectorization method:**

.. code-block:: bash

    medtda extract image.nii.gz --method persistence_image

**Multiple methods:**

.. code-block:: bash

    medtda extract image.nii.gz --method persistence_stats --method betti_curve

Configuration Files
===================

Basic Configuration
-------------------

Create ``config.yaml``:

.. code-block:: yaml

    # Preprocessing
    preprocessing:
      normalize: true
      normalize_method: robust
      spacing: [1.0, 1.0, 1.0]
      crop_to_roi: true
      roi_padding: 5
    
    # Persistent homology
    persistent_homology:
      filtration_type: sublevel
      max_dimension: 2
    
    # Vectorization
    vectorization:
      method: persistence_image
      parameters:
        resolution: 20
        bandwidth: 0.1

**Use configuration:**

.. code-block:: bash

    medtda extract image.nii.gz --config config.yaml

CT-Specific Configuration
--------------------------

Create ``config_ct.yaml``:

.. code-block:: yaml

    preprocessing:
      window: [-60, 340]  # Soft tissue window
      spacing: [1.0, 1.0, 1.0]
      normalize: true
      normalize_method: robust
      crop_to_roi: true
    
    persistent_homology:
      filtration_type: sublevel
      max_dimension: 2
    
    vectorization:
      method: persistence_stats

**Usage:**

.. code-block:: bash

    medtda extract ct_scan.nii.gz \
        --mask liver_mask.nii.gz \
        --config config_ct.yaml \
        --output ct_features.npy

MRI-Specific Configuration
---------------------------

Create ``config_mri.yaml``:

.. code-block:: yaml

    preprocessing:
      normalize: true
      normalize_method: zscore
      spacing: [1.0, 1.0, 1.0]
      crop_to_roi: true
      clip_percentiles: [1, 99]
    
    persistent_homology:
      filtration_type: sublevel
      max_dimension: 2
    
    vectorization:
      method: persistence_image
      parameters:
        resolution: 30
        bandwidth: 0.08

**Usage:**

.. code-block:: bash

    medtda extract brain_mri.nii.gz \
        --mask tumor_mask.nii.gz \
        --config config_mri.yaml

Batch Processing
================

Process Directory
-----------------

**Process all images in directory:**

.. code-block:: bash

    medtda batch process data/images/ --output results/

**With glob pattern:**

.. code-block:: bash

    medtda batch process "data/images/*.nii.gz" --output results/

**With masks:**

.. code-block:: bash

    medtda batch process data/images/ \
        --mask-dir data/masks/ \
        --output results/

**Parallel processing:**

.. code-block:: bash

    medtda batch process data/images/ \
        --output results/ \
        --n-jobs 8

Using File Lists
----------------

Create ``file_list.txt``:

.. code-block:: text

    data/patient001/image.nii.gz
    data/patient002/image.nii.gz
    data/patient003/image.nii.gz

**Process from list:**

.. code-block:: bash

    medtda batch process --file-list file_list.txt --output results/

**With corresponding masks:**

Create ``files.csv``:

.. code-block:: text

    image,mask
    data/patient001/image.nii.gz,data/patient001/mask.nii.gz
    data/patient002/image.nii.gz,data/patient002/mask.nii.gz
    data/patient003/image.nii.gz,data/patient003/mask.nii.gz

.. code-block:: bash

    medtda batch process --file-list files.csv --output results/

Output Formats
==============

NumPy Format
------------

**Save as .npy (default):**

.. code-block:: bash

    medtda extract image.nii.gz --output features.npy

**Load in Python:**

.. code-block:: python

    import numpy as np
    features = np.load('features.npy')

CSV Format
----------

**Save as CSV:**

.. code-block:: bash

    medtda extract image.nii.gz --output features.csv --format csv

**Batch to CSV:**

.. code-block:: bash

    medtda batch process data/images/ \
        --output results.csv \
        --format csv

**Example CSV output:**

.. code-block:: text

    filename,feature_1,feature_2,...,feature_13
    image001.nii.gz,0.234,0.456,...,0.789
    image002.nii.gz,0.123,0.345,...,0.567

JSON Format
-----------

**Save as JSON:**

.. code-block:: bash

    medtda extract image.nii.gz --output features.json --format json

**Example output:**

.. code-block:: none

    {
      "filename": "image.nii.gz",
      "method": "persistence_stats",
      "features": [0.234, 0.456, ..., 0.789],
      "metadata": {
        "n_features": 13,
        "image_shape": [256, 256, 180]
      }
    }

HDF5 Format
-----------

**Batch to HDF5:**

.. code-block:: bash

    medtda batch process data/images/ \
        --output dataset.h5 \
        --format hdf5

**Load in Python:**

.. code-block:: python

    import h5py
    
    with h5py.File('dataset.h5', 'r') as f:
        features = f['features'][:]
        filenames = f['filenames'][:]

Advanced Workflows
==================

Multi-Method Extraction
-----------------------

**Extract multiple vectorization methods:**

.. code-block:: bash

    medtda batch process data/images/ \
        --method persistence_stats \
        --method betti_curve \
        --method persistence_image \
        --output results/

**Output structure:**

.. code-block:: text

    results/
      persistence_stats/
        image001_features.npy
        image002_features.npy
        ...
      betti_curve/
        image001_features.npy
        image002_features.npy
        ...
      persistence_image/
        image001_features.npy
        image002_features.npy
        ...

With Barcodes
-------------

**Extract and save persistence barcodes:**

.. code-block:: bash

    medtda extract image.nii.gz \
        --save-barcodes \
        --output-dir results/

**Output:**

.. code-block:: text

    results/
      image_features.npy
      image_barcodes.npz

**Load barcodes:**

.. code-block:: python

    import numpy as np
    
    barcodes = np.load('image_barcodes.npz')
    h0 = barcodes['H0']
    h1 = barcodes['H1']
    h2 = barcodes['H2']

Quality Control
===============

Validation Mode
---------------

**Validate images before processing:**

.. code-block:: bash

    medtda batch validate data/images/ \
        --mask-dir data/masks/

**Example output:**

.. code-block:: text

    Validating 100 images...
    
    ✓ image001.nii.gz: OK
    ✗ image002.nii.gz: Shape mismatch with mask
    ✓ image003.nii.gz: OK
    ✗ image004.nii.gz: Empty mask
    ...
    
    Summary:
      Valid: 96
      Invalid: 4
    
    See validation_report.txt for details

**Process only valid:**

.. code-block:: bash

    medtda batch process data/images/ \
        --mask-dir data/masks/ \
        --validate \
        --output results/

Error Handling
--------------

**Skip errors and continue:**

.. code-block:: bash

    medtda batch process data/images/ \
        --output results/ \
        --skip-errors

**Log errors:**

.. code-block:: bash

    medtda batch process data/images/ \
        --output results/ \
        --skip-errors \
        --error-log errors.log

**Example error log:**

.. code-block:: text

    image002.nii.gz: Shape mismatch with mask (256,256,180) vs (256,256,160)
    image015.nii.gz: Empty ROI after masking
    image043.nii.gz: Unable to load file: corrupted NIFTI header

Resume Processing
-----------------

**Resume interrupted batch:**

.. code-block:: bash

    medtda batch process data/images/ \
        --output results/ \
        --resume

**How it works:**

- Checks which files already processed
- Skips completed files
- Continues from where it stopped

Filtering and Selection
========================

By ROI Size
-----------

**Only process if ROI is large enough:**

.. code-block:: bash

    medtda batch process data/images/ \
        --mask-dir data/masks/ \
        --min-roi-voxels 1000 \
        --output results/

By Label
--------

**Extract specific label from multi-label mask:**

.. code-block:: bash

    medtda extract image.nii.gz \
        --mask multilabel_mask.nii.gz \
        --label 2 \
        --output features.npy

**Batch for all labels:**

.. code-block:: bash

    for label in 1 2 3; do
        medtda batch process data/images/ \
            --mask-dir data/masks/ \
            --label $label \
            --output results/label_$label/
    done

Preprocessing Exploration
==========================

Different Normalizations
------------------------

**Compare normalization methods:**

.. code-block:: bash

    # MinMax
    medtda extract image.nii.gz \
        --normalize minmax \
        --output features_minmax.npy
    
    # Z-score
    medtda extract image.nii.gz \
        --normalize zscore \
        --output features_zscore.npy
    
    # Robust
    medtda extract image.nii.gz \
        --normalize robust \
        --output features_robust.npy

Different Resolutions
---------------------

**Test different resamplings:**

.. code-block:: bash

    for spacing in 0.5 1.0 2.0; do
        medtda extract image.nii.gz \
            --spacing $spacing $spacing $spacing \
            --output features_spacing${spacing}.npy
    done

Different Windows (CT)
----------------------

**Test CT windowing:**

.. code-block:: bash

    # Soft tissue
    medtda extract ct_scan.nii.gz \
        --window -60 340 \
        --output features_soft_tissue.npy
    
    # Lung
    medtda extract ct_scan.nii.gz \
        --window -1000 400 \
        --output features_lung.npy
    
    # Bone
    medtda extract ct_scan.nii.gz \
        --window 300 1800 \
        --output features_bone.npy

Vectorization Parameter Tuning
===============================

Persistence Image Resolution
-----------------------------

**Test different resolutions:**

.. code-block:: bash

    for res in 10 20 30 50; do
        medtda extract image.nii.gz \
            --method persistence_image \
            --pi-resolution $res \
            --output features_pi${res}.npy
    done

Betti Curve Resolution
----------------------

.. code-block:: bash

    for res in 50 100 200; do
        medtda extract image.nii.gz \
            --method betti_curve \
            --betti-resolution $res \
            --output features_betti${res}.npy
    done

Landscape Layers
----------------

.. code-block:: bash

    for layers in 3 5 10; do
        medtda extract image.nii.gz \
            --method persistence_landscape \
            --landscape-layers $layers \
            --output features_landscape${layers}.npy
    done

Automation Scripts
==================

Bash Script: Process Dataset
-----------------------------

Create ``process_dataset.sh``:

.. code-block:: bash

    #!/bin/bash
    
    # Configuration
    DATA_DIR="data"
    OUTPUT_DIR="results"
    CONFIG="config.yaml"
    N_JOBS=8
    
    # Create output directory
    mkdir -p "$OUTPUT_DIR"
    
    # Process images
    echo "Processing images..."
    medtda batch process "$DATA_DIR/images/" \
        --mask-dir "$DATA_DIR/masks/" \
        --config "$CONFIG" \
        --output "$OUTPUT_DIR/features.csv" \
        --format csv \
        --n-jobs $N_JOBS \
        --skip-errors \
        --error-log "$OUTPUT_DIR/errors.log"
    
    # Validation
    if [ $? -eq 0 ]; then
        echo "✓ Processing complete!"
        echo "Results saved to $OUTPUT_DIR/features.csv"
    else
        echo "✗ Processing failed. Check $OUTPUT_DIR/errors.log"
        exit 1
    fi

**Run:**

.. code-block:: bash

    chmod +x process_dataset.sh
    ./process_dataset.sh

Python Script: Multi-Configuration
-----------------------------------

Create ``batch_configs.py``:

.. code-block:: python

    #!/usr/bin/env python3
    import subprocess
    from pathlib import Path
    
    # Configurations to test
    configs = {
        'ct_soft_tissue': 'configs/ct_soft.yaml',
        'ct_lung': 'configs/ct_lung.yaml',
        'mri_t1': 'configs/mri_t1.yaml',
        'mri_t2': 'configs/mri_t2.yaml'
    }
    
    data_dir = Path('data/images')
    mask_dir = Path('data/masks')
    output_base = Path('results')
    
    for name, config in configs.items():
        print(f"\n{'='*60}")
        print(f"Processing: {name}")
        print(f"{'='*60}\n")
        
        output_dir = output_base / name
        output_dir.mkdir(parents=True, exist_ok=True)
        
        cmd = [
            'medtda', 'batch', 'process',
            str(data_dir),
            '--mask-dir', str(mask_dir),
            '--config', config,
            '--output', str(output_dir / 'features.csv'),
            '--format', 'csv',
            '--n-jobs', '8',
            '--skip-errors'
        ]
        
        result = subprocess.run(cmd)
        
        if result.returncode == 0:
            print(f"✓ {name} complete!")
        else:
            print(f"✗ {name} failed!")

**Run:**

.. code-block:: bash

    python batch_configs.py

Integration with ML Pipelines
==============================

Extract and Train
-----------------

.. code-block:: bash

    #!/bin/bash
    
    # 1. Extract features
    echo "Extracting features..."
    medtda batch process data/train/images/ \
        --mask-dir data/train/masks/ \
        --config config.yaml \
        --output train_features.csv \
        --format csv
    
    medtda batch process data/test/images/ \
        --mask-dir data/test/masks/ \
        --config config.yaml \
        --output test_features.csv \
        --format csv
    
    # 2. Train model (Python script)
    echo "Training model..."
    python train_model.py \
        --train train_features.csv \
        --test test_features.csv \
        --labels data/labels.csv \
        --output model.pkl
    
    # 3. Evaluate
    echo "Evaluating..."
    python evaluate.py \
        --model model.pkl \
        --test test_features.csv \
        --labels data/labels.csv

Cross-Validation  Pipeline
---------------------------

Create ``cv_pipeline.sh``:

.. code-block:: bash

    #!/bin/bash
    
    N_FOLDS=5
    
    for fold in $(seq 0 $((N_FOLDS-1))); do
        echo "Processing fold $fold..."
        
        # Extract train features
        medtda batch process \
            --file-list "folds/fold_${fold}_train.txt" \
            --mask-dir data/masks/ \
            --config config.yaml \
            --output "folds/fold_${fold}_train_features.csv"
        
        # Extract validation features
        medtda batch process \
            --file-list "folds/fold_${fold}_val.txt" \
            --mask-dir data/masks/ \
            --config config.yaml \
            --output "folds/fold_${fold}_val_features.csv"
        
        # Train and evaluate
        python train_fold.py --fold $fold
    done
    
    # Aggregate results
    python aggregate_cv_results.py --n-folds $N_FOLDS

Profiling and Optimization
===========================

Timing
------

**Time extraction:**

.. code-block:: bash

    time medtda extract large_image.nii.gz --output features.npy

**Batch timing:**

.. code-block:: bash

    time medtda batch process data/images/ \
        --output results/ \
        --n-jobs 8

**Compare configurations:**

.. code-block:: bash

    for config in config1.yaml config2.yaml config3.yaml; do
        echo "Testing $config..."
        time medtda batch process data/images/ \
            --config $config \
            --output tmp/
        echo ""
    done

Memory Profiling
----------------

**Monitor memory usage:**

.. code-block:: bash

    /usr/bin/time -v medtda batch process data/images/ \
        --output results/ \
        2>&1 | grep "Maximum resident set size"

Useful Aliases
==============

Add to ``~/.bashrc`` or ``~/.zshrc``:

.. code-block:: bash

    # Quick extraction with defaults
    alias medtda-quick='medtda extract --normalize --method persistence_stats'
    
    # CT processing
    alias medtda-ct='medtda extract --config ~/configs/ct.yaml'
    
    # MRI processing
    alias medtda-mri='medtda extract --config ~/configs/mri.yaml'
    
    # Batch with masks
    alias medtda-batch='medtda batch process --mask-dir masks/ --output results/'

**Usage:**

.. code-block:: bash

    medtda-quick image.nii.gz
    medtda-ct ct_scan.nii.gz --mask liver.nii.gz
    medtda-mri brain.nii.gz --mask tumor.nii.gz

Debugging
=========

Verbose Mode
------------

.. code-block:: bash

    medtda extract image.nii.gz --verbose

**Output:**

.. code-block:: text

    [INFO] Loading image: image.nii.gz
    [INFO] Image shape: (256, 256, 180)
    [INFO] Normalizing with method: robust
    [INFO] Computing persistent homology...
    [INFO] Filtration type: sublevel
    [INFO] Max dimension: 2
    [INFO] Vectorizing with persistence_stats...
    [INFO] Feature vector size: 13
    [INFO] Saving to: features.npy
    [SUCCESS] Done!

Dry Run
-------

**See what would be processed without actually processing:**

.. code-block:: bash

    medtda batch process data/images/ \
        --mask-dir data/masks/ \
        --dry-run

**Output:**

.. code-block:: text

    Would process:
      image001.nii.gz -> mask001.nii.gz
      image002.nii.gz -> mask002.nii.gz
      image003.nii.gz -> mask003.nii.gz
      ...
    Total: 100 images

Next Steps
==========

* :doc:`basic_usage` - Basic feature extraction
* :doc:`batch_workflow` - Python batch processing
* :doc:`../cli` - Complete CLI reference
* :doc:`../user_guide` - User guide

See Also
========

* :doc:`../quickstart` - Quick start guide
* :doc:`../api/cli` - CLI API reference
