{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# QSIRecon Basics: Diffusion MRI Reconstruction\n",
    "\n",
    "This notebook demonstrates how to use the QSIRecon runner for diffusion reconstruction and connectivity analysis.\n",
    "\n",
    "## Overview\n",
    "\n",
    "QSIRecon performs:\n",
    "- Diffusion model fitting (DTI, CSD, MAPMRI, etc.)\n",
    "- Tractography\n",
    "- Structural connectivity matrix generation\n",
    "- Regional quantification\n",
    "\n",
    "## Prerequisites\n",
    "\n",
    "- Docker installed and running\n",
    "- QSIPrep preprocessed data\n",
    "- Reconstruction specification file (JSON/YAML)\n",
    "- FreeSurfer license file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "from voxelops import (\n",
    "    QSIReconDefaults,\n",
    "    QSIReconInputs,\n",
    "    run_qsirecon,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define Paths"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Input paths -- update these to match your data\n",
    "qsiprep_dir = Path(\"/data/derivatives/qsiprep/\")\n",
    "participant = \"01\"\n",
    "recon_spec = Path(\"/config/recon_specs/dsi_studio_gqi.json\")\n",
    "fs_license = Path(\"/opt/freesurfer/license.txt\")\n",
    "\n",
    "# Output paths (optional)\n",
    "output_dir = Path(\"/data/derivatives/qsirecon/\")\n",
    "work_dir = Path(\"/data/work/qsirecon/\")\n",
    "\n",
    "# Datasets to include (optional -- for custom atlas datasets)\n",
    "datasets = {\n",
    "    \"atlases\": \"/data/atlases/Schaefer2018Tian2020_atlases\"\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understanding Reconstruction Specs\n",
    "\n",
    "QSIRecon requires a reconstruction specification file that defines:\n",
    "- Which diffusion models to fit\n",
    "- Tractography algorithms to use\n",
    "- Parcellation schemes for connectivity\n",
    "- Atlas registrations\n",
    "\n",
    "Common specs:\n",
    "- `dsi_studio_gqi`: DSI Studio's GQI model with tractography\n",
    "- `mrtrix_multishell_msmt_ACT-fast`: Multi-shell MSMT-CSD with ACT\n",
    "- `amico_noddi`: AMICO NODDI model fitting\n",
    "- `dipy_mapmri`: DIPY MAPMRI reconstruction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reconstruction Specification:\n",
      "name: gal_multishell_scalars\n",
      "nodes:\n",
      "- action: DKI_reconstruction\n",
      "  name: dipy_dki\n",
      "  parameters:\n",
      "    wmti: true\n",
      "    write_fibgz: false\n",
      "    write_mif: false\n",
      "  qsirecon_suffix: DIPYDKI\n",
      "  software: Dipy\n",
      "- action: fit_noddi\n",
      "  name: amico_noddi\n",
      "  parameters:\n",
      "    dIso: 0.003\n",
      "    dPar: 0.0017\n",
      "    isExvivo: false\n",
      "  qsirecon_suffix: AMICONODDI\n",
      "  software: AMICO\n",
      "- action: MAPMRI_reconstruction\n",
      "  name: dipy_mapmri\n",
      "  parameters:\n",
      "    anisotropic_scaling: false\n",
      "    big_delta: null\n",
      "    bval_threshold: 2000\n",
      "    dti_scale_estimation: false\n",
      "    laplacian_regularization: true\n",
      "    laplacian_weighting: 0.2\n",
      "    radial_order: 6\n",
      "    small_delta: null\n",
      "    write_fibgz: true\n",
      "    write_mif: true\n",
      "  qsirecon_suffix: DIPYMAPMRI\n",
      "  software: Dipy\n",
      "- action: reconstruction\n",
      "  input: qsirecon\n",
      "  name: dsistudio_gqi\n",
      "  parameters:\n",
      "    method: gqi\n",
      "  qsirecon_suffix: DSIStudio\n",
      "  software: DSI Studio\n",
      "- action: export\n",
      "  input: dsistudio_gqi\n",
      "  name: scalar_export\n",
      "  qsirecon_suffix: DSIStudio\n",
      "  software: DSI Studio\n",
      "- action: template_map\n",
      "  input: qsirecon\n",
      "  name: template_map\n",
      "  parameters:\n",
      "    interpolation: Linear\n",
      "  scalars_from:\n",
      "  - dipy_dki\n",
      "  - amico_noddi\n",
      "  - dipy_mapmri\n",
      "  - scalar_export\n",
      "  software: qsirecon\n",
      "space: T1w\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# View reconstruction spec (yaml file)\n",
    "import yaml\n",
    "\n",
    "if recon_spec.exists():\n",
    "    spec_content = yaml.safe_load(recon_spec.read_text())\n",
    "    print(\"Reconstruction Specification:\")\n",
    "    print(yaml.dump(spec_content, indent=2))\n",
    "else:\n",
    "    print(f\"Recon spec not found: {recon_spec}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Usage\n",
    "\n",
    "### Option 1: Use Default Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create inputs\n",
    "inputs = QSIReconInputs(\n",
    "    qsiprep_dir=qsiprep_dir,\n",
    "    participant=participant,\n",
    "    recon_spec=recon_spec,\n",
    "    output_dir=output_dir,\n",
    "    work_dir=work_dir,\n",
    "    datasets=datasets,\n",
    "    atlases=[\"Schaefer2018N100n7Tian2020S2\", \"Schaefer2018N200n7Tian2020S2\"],\n",
    ")\n",
    "\n",
    "# Run with defaults\n",
    "result = run_qsirecon(inputs, fs_license=fs_license)\n",
    "\n",
    "print(f\"Success: {result['success']}\")\n",
    "print(f\"Duration: {result['duration_human']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Option 2: Increase Resources for Tractography"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tractography is computationally intensive\n",
    "result = run_qsirecon(\n",
    "    inputs,\n",
    "    fs_license=fs_license,\n",
    "    nprocs=32,       # More cores = faster tractography\n",
    "    mem_mb=64000,    # Sufficient memory for large atlases\n",
    ")\n",
    "\n",
    "print(f\"Success: {result['success']}\")\n",
    "print(f\"Processing time: {result['duration_human']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Option 3: Custom Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create custom configuration\n",
    "config = QSIReconDefaults(\n",
    "    nprocs=24,\n",
    "    mem_mb=48000,\n",
    "    fs_license=fs_license,\n",
    "    docker_image=\"pennlinc/qsirecon:latest\",\n",
    ")\n",
    "\n",
    "result = run_qsirecon(inputs, config)\n",
    "\n",
    "print(f\"Success: {result['success']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inspect Execution Record"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Execution Details:\")\n",
    "print(f\"  Tool: {result['tool']}\")\n",
    "print(f\"  Participant: {result['participant']}\")\n",
    "print(f\"  Duration: {result['duration_human']}\")\n",
    "print(f\"  Success: {result['success']}\")\n",
    "\n",
    "print(\"\\nConfiguration Used:\")\n",
    "config_used = result[\"config\"]\n",
    "print(f\"  Cores: {config_used.nprocs}\")\n",
    "print(f\"  Memory: {config_used.mem_mb}MB\")\n",
    "print(f\"  Docker image: {config_used.docker_image}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Check Expected Outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "outputs = result[\"expected_outputs\"]\n",
    "\n",
    "print(\"Expected Output Locations:\")\n",
    "print(f\"  QSIRecon directory: {outputs.qsirecon_dir}\")\n",
    "print(f\"  Participant directory: {outputs.participant_dir}\")\n",
    "print(f\"  HTML report: {outputs.html_report}\")\n",
    "print(f\"  Work directory: {outputs.work_dir}\")\n",
    "\n",
    "# Verify outputs\n",
    "print(\"\\nOutput Validation:\")\n",
    "print(f\"  HTML report exists: {outputs.html_report.exists()}\")\n",
    "print(f\"  Participant dir exists: {outputs.participant_dir.exists()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Explore Reconstruction Outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if outputs.participant_dir.exists():\n",
    "    print(f\"Reconstruction outputs for {participant}:\\n\")\n",
    "\n",
    "    # Organize by type\n",
    "    scalar_maps = []\n",
    "    tractography = []\n",
    "    connectivity = []\n",
    "    other = []\n",
    "\n",
    "    for f in outputs.participant_dir.rglob(\"*\"):\n",
    "        if f.is_file():\n",
    "            name = f.name\n",
    "            if name.endswith(\".nii.gz\"):\n",
    "                scalar_maps.append(f)\n",
    "            elif name.endswith((\".trk\", \".tck\", \".fib\")):\n",
    "                tractography.append(f)\n",
    "            elif name.endswith(\".csv\"):\n",
    "                connectivity.append(f)\n",
    "            else:\n",
    "                other.append(f)\n",
    "\n",
    "    print(f\"Scalar Maps ({len(scalar_maps)}):\")\n",
    "    for f in sorted(scalar_maps)[:10]:  # Show first 10\n",
    "        print(f\"  {f.name}\")\n",
    "\n",
    "    print(f\"\\nTractography Files ({len(tractography)}):\")\n",
    "    for f in sorted(tractography):\n",
    "        print(f\"  {f.name} ({f.stat().st_size / (1024**2):.1f} MB)\")\n",
    "\n",
    "    print(f\"\\nConnectivity Matrices ({len(connectivity)}):\")\n",
    "    for f in sorted(connectivity):\n",
    "        print(f\"  {f.name}\")\n",
    "else:\n",
    "    print(\"Participant directory not found\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and Visualize Connectivity Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "\n",
    "# Find connectivity matrices\n",
    "connectivity_files = list(outputs.participant_dir.rglob(\"*connectivity*.csv\"))\n",
    "\n",
    "if connectivity_files:\n",
    "    # Load first connectivity matrix\n",
    "    conn_file = connectivity_files[0]\n",
    "    print(f\"Loading: {conn_file.name}\\n\")\n",
    "\n",
    "    # Load matrix\n",
    "    conn_matrix = pd.read_csv(conn_file, index_col=0)\n",
    "\n",
    "    print(f\"Matrix shape: {conn_matrix.shape}\")\n",
    "    print(f\"ROIs: {len(conn_matrix)}\")\n",
    "    print(\"\\nFirst few ROI names:\")\n",
    "    print(conn_matrix.index[:5].tolist())\n",
    "\n",
    "    # Visualize\n",
    "    plt.figure(figsize=(12, 10))\n",
    "    plt.imshow(conn_matrix.values, cmap=\"hot\", interpolation=\"nearest\")\n",
    "    plt.colorbar(label=\"Connection Strength\")\n",
    "    plt.title(f\"Structural Connectivity Matrix\\n{conn_file.name}\")\n",
    "    plt.xlabel(\"ROI Index\")\n",
    "    plt.ylabel(\"ROI Index\")\n",
    "    plt.tight_layout()\n",
    "    plt.show()\n",
    "\n",
    "    # Basic statistics\n",
    "    print(\"\\nMatrix Statistics:\")\n",
    "    print(f\"  Mean connection strength: {conn_matrix.values.mean():.4f}\")\n",
    "    print(f\"  Max connection strength: {conn_matrix.values.max():.4f}\")\n",
    "    print(f\"  Sparsity: {(conn_matrix.values == 0).sum() / conn_matrix.size:.1%}\")\n",
    "else:\n",
    "    print(\"No connectivity matrices found\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## View HTML QC Report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import IFrame\n",
    "\n",
    "if outputs.html_report.exists():\n",
    "    IFrame(src=str(outputs.html_report), width=900, height=600)\n",
    "else:\n",
    "    print(f\"HTML report not found: {outputs.html_report}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Processing Multiple Reconstruction Specs\n",
    "\n",
    "You might want to run different reconstruction pipelines on the same data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define multiple reconstruction specs\n",
    "recon_specs = [\n",
    "    Path(\"/config/recon_specs/dsi_studio_gqi.json\"),\n",
    "    Path(\"/config/recon_specs/mrtrix_msmt_csd.json\"),\n",
    "    Path(\"/config/recon_specs/amico_noddi.json\"),\n",
    "]\n",
    "\n",
    "config = QSIReconDefaults(\n",
    "    nprocs=24,\n",
    "    mem_mb=48000,\n",
    "    fs_license=fs_license,\n",
    ")\n",
    "\n",
    "results = []\n",
    "\n",
    "for spec in recon_specs:\n",
    "    if not spec.exists():\n",
    "        print(f\"Skipping {spec.name} (not found)\")\n",
    "        continue\n",
    "\n",
    "    print(f\"\\nRunning reconstruction: {spec.stem}\")\n",
    "\n",
    "    inputs = QSIReconInputs(\n",
    "        qsiprep_dir=qsiprep_dir,\n",
    "        participant=participant,\n",
    "        recon_spec=spec,\n",
    "    )\n",
    "\n",
    "    try:\n",
    "        result = run_qsirecon(inputs, config)\n",
    "        results.append(result)\n",
    "        print(f\"  Success in {result['duration_human']}\")\n",
    "    except Exception as e:\n",
    "        print(f\"  Failed: {e}\")\n",
    "        results.append({\"recon_spec\": spec.name, \"success\": False, \"error\": str(e)})\n",
    "\n",
    "# Summary\n",
    "print(f\"\\nCompleted {len(results)} reconstructions\")\n",
    "for r in results:\n",
    "    if r.get(\"success\"):\n",
    "        print(f\"  {r['inputs'].recon_spec.stem}: {r['duration_human']}\")\n",
    "    else:\n",
    "        print(f\"  {r.get('recon_spec', 'unknown')}: Failed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Error Handling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from voxelops.exceptions import (\n",
    "    InputValidationError,\n",
    "    ProcedureExecutionError,\n",
    ")\n",
    "\n",
    "try:\n",
    "    result = run_qsirecon(\n",
    "        inputs,\n",
    "        fs_license=fs_license,\n",
    "        nprocs=24,\n",
    "    )\n",
    "    print(f\"Success: {result['success']}\")\n",
    "\n",
    "except InputValidationError as e:\n",
    "    print(f\"Input validation failed: {e}\")\n",
    "    print(\"Common issues:\")\n",
    "    print(\"  - QSIPrep directory doesn't exist\")\n",
    "    print(\"  - Participant not found in QSIPrep output\")\n",
    "    print(\"  - Reconstruction spec file not found or invalid\")\n",
    "\n",
    "except ProcedureExecutionError as e:\n",
    "    print(f\"Execution failed: {e}\")\n",
    "    print(f\"Check logs: {result.get('log_file')}\")\n",
    "\n",
    "except Exception as e:\n",
    "    print(f\"Unexpected error: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "After QSIRecon reconstruction:\n",
    "\n",
    "1. Review the HTML QC report\n",
    "2. Inspect connectivity matrices\n",
    "3. Visualize tractography (using DSI Studio, TrackVis, etc.)\n",
    "4. Run parcellation with QSIParc (see `04_qsiparc_basics.ipynb`)\n",
    "5. Perform network analysis on connectivity matrices\n",
    "\n",
    "## Tips\n",
    "\n",
    "- **Reconstruction specs**: Choose based on your acquisition (single-shell vs multi-shell)\n",
    "- **Resources**: Tractography is CPU-intensive - allocate sufficient cores\n",
    "- **Memory**: Some atlases require significant memory (64GB+ for high-resolution)\n",
    "- **Validation**: Always review the HTML report for quality control\n",
    "- **Multiple specs**: You can run different models on the same preprocessed data\n",
    "- **Connectivity matrices**: Different atlases produce different sized matrices\n",
    "- **Version pinning**: Use specific Docker versions for reproducibility"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "VoxelOps",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}