---
title: ANTsPyMM processing summary and data dictionary
author: Brian B. Avants and Nicholas J. Tustison
output: html_document
---

```{r libs,echo=FALSE,include=FALSE}
dev.flush()
library("permute")
library(MGMM) # GMM methods (FIXME)
library(WeightedCluster) # cluster quality (FIXME)
library( kmed ) # silhouette plots and such
library( clustree ) # cluster validation tools
library( factoextra )
library(flexclust)
library(tidyverse)
library(magrittr)
library(cluster)
library(cluster.datasets)
library(cowplot)
library(NbClust)
library(clValid)
library(ggfortify)
library(clustree)
library(dendextend)
library(factoextra)
library(FactoMineR)
library(corrplot)
library(GGally)
library(ANTsR)
library(subtyper)
library(plyr)
library(data.table)
library(ztable)
library(moonBook)
library(kableExtra)
library(flextable)
library(plotly)
library( ztable )
library(ggpubr)
library(gridExtra)
library(ggsci)
options(digits=3)
library(lmerTest)
library(lme4)
library(mlr3verse)
library(Evacluster)
options( ztable.type="html" )
tblcmd = ztable
```

## [ANTsPyMM](https://github.com/stnava/ANTsPyMM) imaging-derived phenotypes and associated coordinate systems / neuroanatomy

A quick overview of the labels used/generated by this framework: ![is here](https://i.imgur.com/qKqYjU9.jpeg).  

An example annotated output tree is [here](https://github.com/ANTsX/ANTsPyMM/blob/main/docs/antspymm_annotated_output_tree.txt).  This shows the results of a real run of ANTsPyMM on multiple modality  MRI (M3RI).


The `ANTsPyMM` system takes advantage of templates, core T1 processing and super-resolution described in [this paper](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8079440/).  ![ANTsX ecosystem](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/8079440/bin/41598_2021_87564_Fig1_HTML.jpg)

* We quantify cortex with the Desikan-Killiany-Tourville parcellation ([DKT](https://doi.org/10.3389/fnins.2012.00171)).   ![DKT atlas](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/3514540/bin/fnins-06-00171-g001.jpg)

    * `*dktregions` refers to the segmentation label directly from the deep learning method.

    * `*dktcortex` refers to the segmentation label restricted to the cortical segmentation that is provided by the tissue segmentation method.

* Subcortical segmentation with deep learning and also SyN registration leverages the CIT168 atlas [10.1101/211201](https://doi.org/10.1101/211201).  ![Pauli CIT168](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/5903366/bin/sdata201863-f2.jpg)

* A medial temporal lobe parcellation is based on manual labels derived from M. Yassa's research group ([preprint](https://doi.org/10.1101/2023.01.17.23284693)) and [described here](https://pubmed.ncbi.nlm.nih.gov/36586358/). ![Yassa MTL](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/9830310/bin/gr1.jpg)

* A cerebellum parcellation based on a template from the [CoBrALab](https://www.cobralab.ca/cerebellum-lobules) with additional manual editing by the Tustison family (publication in progress).  The parcellation scheme is defined using the Schmahmann nomenclature ([reviewed here](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6271471/)).  The core tool is named `cerebellum_morphology` and based on a concatentation of two U-nets.  ![Schmahmann parcellation](https://github.com/stnava/ANTsPyMM/blob/main/docs/nihms-1510954-f0146.jpg?raw=true)

* Brain stem is subdivided into the mid-brain, pons and medulla; This segmentation derives from registration between the individual T1w and the manually labeled CIT168 template.  The protocol is based on [this](https://pubmed.ncbi.nlm.nih.gov/25776214/) ![protocol](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/4434226/bin/nihms678808f2.jpg) used in Freesurfer.

* We segment anatomy related to the basal forebrain and associated cholinergic neurons with a deep learning method (`deepNBM`) derived from manual labeling of ADNI data.  

    * This approach is based on anatomical landmarks proposed in [Liu, A. et al. (2015)](https://doi.org/10.1007/s00401-015-1392-5).  The `deepNBM` method uses super-resolution segmentation to take advantage of manual labels performed on super-resolution training data.

    * The classic [Zaborszky, L. et al. (2008) "Stereotaxic probabilistic maps of the magnocellular cell groups in human basal forebrain." Neuroimage 42: 1127--1141](https://www.sciencedirect.com/science/article/abs/pii/S1053811908006903) also informed the manual labeling.

    * Presented at Human Brain Mapping conference 2022. ![deep NBM](https://github.com/stnava/ANTsPyMM/blob/main/docs/deepnbm.jpg?raw=true)

* The Mori JHU white matter atlas provides a parcellation of white matter regions ([paper here](https://www.sciencedirect.com/science/article/abs/pii/S105381190700688X?via%3Dihub)) which are applied to diffusion weighted images.   ![Mori atlas](https://www.ncbi.nlm.nih.gov/pmc/articles/instance/2883814/bin/nihms207341f5.jpg)

* Jonathan Power's coordinates from "Functional Network Organization of the Human Brain" [10.1016/j.neuron.2011.09.006](10.1016/j.neuron.2011.09.006) were previously used to guide our analysis of resting state function MRI (rsfMRI).  We also based our processing evaluation on these data.  However, we now use the homotopic coordinates described below.

* Yeo's [Homotopic local-global parcellation of the human cerebral cortex from resting-state functional connectivity](https://pubmed.ncbi.nlm.nih.gov/36918136/) is the new default that we use for rsfMRI instead of the JP coordinates.  Additional details about the parcellation are [here](https://github.com/ThomasYeoLab/CBIG/tree/master/stable_projects/brain_parcellation/Yan2023_homotopic).  We use the variant with 500 homotopic parcels i.e. 250 per hemisphere.

    * these parcels are used to generate inter and intra network correlations using the following networks (which are further subdivided s.t. 17 network names are available)

        * Visual Network: Processes visual information, including spatial and motion processing.

        * Somatomotor Network: Involved in the control of movement and processing of somatosensory information.

        * Dorsal Attention Network: Plays a key role in top-down attentional control and spatial orientation.

        * Ventral Attention Network: Involved in detecting unexpected stimuli and bottom-up attention.

        * Limbic Network: Associated with emotion processing, memory, and autonomic functions.

        * Frontoparietal Network: Important for cognitive functions including working memory and executive control.

        * Default Mode Network (DMN): Active during rest and involved in self-referential thought, memory retrieval, and planning for the future.

    * The 17 Network abbreviations are adapted from [Ru Kong's work](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6519695/).  ![Network names](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6519695/bin/bhy123f02.jpg)

    * rsfMRI phenotype variables like "*fcnxpro129*" indicate that parameter set 129 was used;  currently 122 is considered "best" but 129 and 134 are also provided.  they differ in the censoring and nuisance variables used in the processing.

    * we also provide voxel-wise measurements within each parcel including: 
        
        * fALFF and ALFF each divided by the global mean of the brain (mfALFF and mALFF)

        * [percent absolute fluctuation](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0227021) divided by the global PerAF mean of the brain (mPerAF)


## [ANTsPyMM](https://github.com/stnava/ANTsPyMM) data dictionary

The `ANTsPyMM` data dictionary labels associates each column name with each of the above anatomical references.  Table 1 below summarizes the number of variables associated with each modality and anatomical prior space.  DTI-derived connectivity yields the largest number of variables because we pairwise map each DKT cortical region as well as the primary CIT168 regions.


### Table 1
```{r t1,echo=FALSE,results='asis',warning=FALSE,message=FALSE}
zz=read.csv("docs/antspymm_data_dictionary.csv")
myt=mytable( Modality~. ,data=zz )
tblcmd( myt )
```

```{r detailsandcitations,echo=FALSE,eval=TRUE,warning=FALSE,message=FALSE,cache=FALSE}
# yeo homotopic https://pubmed.ncbi.nlm.nih.gov/36918136/
yeo=read.csv("~/.antspymm/ppmi_template_500Parcels_Yeo2011_17Networks_2023_homotopic.csv")
yeoanat=parcel_data <- data.frame(
  Abbreviation = c("Cingm", "ExStr", "ExStrInf", "ExStrSup", "FPole", "FrMed", "FrOper", "Ins", "IPL", "IPS", 
                   "OFC", "ParMed",  "PCC", "pCun", "pCunPCC", "PFCd", "PFCl", "PFCm", "PFCv", "PHC", "PostC", 
                   "PrC", "PrCd", "PrCv", "RSC", "SPL", "ST", "Striate", "Temp", "TempOcc", "TempPole", "ParOper"),
  full_parcel_name = c("mid-cingulate", "extrastriate cortex", "extra-striate inferior", "extra-striate superior", 
                          "frontal pole", "frontal medial", "frontal operculum", "insula", "inferior parietal lobule", 
                          "intraparietal sulcus", "orbital frontal cortex", "parietal medial", "posterior cingulate cortex", 
                          "precuneus", "precuneus posterior cingulate cortex", "dorsal prefrontal cortex", 
                          "lateral prefrontal cortex", "medial prefrontal cortex", "ventral prefrontal cortex", 
                          "parahippocampal cortex", "post central", "precentral", "precentral dorsal", 
                          "precentral ventral", "retrosplenial cortex", "superior parietal lobule", "superior temporal", 
                          "striate cortex", "temporal", "temporal occipital", "temporal pole", "parietal operculum")
)

yeo$anat=yeo$SystemName
for ( k in 1:nrow(yeo) ) {
    subname =  gsub("_right", "", gsub("_left", "",yeo$AAL[k])) 
    losel = yeoanat$Abbreviation == subname
    if (sum(losel)==1) {
        yeo$anat[k] = yeoanat$full_parcel_name[ losel ]
    } else yeo$anat[k]="somatosensory cortex"
}


# powers points 10.1016/j.conb.2012.12.009
powers=read.csv("~/.antspymm/powers_mni_itk.csv")
# 10.1016/j.neuroimage.2008.07.009
# https://doi.org/10.1016/j.neuroimage.2007.07.053
jhu=read.csv("~/.antspyt1w/FA_JHU_labels_edited.csv")
# doi: 10.3389/fnins.2012.00171
# http://dx.doi.org/10.1016/j.neuroimage.2006.01.021
dktcsv=read.csv("~/.antspyt1w/dkt.csv")
dktcsv=dktcsv[dktcsv$Label>0,]
cnxcsv=read.csv("~/.antspyt1w/dkt_cortex_cit_deep_brain.csv")
cnxcsv=cnxcsv[cnxcsv$Label>0,]
# hipp https://doi.org/10.1101/2023.01.17.23284693
hipp=read.csv("~/.antspyt1w/mtl_description.csv")
hipp$Anatomy=hipp$Description
hipp$Anatomy=gsub("alEC"," antero-lateral entorhinal cortex",hipp$Anatomy)
hipp$Anatomy=gsub("pMEC"," postero-medial entorhinal cortex",hipp$Anatomy)
hipp$Anatomy=gsub("DG"," dentate gyrus",hipp$Anatomy)
hipp$Anatomy=gsub("CA"," cornu ammonis", hipp$Anatomy)

# https://doi.org/10.1101/211201
cit=read.csv("~/.antspyt1w/CIT168_Reinf_Learn_v1_label_descriptions_pad.csv")
cit$Anatomy=NA
cit$Anatomy[  grep("STR_Ca", cit$Description )] = 'caudate'
cit$Anatomy[  grep("STR_Pu", cit$Description )] = 'putamen'
cit$Anatomy[  grep("STR_NAC", cit$Description )] = 'Nucleus Accumbens'
cit$Anatomy[  grep("VTA", cit$Description )] = 'Ventral Tegmental Area'
cit$Anatomy[  grep("PBP", cit$Description )] = 'Parabrachial Pigmented Nucleus'
cit$Anatomy[  grep("SNc", cit$Description )] = 'Substantia Nigra pars compacta'
cit$Anatomy[  grep("SNr", cit$Description )] = 'Substantia Nigra pars reticulated'
cit$Anatomy[  grep("GPe", cit$Description )] = 'globus pallidus externa'
cit$Anatomy[  grep("GPi", cit$Description )] = 'globus pallidus interna'
cit$Anatomy[  grep("RN", cit$Description )] = 'red nucleus'
cit$Anatomy[  grep("STH", cit$Description )] = 'Subthalamic Nucleus'
cit$Anatomy[  grep("HTH", cit$Description )] = 'Hypothalamus'
cit$Anatomy[  grep("HN", cit$Description )] = 'Habenular Nuclei'
cit$Anatomy[  grep("EXA", cit$Description )] = 'extended amygdala'
cit$Anatomy[  grep("BNST", cit$Description )] = 'bed nuclei of the stria terminali'
cit$Anatomy[  grep("MN", cit$Description )] = 'mammillary nucleus'
cit$Anatomy[  grep("SLEA", cit$Description )] = 'sublenticular extended amygdala'
cit$Anatomy[  grep("VeP", cit$Description )] = 'ventral pallidum'

interpretcnx<-function( x ) {
    breaker=gsub("DTI_cnxcount","",x)
    temp = unlist(strsplit(breaker,"_"))
    ind=temp[1]
    anat=paste( temp[-1],collapse='_')
    return( paste( anat, "to", cnxcsv[as.integer(ind)+1,'Description'] ) )
}
interpretcnx2<-function( x ) {
    breaker=gsub("DTI_cnxcount","",x)
    temp = unlist(strsplit(breaker,"_"))
    ind=temp[1]
    anat=paste( temp[-1],collapse='_')
    return( dktcsv[as.integer(ind),'Description'] )
}

if ( ! exists("dd") ) {
    dfn='./docs/example_antspymm_output.csv'
    dd=read.csv( dfn )
    dd = dd[ , -grep("fcnxpro134", colnames(dd))]
    dd = dd[ , -grep("fcnxpro129", colnames(dd))]
    ddnms=names(dd)
}

gg=grep("mmwide_filename",zz$Label)
zz[ gg,'Description']="unique NRG format filename identifier"
zz[ gg,'Measurement']=NA
zz[ gg,'Atlas']=NA
zz[ gg,'side']=NA
zz[ zz$Label == 'T1wHierarchical_mmwide_filename','Modality']='T1 hierarchical processing'
zz[ zz$Label == 'T1w_mmwide_filename','Modality']='T1 DiReCT thickness processing'
zz[ zz$Label == 'rsfMRI_mmwide_filename','Modality']='restingStatefMRI'
zz[ zz$Label == 'DTI_mmwide_filename','Modality']='DTI'
zz[ zz$Label == 'T2Flair_mmwide_filename','Modality']='Flair'
zz[ zz$Label == 'NM2DMT_mmwide_filename','Modality']='Neuromelanin'
zz[ grep("SNR",zz$Label),'Measurement']='QC'
zz[ grep("fn",zz$Label),'Measurement']='original source data filename'
zz[ grep("noise",zz$Label),'Measurement']='QC'
zz[ grep("snr",zz$Label),'Measurement']='QC'
zz[ grep("cnr",zz$Label),'Measurement']='QC'
zz[ grep("X",zz$Label),'Measurement']='ignore'
zz[ grep("ssim",zz$Label),'Measurement']='QC'
zz[ grep("loop",zz$Label),'Measurement']='QC'
zz[ grep("high_motion",zz$Label),'Measurement']='QC'
zz[ grep("lof",zz$Label),'Measurement']='QC'
zz[ grep("nmid",zz$Label),'Measurement']='NM unique filename'
zz[ grep("_dvars",zz$Label),'Measurement']='QC'
zz[ grep("_evr",zz$Label),'Measurement']='QC'
zz[ grep("slice",zz$Label),'Measurement']='slice number in time series (ignore)'
zz[ grep("subjectID",zz$Label),'Measurement']='unique subject ID'
zz[ zz$Label=='NM2DMT_NM_substantianigra_z_coordinate','Measurement']='Estimate of the normalize (zero to one) z-coordinate of the substantia nigra in a neuromelanin scan; higher values mean higher in the slab'
zz[ zz$Label=='sid','Measurement']='unique subject ID'
zz[ grep("mrimfg",zz$Label),'Measurement']='MRI manufacturer (often missing)'
zz[ grep("mrimodel",zz$Label),'Measurement']='MRI model (often missing)'
zz[ grep("mriMagneticFieldStrength",zz$Label),'Measurement']='MRI field strength (often missing)'
zz[ grep("mriSAR",zz$Label),'Measurement']='MRI Specific Absorption Rate is a measure of the rate at which energy is absorbed by the body when exposed to a radiofrequency (RF) electromagnetic field (often missing)'
zz[ grep("mriPixelBandwidth",zz$Label),'Measurement']='MRI range of frequencies encoded in each pixel of an image (often missing)'
zz[ grep("mriPixelBandwidthPE",zz$Label),'Measurement']='MRI Pixel bandwidth in the phase encoding (PE) direction (often missing)'
zz[ grep("dti_failed",zz$Label),'Measurement']='heuristic decision about dti usability (eg its 3D)'
zz[ grep("rsf_failed",zz$Label),'Measurement']='heuristic decision about rsfMRI usability (eg its 3D)'
zz[ grep("dti_bvalueMax",zz$Label),'Measurement']='MRI DTI maximum bvalue (often missing)'
zz[ grep("dti_bvecnorm",zz$Label),'Measurement']='MRI DTI bvec norm (usually close to 1, often missing)'
zz[ grep("T1w_mean_",zz$Label),'Measurement']='Regional average of DiReCT estimated thickness'
zz[ grep("NM2DMT_mean",zz$Label),'Measurement']='Neuromelanin signal summary'
zz[ grep("NM2DMT_NM_avg",zz$Label),'Measurement']='Neuromelanin signal summary'
zz[ grep("NM2DMT_NM_std",zz$Label),'Measurement']='Neuromelanin signal summary'
zz[ grep("NM2DMT_NM_count",zz$Label),'Measurement']='count of neuromelanin scans that were averaged at this date'
zz[ grep("NM2DMT_NM_m",zz$Label),'Measurement']='Neuromelanin signal summary'
zz[ grep("NM2DMT_NM_sd",zz$Label),'Measurement']='Neuromelanin signal summary'
zz[ zz$Label %in% c("flairid",'nmid1','nmid2','nmid3','nmid4','nmid5','nmid6','nmid7','nmid8','nmid9','nmid10','nmid11','rsfid1','rsfid2','dtid1','dtid2'),'Measurement']='unique image id for modalities or their components (e.g. LR RL acquisitions for DTI or rsfMRI)'
zz[ zz$Label == 'T2Flair_flair_wmh','Measurement']='project ID or name eg ADNI'
zz[ zz$Label == 'projectID','Measurement']='project ID or name eg ADNI'
zz[ zz$Label == 'modality','Measurement']='generic modality name'
zz[ zz$Label == 'negol','Measurement']='negative outlier variable (higher is better)'
zz[ zz$Label == 'negative_outlier_factor','Measurement']='negative outlier variable (higher is better)'
zz[ zz$Label == 'mmimageuid','Measurement']='unique id for the T1w used in ANTsPyMM same as t1imageuid'
zz[ zz$Label == 't1imageuid','Measurement']='unique id for the T1w used in ANTsPyMM should equal mmimageuid'
zz[ zz$Label == 'visitdate','Measurement']='image acquisition date'
zz[ zz$Label == 'date','Measurement']='image acquisition date'
zz[ zz$Label == 'subjectIDdate','Measurement']='concatenation of subject ID and date'
zz[ zz$Label == 'imageID','Measurement']='unique image ID (ideally unique but not always)'
zz[ grep("brainstem",zz$Label),'Modality']='T1 hierarchical processing'
zz[ grep("brainstem",zz$Label),'side']=NA
zz[ grep("brainstem",zz$Label),'Atlas']='CIT168'
zz[ grep("brainstem",zz$Label),'Description']='brainstem subdivision'
zz[ grep("_ch13_",zz$Description),'Description']='basal nucleus ch1 to ch3'
zz[ grep("nbmant",zz$Description),'Description']='basal nucleus anterior ch4'
zz[ grep("nbmpos",zz$Description),'Description']='basal nucleus posterior ch4'
zz[ grep("nbmmid",zz$Description),'Description']='basal nucleus middle ch4'


cerenames = getNamesFromDataframe("cerebellum",dd,exclusions=c("dkt","tissu"))
for ( ccc in cerenames ) {
    zz[zz$Label == ccc,'Atlas']='TustisonCobra'
    zz[zz$Label == ccc,'Description']='cerebellum label using Schmahmann nomenclature'
    zz[zz$Label == ccc,'side']='left'
    if ( length(grep("_r_",ccc) == 1 ) ) zz[zz$Label == ccc,'side']='right'
}
zz[zz$Label %in% c("mi",'EVR','reflection_err','org0','org1','org2','spc0','spc1','spc2','dimz','dimy','dimx','dimt'),"Measurement"]='QC'
zz[zz$Label %in% c("EVR"),"Measurement"]='QC - eigenvalue ratio'
zz[zz$Label %in% c("mi",'reflection_err','ssim','psnr'),"Measurement"]='QC - similarity measurements'
zz[zz$Label %in% c('org0','org1','org2','spc0','spc1','spc2','dimz','dimy','dimx','dimt'),"Measurement"]='QC - physical space'
zz[zz$Label %in% c('snr','cnr','noise'),"Measurement"]='QC - noise/contrast'

zz[ subtyper::na2f(zz$Label == zz$Description),'Description']=NA
zz$Description=gsub( "fcnxpro122_" , "", zz$Description )
zz$Description=gsub( ".", " ", zz$Description, fixed=TRUE )
zz$Description=gsub( "_", " ", zz$Description, fixed=TRUE )

# deal with resting state
fixrsentry <- function( zz, older, newer ) {
    zz[ grep(older,zz[,1]),'Measurement']=newer
    zz[ grep(older,zz[,1]),'Description']=NA
    return( zz )
}
zz[ grep("paramset",zz[,1]),'Measurement']='parameter id'
zz[ grep("paramset",zz[,1]),'Description']=NA
zz[ grep("bandpass",zz[,1]),'Measurement']='band pass frequency limit'
zz[ grep("bandpass",zz[,1]),'Description']=NA
zz[ intersect( grep( "rsfMRI_", zz[,1] ), 
    grep("left", zz[,1]) ), "side"]='left'
zz[ intersect( grep( "rsfMRI_", zz[,1] ), 
    grep("right", zz[,1]) ), "side"]='right'
zz[ intersect( grep( "rsfMRI_", zz[,1] ), 
    grep("tractography-based connectivity", zz$Measurement) ), "Measurement"]='network connectivity'
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_minutes_original_data', 'minutes of rs imaging'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_minutes_censored_data', 'minutes of rs imaging after censoring'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_nc_csf', 'number of csf compcor components'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_nc_wm', 'number of white matter compcor components'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_n_outliers', 'number of outlier volumes'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_bold_evr', 'eigenvector ratio for mean bold'  )
zz = fixrsentry( zz, 'FD_mean', 'mean framewise displacement'  )
zz = fixrsentry( zz, 'FD_max', 'max framewise displacement'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_despiking_count_summary', 'changes made in despiking'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_high_motion_pct', 'percent high motion count'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_high_motion_count', 'high motion count'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_FD_threshold', 'FD threshold paramter'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_outlier_threshold', 'outlier threshold parameter'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_spatial_smoothing', 'spatial smoothing parameter'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_censor', 'ignore'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_dvars_mean', 'mean DVARS'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_ssnr_mean', 'mean spatial SNR in mean bold'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_tsnr_mean', 'mean TSNR in time series'  )
zz = fixrsentry( zz, 'rsfMRI_fcnxpro122_upsampling', 'upsampling parameter'  )
wrsfmri = intersect( grep( "rsfMRI", zz[,1] ), which( !is.na( zz$Description )) )
for ( k in wrsfmri ) {
    ss = unlist( strsplit( zz$Description[k], '_' ))
    ss = ss[length(ss)]
    iscnx = length(grep("_2_", zz$Description[k] )) > 0
    if ( substr(zz[k,1],nchar(zz[k,1])-4, nchar(zz[k,1])) == '_mean') {
        zz$Description[k]=NA
        zz$Measurement[k]="global mean of image"
    } else if ( substr(zz[k,1],nchar(zz[k,1])-2, nchar(zz[k,1])) == '_sd') {
        zz$Description[k]=NA
        zz$Measurement[k]="global sd of image"
    } else if ( !iscnx & !(ss %in% yeoanat$full_parcel_name) ) {
        wanat = which( yeoanat$Abbreviation == ss )
        if ( length( wanat ) > 0 ) {
            zz$Description[k]=yeoanat$full_parcel_name[wanat]
        } else {
#            zz$Description[k]="unlabeled / likely somatosensory"
#            print( zz[k,])
        }
    }
}
#######

zz$tidynames=shorten_pymm_names( zz$Label )
write.csv( zz, "./docs/antspymm_data_dictionary_2.csv", row.names=FALSE)
######
```

## Quality control metrics

Automated (blind) QC is part of standard ANTsPyMM processing.  QC metrics 
are therefore present in the standard output tables.  Notes on the meaning of these 
metrics, along with examples of the metrics on images, are [here](https://htmlpreview.github.io/?https://github.com/stnava/ANTsPyMM/blob/main/docs/blind_qc.html).

The most important QC metric is `T1Hier_resnetGrade` which rejects truly bad images.  We recommend rejecting images with `T1Hier_resnetGrade < 1.01` where higher thresholds may be useful in some cases.  This grading function uses deep learning to generate a pseudocontinuous measurement of aggregate T1w image quality.  Several factors will contribute to these scores and it is not a perfect approach but it is well-defined and evaluated based on manual ratings [here](https://doi.org/10.1101/2023.02.02.23285376).

The `outlierness` function produces additional blind QC measurements via automated outlierness calculations.  These measurements are denoted by Local Outlier Probability (LOOP `*loop*` ) and local outlier factor ( LOF `*lof*` ) column names.

Time series images may have additional QC metrics such as:

* TSNR: temporal signal to noise ratio

* SSNR: spatial signal to noise ratio on the mean image (different from TSNR)

* DVARS: the spatial root mean square of the data after temporal differencing

* Neuromelanin (NM) has its own set of variables including:

    * count: the number of acquisitions at that date 

    * min/max/mean/sd: a statistic on the raw image signal after averaging the acquisitions

    * `avg/std_refregion` : average and standard deviation of signal in the reference region

    * `avg/std_substantianigra` : average and standard deviation of signal in the substantia nigra or a related CIT168 region (determined by deepCIT168)

    * `NM2DMT_NM_substantianigra_z_coordinate`: estimate of the normalized (zero to one) z-coordinate of the substantia nigra in a neuromelanin scan; higher values mean higher in the slab.

* the framewise displacement (FD) values are 10x in DTI data compared to rsfMRI data. E.g. if 0.3 were a reasonable high motion threshold in rsfMRI; then 3.0 would be a comparable threshold for DTI.

## the full data dictionary 

A sortable version of the data dictionary is below (if viewing on local machine).

### Table 2

```{r printdict,echo=FALSE,eval=TRUE,warning=FALSE,message=FALSE}
reactable::reactable( zz , highlight = TRUE, bordered = TRUE, striped = TRUE, compact = FALSE, defaultPageSize = 12000, static = TRUE )
```



```{r checking,echo=FALSE,eval=TRUE,warning=FALSE,message=FALSE}
# find the missing names
good=ddnms[  (ddnms %in% zz$Label) ]
bad=ddnms[ ! (ddnms %in% zz$Label) ]
# stopifnot(length(bad)==0)
bad2=zz[ is.na(zz$Measurement),'Label']
# print( bad2 )
# print( length( bad2 ) )
```


## Joining with clinical data: PPMI

We provide an example of how to join derived `ANTsPyMM` summary data with clinical variables from PPMI. 

```{r joiner,eval=FALSE,echo=TRUE,warning=FALSE,message=FALSE}
library( subtyper)
?merge_ppmi_imaging_clinical_demographic_data
```
