.. image:: ./ikiss/logo_ikiss.png
   :width: 400
   :alt: ikiss Logo
   :align: center


|PythonVersions| |SnakemakeVersions| |Singularity|

.. contents:: Table of Contents
    :depth: 2


**Homepage:** https://forge.ird.fr/diade/iKISS

About iKISS 
===============

**iKISS (Kmer Inference sSelection)** is a snakemake pipeline able to decompose reads into kmers and extract kmers under selection. 

iKISS uses KmersGWAS https://github.com/voichek/kmersGWAS, pcadapt https://cran.r-project.org/web/packages/pcadapt/readme/README.html and lfmm https://bcm-uga.github.io/lfmm/articles/lfmm to select genomics regions under selection.

1. Install dependencies and clone iKISS
=============================================

Check dependencies for iKISS : python and singularity

Install  singularity and python3 in your local machine OR use module load to add singularity and python3 in your environment if you are working in a cluster :

.. code-block:: bash

   module load system/python/3.8.12
   module load system/singularity/3.6.0


iKISS is NOW available as a PyPI package (recommended)

.. code-block:: bash

   python3 -m pip install ikiss


OR you can also install iKISS from git repository

.. code-block:: bash

   python3 -m pip install ikiss@git+https://forge.ird.fr/diade/iKISS.git 
   
   #OR
   
   git clone https://forge.ird.fr/diade/iKISS.git 
   cd iKISS
   python3 -m pip install .


1.1 Installing in cluster mode
-------------------------------

Install iKISS in cluster mode using **singularity** container from ikiss_utilities https://itrop.ird.fr/ikiss_utilities/

.. code-block:: bash

   ikiss install_cluster --help
   ikiss install_cluster --scheduler slurm --env singularity
   

1.2 Installing in local mode 
----------------------------

.. code-block:: bash

   ikiss install_local --help
   ikiss install_local


2. Running a datatest
=============================================

Running test with a datatest from iKISS_utilities in a repertory TEST

.. code-block:: bash

   ikiss test_install --help
   ikiss test_install -d TEST


2.1 In CLUSTER mode
-------------------

Launching suggested command line done by iKISS, in CLUSTER mode : 

Please run command line 'ikiss create_cluster_config' before the first run and modify theads, ram, node and computer ressources. 
iKISS do a copy of cluster_config.yaml file into your home "/home/$USER/.config/ikiss/cluster_config.yaml"

   
.. code-block:: bash

   ikiss run_cluster --help
   ikiss create_cluster_config

If singularity was selected in installation of iKISS, it could be needed to give argument --singularity-args \"--bind $HOME\" to Snakemake, by using :

.. code-block:: bash

   ikiss run_cluster --help
   ikiss run_cluster -c TEST/data_test_config.yaml --singularity-args "--bind $HOME"
   # @IFB
   ikiss run_cluster -c TEST/data_test_config.yaml --singularity-args "--bind /shared:/shared"
   #you can also use snakemake parametters as --rerun-incomplete --nolock


**Important Note** : In i-Trop cluster, run iKISS using ONLY a node, data has to be in "/scratch" of chosen node. Use `nodelist : nodeX` parametter inside of cluster_config.yaml file.


2.2 In LOCAL mode
-----------------

launching suggested command line done by iKISS, in LOCAL mode: 

.. code-block:: bash

   ikiss run_local --help
   ikiss run_local -t 8 -c TEST/data_test_config.yaml --singularity-args "--bind $HOME"

In local mode, its possible to allocate threads to some rules using `--set-threads` snakemake argument such as

.. code-block:: bash

    ikiss run_local -t 8 -c TEST/data_test_config.yaml --set-threads kmers_gwas_per_sample=4 mapping_kmers=2 filter_bam=2 kmer_position_from_bam=4 pcadapt=2 extract_kmers_from_bed=2


3. Running your data
========================


3.1. Adapt config.yaml
------------------------

Before to run iKISS, adapt `config.yaml` by using : 

.. code-block:: bash

   ikiss create_config


Adapt `config.yaml` file with path to fastq files (FASTQ) and outfile (OUTPUT) in the `DATA` section. 

.. code-block:: yaml

   DATA:
      FASTQ: './DATATEST/fastq'
      OUTPUT: './OUTPUT-KISS/'

:warning if yours reads are ilumina paired, you need rename reads SAMPLE_R1.fastq.gz and SAMPLE_R2.fastq.gz. For single reads use SAMPLE_R1.fastq.gz

iKISS uses compressed ans decompressed fastq files.


3.1.1 WORKFLOW section
-----------------------

Parameter iKISS steps using the section WORKFLOW and parameter it with the PARAMS sections.

In WORKFLOW section:

   KMERS_GWAS step has to be activated by default. 

   PCADAPT, LFMM, MAPPING or ASSEMBLY are optional. Active or deactivate these steps using true or false.


**KMERS_GWAS** convert reads in kmers, filter them and create a format ready to use in population genomics!

**PCADAPT** detects genetic markers (kmers here ^^) involved in biological adaptation and provides outlier detection based on Principal Component Analysis (PCA).

**LFMM** is used by iKISS for testing correlations between kmers and environmental data.

**MAPPING** can optionally be used to align kmers to a genomic reference (if it is available ! ).

**ASSEMBLY** can optionally assembly significant kmers obtained by pcadapt or lfmm

.. code-block:: yaml

   WORKFLOW:
      KMERS_MODULE : true
      PCADAPT : true
      LFMM : true
      MAPPING: true
      ASSEMBLY: true

3.1.2 PARAMS section
--------------------

In the PARAMS section, tools parameters can be modified and adapted.


=> 1. KMERS_MODULE
-------------------

KMERS_GWAS module decompose reads into kmers and create a binary table of presence/absence of kmers. This table can be filter to use only most informative kmers into the populations. PLINK format outfiles are obtained in this module.

.. code-block:: yaml

   PARAMS:
      KMERS_MODULE:
         KMER_SIZE : 31
         MAC : 2
         P : 0.2
         MAF : 0.05
         B : 1000000 # nb kmers in each bed file
         SPLIT_LIST_SIZE : 100000
         MIN_LIST_SIZE : 50000


**KMER_SIZE** is the length of kmers (should be between 15-31)

**MAC** is the minor allele count (min allowed appearance of a kmer) 

**P** is the minimum percent of appearance in each strand form

**MAF** is the minimum allele frequency

**B** is the number of kmers in each bed file

**SPLIT_LIST_SIZE** is the nb of kmers by bed file

**MIN_LIST_SIZE** indicates the minimal number of kmers allowed in the smaller bed file after splitting


=> 2. PCADAPT
--------------

PCADAPT detects kmers involved in biological adaptation and provides outlier detection based on Principal Component Analysis (PCA)

.. code-block:: yaml

   PARAMS:        
      PCADAPT:
         K : 2
         SAMPLES: "samples.txt"
         CORRECTION: 'FDR'
         ALPHA : 0.05


**K** : number K of principal components

**SAMPLES** : you need to generate a *samples.txt* file.  This file contains two columns (tab delimitations) : accession_id and phenotype_value. It will be used by PCADAPT.

   **accession_id** : contains exactly same name of samples in FASTQ. 

   **phenotype_value** (int): contains sample group (wild=1, cultivated=2 for example)

.. code-block:: bash

   accession_id	group
   Clone12	2
   Clone14	2
   Clone16	2
   Clone20	2
   Clone2	1
   Clone4	1
   Clone8	1

**CORRECTION**: kmers outliers are obtained using a correction of BONFERONNI, BH or FDR model.

**ALPHA**: modify the alpha cutoff for outlier detection


=> 3. LFMM
----------

LFMM is used by iKISS for testing correlations between kmers and environmental data.

.. code-block:: yaml

   PARAMS:
      LFMM:
         K : 2
         PHENOTYPE_FILE: "pheno.txt"
         PHENOTYPE_PCA_ANALYSIS : false
         CORRECTION: 'BH'
         ALPHA : 0.05


**K** are the latent factors used in LFMM association analyses 

**PHENOTYPE_FILE**: an phenotype file is obligatory in LFMM analysis. You can give to iKISS PCA results, climate variables, etc.

A PCA can reveal some 'structure' in the genotype data and it could help you to fix K parameter.

**PHENOTYPE_PCA_ANALYSIS** 

   * If **PHENOTYPE_PCA_ANALYSIS** is true, iKISS automatically run PCA using the file given by user in the PHENOTYPE_FILE key. This PHENOTYPE_FILE can be a PCA result for example.

   * If **PHENOTYPE_PCA_ANALYSIS** is false, iKISS use directly the PHENOTYPE_FILE as 'phenotype' to LFMM analysis. Kmers are used as 'genotype' data.

Here, a example of a phenotype file with climate variables

.. code-block:: bash

    accession_id	group	b2.Mean_Diurnal_Range	b3.Isothermality	b4.Temp_Seasonality	b5.Max_Temp_of_Warmest_Month	b6.Min_Temp_of_Coldest_Month	b7.Temp_Annual_Range	b8.Mean_Temp_of
    _Wettest_Quarter	b9.Mean_Temp_of_Driest_Quarter	b10.Mean_Temp_of_Warmest_Quarter	b11.Mean_Temp_of_Coldest_Quarter	b12.Annual_Precipitation	b13.Precipitation_of_Wettest_Mo
    nth	b14.Precipitation_of_Driest_Month	b15.Precipitation_Seasonality	b16.Precipitation_of_Wettest_Quarter	b17.Precipitation_of_Driest_Quarter	b18.Precipitation_of_Warmest_Quarter	b19.Precipitation_of_Coldest_Quarter
    Clone12	2	99	68	1230	310	166	144	250	226	258	226	1462	249	3	68	573	17	549	17
    Clone14	2	100	68	1235	301	155	146	241	217	248	217	1525	259	3	67	603	18	575	18
    Clone16	2	93	65	1389	310	168	142	250	223	258	223	1416	264	0	73	579	8	544	8
    Clone20	2	154	55	3955	403	123	280	296	234	315	214	118	62	0	184	107	0	45	0
    Clone2	1	152	55	3617	403	128	275	287	242	316	220	173	80	0	167	153	0	18	0
    Clone4	1	168	51	5719	414	86	328	315	201	322	181	20	12	0	166	18	0	17	0
    Clone8	1	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA	NA


**CORRECTION**: kmers outliers are obtained using a correction of BONFERONNI, BH or FDR model.

**ALPHA**: modify the alpha cutoff for outlier detection


=> 4. MAPPING
-------------

MAPPING section in PARAMS can optionally be used to align kmers to a genomic reference. It could give a idea of selected regions in a genome. 

.. code-block:: yaml

   PARAMS:
      MAPPING:
         REF: "reference.fasta"
         MODE : bwa-aln
         INDEX_OPTIONS: ""
         OPTIONS : "-n 0.04"
         FILTER_FLAG : 4
         FILTER_QUAL : 10


Use a reference file in the **REF** section. 

Parametter **MODE** using  *bwa-aln* or *bwa-mem2* 

Set up the **INDEX_OPTIONS** according to the MODE you have chosen.

   If *bwa-mem2* leaf empty
   
   If *bwa-aln* "-a bwtsw" or "" 

Set options according of chosen mapper in the **OPTIONS** key. 

   If *bwa-mem2* default parameters -A 1 -B 4;
   
   If *bwa-aln* -n 0.04

Obtained bam could be filtered using `FILTER_FLAG` (-F 4 by default) and `FILTER_QUAL` (mapq>10 by defaut) params.

=> 5. ASSEMBLY
---------------

ASSEMBLY section in PARAMS can optionally be used to assembly significant kmers obtained by pcadapt or/and lfmm.

Contigs are assembled by iKISS using  mergeTags from dekupl package https://github.com/Transipedia/dekupl-mergeTags.

Chose minimal overlap size "OVERLAP_SIZE" allowed to assembly kmers.

Feel free to filter contigs by size "FILTER_CONTIG_SIZE".

Assembled contigs could be used by blastn against a database, you can also try to annotate them!

.. code-block:: yaml

   PARAMS:
      ASSEMBLY:
         OVERLAP_SIZE : 15
         FILTER_CONTIG_SIZE : 100


3.2. Adapt cluster_config.yaml
-------------------------------


If you will run ikiss in cluster, adapt `cluster_config.yaml` :  

.. code-block:: bash

   ikiss edit_cluster_config

Inside `cluster_config.yaml`, adapt partition to your favorite cluster and change memory and cpu number in by `__default__` key or in rules you need :

.. code-block:: bash

   __default__:
      cpus-per-task : 4
      mem-per-cpu : 10G
      partition : "normal"
      nodelist: node19
      output : 'slurm_logs/stdout/{rule}/{wildcards}.o'
      error : 'slurm_logs/error/{rule}/{wildcards}.e'
      job-name : '{rule}.{wildcards}'
      
   kmers_gwas_per_sample:
      cpus-per-task : 4
      mem-per-cpu : 10G


RULES  
-----

Here you can quickly find iKISS snakemake rules list : 

.. code-block:: bash

   rule kmers_gwas_per_sample *
   rule kmers_to_use
   rule kmers_table
   rule extract_kmers_from_bed
   rule index_ref
   rule mapping_kmers
   rule filter_bam *
   rule kmer_position_from_bam *
   rule merge_kmer_position
   rule samtools_merge
   rule pcadapt * 
   rule merge_pcadapt
   rule outliers_pcadapt_position
   rule get_pca_from_phenotype
   rule lfmm * 
   rule merge_lfmm
   rule outliers_lfmm_position
   rule mergetags_lfmm
   rule mergetags_pcadapt
   rule report_ikiss
   rule html_ikiss

* rules with a `*` can be parallelised.


4. Running iKISS
================

Run iKISS by `ikiss run_local` or `ikiss run_cluster` as explained in "Running a datatest" section.



5. iKISS output
================

This is a overwiew of iKISS output directory:

.. code-block:: bash

      OUTPUT-KISS/
      ├── 1.KMERS_MODULE
      │   ├── Clone1
      │   │   ├── Clone1_files.txt
      │   │   ├── Clone1_kmc3_all.kmc_pre
      │   │   ├── Clone1_kmc3_all.kmc_suf
      │   │   ├── Clone1_kmc3_canon.kmc_pre
      │   │   ├── Clone1_kmc3_canon.kmc_suf
      │   │   ├── Clone1_kmers_with_strand
      │   ├── Clone2
      │   └── CloneX
      ├── 2.KMERS_TABLE
      │   ├── kmers_table.names
      │   ├── kmers_table.table
      │   ├── kmers_to_use
      │   ├── kmers_to_use.no_pass_kmers
      │   ├── kmers_to_use.shareness
      │   ├── kmers_to_use.stats.both
      │   ├── kmers_to_use.stats.only_canonical
      │   ├── kmers_to_use.stats.only_non_canonical
      ├── 3.TABLE2BED
      │   ├── output_file.0.bed
      │   ├── output_file.0.bim
      │   └── output_file.0.fam
      ├── 4.EXTRACT_FASTQ
      │   └── output_file.0.fastq.gz
      ├── 5.RANGES
      │   └── output_file.0
      │       ├── 1.txt
      │       ├── 2.txt
      │       ├── 3.txt
      │       ├── 4.txt
      │       └── 5.txt
      ├── 6.LFMM
      │   ├── output_file.0_1_lfmm_outliers.csv
      │   ├── output_file.0_1_lfmm.rplot.pdf
      │   ├── output_file.0_2_lfmm_outliers.csv
      │   ├── output_file.0_2_lfmm.rplot.pdf
      │   ├── output_file.0_3_lfmm_outliers.csv
      │   ├── output_file.0_3_lfmm.rplot.pdf
      │   ├── output_file.0_4_lfmm_outliers.csv
      │   ├── output_file.0_4_lfmm.rplot.pdf
      │   ├── output_file.0_5_lfmm_outliers.csv
      │   └── output_file.0_5_lfmm.rplot.pdf
      ├── 6.PCADAPT
      │   ├── output_file.0_1_BH0.05.pcadapt_outliers.csv
      │   ├── output_file.0_1_BH0.05.pcadapt_pvalues.csv
      │   ├── output_file.0_1_BH0.05.pcadapt.rplot.pdf
      │   ├── output_file.0_1_BH0.05.pcadapt_scores.csv 
      │   ├ ...   
      ├── 7.MERGED_LFMM
      │   └── merged_lfmm_pvalues.csv
      ├── 7.MERGED_PCADAPT
      │   ├── merged_pcadapt_outliers.csv
      │   └── merged_pcadapt_pvalues.csv
      ├── 8.MAPPING
      │   ├── output_file.0_vs_reference.bam
      │   ├── output_file.0_vs_reference_F4MQ10.bam
      │   ├── output_file.0_vs_reference_sorted.bam
      │   ├── output_file.0_vs_reference_sorted.bam.stats
      │   └── REF
      ├── 9.KMERPOSITION
      │   └── output_file.0_vs_reference_KMERPOSITION.txt
      ├── 10.MERGE_KMERPOSITION
      │   ├── kmer_position_merged.txt
      │   └── kmer_position_samtools_merge.bam      
      ├── 11.OUTLIERS_LFMM_POSITION
      │   └── outliers_with_position.csv
      ├── 11.OUTLIERS_PCADAPT_POSITION
      │   └── outliers_with_position.csv
      ├── 12.ASSEMBLY_OUTLIER_PCADAPT
      │   └── outliers_pcadapt_mergetags.fasta
      ├── 12.ASSEMBLY_OUTLIER_LFMM
      │   └── outliers_lfmm_mergetags.fasta
      ├── BENCHMARK
      ├── LOGS
      └── config_corrected.yaml

Note : we recommended to remove 1.KMER_GWAS repertory after analysis.

Authors
========

Julie Orjuela (IRD) develops iKISS

Yves Vigouroux (IRD) is the big boss with a lot of ideas and contributions! 

Contributeurs 
==============

Djamel Boubred (Bioinformatics Student at IRD) and Tram VI (Ph.D student IRD) have also contributed by debugging and test with rice and coffea datasets. 

Sebastien Ravel has also contributed with the snakecdysis python package developpement.

Thanks
=======

Thanks to Ndomassi Tando (i-Trop IRD) for his administration support.

The authors acknowledge the IRD i-Trop HPC (South Green Platform) from IRD Montpellier for providing HPC resources that contributed to this work. https://bioinfo.ird.fr/ - http://www.southgreen.fr
 
License
=======

Licensed under MIT.

Intellectual property belongs to IRD and authors.

iKISS uses recycled code from the culebrONT project of SouthGreen platform https://culebront-pipeline.readthedocs.io/en/latest/.
iKISS uses SnakEcdysis package https://snakecdysis.readthedocs.io/en/latest/package.html to perform installation and execution in local and cluster mode.

.. |PythonVersions| image:: https://img.shields.io/badge/python-3.7%2B-blue
   :target: https://www.python.org/downloads
.. |SnakemakeVersions| image:: https://img.shields.io/badge/snakemake-≥5.10.0-brightgreen.svg?style=flat
   :target: https://snakemake.readthedocs.io
.. |Singularity| image:: https://img.shields.io/badge/singularity-≥3.3.0-7E4C74.svg
   :target: https://sylabs.io/docs/
.. |readthedocs| image:: https://pbs.twimg.com/media/E5oBxcRXoAEBSp1.png
   :target: https://culebront-pipeline.readthedocs.io/en/latest/
   :width: 400px


