Purpose
-------

Quickly call lineages from ``.vcf`` files using different SNP schemes.
The package will be developed primarily to call *Mtb* (*Mycobacterium
tuberculosis*) lineages, but if you build your own SNP scheme you can
use it to call lineages for virtually any bacterial species.

Installation
------------

with pip
~~~~~~~~

::

   pip install fast-lineage-caller

with conda
~~~~~~~~~~

::

   conda install -c ejfresch fast-lineage-caller

Usage
-----

Basic usage
~~~~~~~~~~~

The simplest way to call the lineages is to provide a ``.vcf`` file to
``fast-lineage-caller``:

::

   ./bin/fast-lineage-caller genomic_data/SAMEA968141.vcf 

The program will output the lineage calls according to the available SNP
schemes

::

   Isolate coll2014        freschi2020     lipworth2019    shitikov2017
   SAMEA968141     lineage2.2.1    2.2.1.1.1       beijing lin2.2.1,asian_african_2 

Saving the output on a text file
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It is possible to save the lineage calls on a text (``.tsv``) file using
the ``--out`` option:

::

   fast-lineage-caller genomic_data/SAMEA968141.vcf --out results.tsv

Using a custom SNP scheme
~~~~~~~~~~~~~~~~~~~~~~~~~

It is possible to call the lineages using a custom SNP scheme defined by
the user:

::

   fast-lineage-caller genomic_data/SAMEA968141.vcf --scheme my_snp_schemes/freschi.tsv

No header, please
~~~~~~~~~~~~~~~~~

It is possible to tell the ``fast-lineage-caller`` to do not display the
header, *i.e.* the first line of the output that shows for instance the
names of the SNP schemes:

::

   fast-lineage-caller genomic_data/SAMEA968141.vcf --noheader
   SAMEA968141                     beijing asian_african_2,lin2.2.1

Sometimes this option is very useful, for instance when you have a lot
of ``vcf`` files (see below).

Counting the SNPs that support a given lineage call
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

SNP schemes can include one or multiple SNP that define one lineage /
sub-lineage. In order to get the conts of the number of SNPs that
support each lineage call, you can use the ``--count`` option:

::

   ./bin/fast-lineage-caller ~/mfarhat/rollingDB/genomic_data/SAMEA968141/pilon/SAMEA968141.vcf --count
   Isolate coll2014        freschi2020     lipworth2019    shitikov2017
   SAMEA968141     lineage2.2.1(1/1) 2.2.1.1.1(1/1)    beijing(296/296)    lin2.2.1(3/3),asian_african_2(2/2)

Calling lineages on thousands of VCFs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The simplest way to process thousands of VCFs is to use a ``for`` loop.
You put your ``vcf`` files in one folder and then type:

::

   for i in `ls |grep ".vcf"`; do
   fast-lineage-caller genomic_data/SAMEA968141.vcf --noheader
   done >> results.tsv

Getting all the lineage calls without removing any redundancy
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

By default *fast-lineage-caller* will try to remove the redundancy in
the lineage calls. What does this mean? The isolate ``SAMEA968141``, for
instance, belongs to the ``lineage2.2.1`` (according to the *coll2014*
SNP scheme). When *fast-lineage-caller* checks the variants present in
the *vcf* of this isolate, it will find that it actually belongs to
``lineage2``, ``lineage2.2`` and ``lineage2.2.1``. The information
contained in some these labels is redundant, so *fast-lineage-caller* by
default will output only ``lineage2.2.1``. However, for some use-cases
it is relevant to get all the calls. You can do that by using the
``--keep`` (keep redundancy) option:

::

   ./bin/fast-lineage-caller genomic_data/SAMEA968141.vcf --keep
   Isolate coll2014        freschi2020     lipworth2019    shitikov2017    stucki2016
   SAMEA968141     lineage2,lineage2.2.1,lineage2.2        2,2.2.1.1,2.2.1,2.2.1.1.1,2.2   beijing lin2,lin2.2.1,asian_african_2,lin2.2

SNP schemes
-----------

Default SNP schemes (for *Mtb*)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

These are the SNP schemes currently available on
``fast-lineage-caller``. They are the most widely used SNP schemes to
call Mycobacterium tuberculosis lineages / sub-lineages. However, if
this is not enough or you want to use ``fast-lineage-caller`` to call
lineages in other bacterial species, please remember that you can build
your own SNP scheme (see section below).

+-----+-------------------------------+-------------------------------+
| Tag | Reference                     | Notes                         |
+=====+===============================+===============================+
| col | Coll, F. *et al.* A robust    |                               |
| l20 | SNP barcode for typing        |                               |
| 14  | Mycobacterium tuberculosis    |                               |
|     | complex strains. *Nat.        |                               |
|     | Commun.* **5**, 4812 (2014)   |                               |
+-----+-------------------------------+-------------------------------+
| lip | Samuel Lipworth *et al.*      | This SNP barcode is           |
| wor | SNP-IT Tool for Identifying   | implemented in                |
| th2 | Subspecies and Associated     | `SNPit <https://github.com/ph |
| 019 | Lineages of Mycobacterium     | ilipwfowler/snpit>`__.        |
|     | tuberculosis Complex.         |                               |
|     | *Emerging Infectious Disease  |                               |
|     | journal* **25**, (2019)       |                               |
+-----+-------------------------------+-------------------------------+
| fre | Soon on BioRXiv!              |                               |
| sch |                               |                               |
| i20 |                               |                               |
| 20  |                               |                               |
+-----+-------------------------------+-------------------------------+
| shi | Shitikov, E. *et al.*         |                               |
| tik | Evolutionary pathway analysis |                               |
| ov2 | and unified classification of |                               |
| 017 | East Asian lineage of         |                               |
|     | Mycobacterium tuberculosis.   |                               |
|     | *Sci. Rep.* **7**, 9227       |                               |
|     | (2017)                        |                               |
+-----+-------------------------------+-------------------------------+
| stu | Stucki, D. *et al.*           |                               |
| cki | Mycobacterium tuberculosis    |                               |
| 201 | lineage 4 comprises globally  |                               |
| 6   | distributed and               |                               |
|     | geographically restricted     |                               |
|     | sublineages. *Nat. Genet.*    |                               |
|     | **48**, 1535–1543 (2016)      |                               |
+-----+-------------------------------+-------------------------------+

Building your own SNP scheme
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You just need to create a ``.tsv`` file (it is a tab separated value
file) with the following columns:

+-----------+----------------------------------------------------------+
| Column    | Description                                              |
+===========+==========================================================+
| lineage   | Name of the lineage or group of isolates                 |
+-----------+----------------------------------------------------------+
| position  | Genomic position of the SNP in the reference sequence    |
|           | (usually H37Rv for *Mtb*)                                |
+-----------+----------------------------------------------------------+
| allele_ch | ``<Reference_allele>/<Alternate_allele>``. For instance: |
| ange      | ``C/T``.                                                 |
+-----------+----------------------------------------------------------+
| tag       | Name of this SNP scheme                                  |
+-----------+----------------------------------------------------------+

**Note: please remember that the order the columns is important!**

Here is an example. I created a new file named ``my_snp_scheme.tsv``:

::

   #lineage        position        allele_change   tag
   lineage2        497491  G/A     new_SNP_scheme
   lineage2.2      2505085 G/A     new_SNP_scheme

**Note**: the SNP scheme parser skips all the lines that start with a
``#``, so you can put a ``#`` at the beginning of the line if you want
to add comments or temporarily remove SNPs from a SNP scheme.

Now you can use ``fast-lineage-caller`` to call the lineages with the
new SNP scheme:

::

   fast-lineage-caller ~/mfarhat/rollingDB/genomic_data/SAMEA968141/pilon/SAMEA968141.vcf --scheme ./my_snp_scheme.tsv 
   Isolate new_SNP_scheme
   SAMEA968141     lineage2.2

Changelog
---------

Version 0.3

-  the user can choose to use all the variants (default) or only the
   ones with the PASS flag for the lineage calling: ``--pass`` option
   (feature)
-  show a denominator when counting the SNPs (``--count`` option)
   (bugfix)
-  renamed the option ``--keepred`` to ``--keep`` (bugfix)
-  use ``NA`` instead of ``<blank>`` as output when no SNP is found in
   the .vcf for a given SNP scheme (bugfix)

Version 0.2

-  the user can decide to remove (default) or keep the redundant lineage
   calls: ``--keep`` option (feature)
-  the user can get the count of the SNPs that support a given lineage
   call: ``--count`` option (feature)
-  module available on conda (feature)
-  added information on how to build a custom SNP scheme (feature)
-  if multiple SNP schemes had the same SNP defining a given lineage /
   sub-lineage, only the lineage call of the last parsed SNP scheme was
   present in the output (bugfix; critical; please upgrade from 0.1)

Version 0.1

-  working python module (uploaded to pypi), only able to accept ``vcf``
   files
