# SigProfilerExtractor
SigProfilerExtractor allows de novo extraction of mutational signatures from data generated in a matrix format. 
The tool identifies the number of operative mutational signatures, their activities in each sample, and the probability 
for each signature to cause a specific mutation type in a cancer sample. The tool makes use of SigProfilerMatrixGenerator 
and SigProfilerPlotting. 

## PREREQUISITES

This module requires all the prerequisites of the 
[SigProfilerMatrixGenerator](https://github.com/AlexandrovLab/SigProfilerMatrixGenerator/blob/master/README.md) and 
[SigProfilerPlotting](https://github.com/AlexandrovLab/SigProfilerPlotting/blob/master/README.md) plus the following
packages referred in the table below:

| Name of packages | Version     | How to install  |
| ------------- |:-------------:| :-----:|
| nimfa         | 1.3.4 or more | ``` conda install -c cdeepakroy nimfa   ```|
| sigProfilerPlotting | 1.1 or more      |  ``` conda install sigProfilerPlotting ```|
| scipy          | 1.1 or more      |  ``` conda install scipy ``` |
| scikit-learn   | 0.19.1 or more | ``` conda install scikit-learn ```|
| mpi4py(only to use MPI)  | 2.0 or more      |  ``` conda install mpi4py ```|

"Conda" is a cross-platform, open source package manager that can be used to install different versions of software packages and libraries. To get help to install and use conda, please visit [SigProfilerMatrixGenerator](https://github.com/AlexandrovLab/SigProfilerMatrixGenerator/blob/master/README.md). Currently, SigProfilerExtractor supports only the Linux/Unix operating system. 

## INSTALLATION 
To install the SigProfilerExtractor, the users should download the [SigProfilerExtractor](https://github.com/AlexandrovLab/SigProfilerExtractor) repository from github. 

After downloading the repository users have to enter into the "SigProfilerExtractor" directory and run the following code:

``` bash installer.sh ```

This command will install the program which normally takes a couple of hours. After SigProfilerExtractor is installed, it is ready to use in the Linux/Unix commandline. 

## HOW TO USE
Accurately extracting signatures of mutational processes is computationally intensive. SigProfilerExtractor is usually executed on a computational cluster. This type of parallel execution can be performed in methods using  SigProfilerExtractor. 
1. Multiprocessing , 
2. Message Passing Interface (MPI)  

The following commands are used to run the SigProfilerExtractor
### COMMANDS
##### Required Arguments--
These commands are required to run SigProfilerExtractor

**-t, --type**: The _input type_. There are three available input types: "vcf", "text", "matobj". The "vcf" type input will
                load the mutational catalogue from a varriant caller data.
                

 
 

 * In order to use the **"vcf"** type input, users have to follow the following list of procedure first, 
   * 1. Create a new folder for each project/job that you run within the SigProfilerMatrixGenerator/references/vcf files/ folder. Use a unique name for each project/job.
   * 2. Separate your INDEL mutations from your SNV mutations if \ they are present in the same files, and create a folder for each mutation type (ex: SigProfilerMatrixGenerator/references/vcf files/[project]/SNV/ or SigPro- filerMatrixGenerator/references/vcf files/[project]/INDEL/).
   * 3. Place your vcf files within these new folders (either SigProfilerMatrix- Generator/references/vcf files/[project]/SNV or SigProfilerMatrixGener- ator/references/vcf files/[project]/INDEL/).
 
 * In order to use the **"matobj"** type input, first users have to the place matlab object file in the input folder.
 * In order to use the **"text"** type input, first users have to the place tab delimited text file (usually generated by the [SigProfilerMatrixGenerator](https://github.com/AlexandrovLab/SigProfilerMatrixGenerator)) in the input folder.
 
 **-o, --output**: Users have to set the name of the _output directory_ where the results will be stored.
 
 ##### Semi Required Arguments--
 These commands are required depending on the _input type_ 
 
 **-p _or_ --project**: Name of the _project file_ (created earlier). This argument is mandatory for the "vcf" type input.
 
 **-r _or_  --refgen**: Name of the _reference genome_. This argument is mandatory for the "vcf" type input.
 
 **-i _or_  --inputfile**: The name of the _input file_. This argument is mandatory for "text" or "matobj" type input.
 
 
 ##### Optional Arguments--
 These commands are optional to run SigProfilerExtractor

 **-s _or_  --startprocess**: The _minimum number of processes_ to be extracted. The default value is 1. 

 **-e _or_  --endprocess**: The _maximum number of processes_ to be extracted. The default value is 2.

 **-n _or_  --n_iterations**: The _number of iterations_ to be executed. The default value is 3. 

 **-c _or_  --cpu**: The _number of cpu_ to be executed for parallel computation. The default value will use the maximum number of  the available cpus.

 **-m _or_  --mtypes**: The context of mutations and is optional. This is
                        valid when the _input type_ is "vcf". User should pass
                        the inteded mutation types among to be analyzed
                        separeted by coma "," with no space. The sigporfiler
                        engine will analyze the specific mutation types those
                        are passed to this argument. The valid mutation type
                        are 6, 12, 96, 1536, 192, 3072 and DINUC. For example,
                        if the user wants analyze mutation type 96, 192 and
                        DINUC, that person should pass "--mtypes 96,192,DINUC"
                        as in the argument. If the argument is not used, "96", "DINUC", 
                        "INDEL" (if --indel is used) will be extracted. 
 
 **-l _or_  --layer**: Optional parameter that set if the signatures will be extrated in a _hierarchical manner_.

 
 **--indel**: Optional parameter instructs script to create the
                        catalogue for _limited INDELs_. This parameter is valid
                        only for the "vcf" input.
   
 **--extended_indel**: Optional parameter instructs script to create the
                        catalogue for _extended INDELs_. This parameter is valid
                        only for the "vcf" input.
                    
**--exome**: Optional parameter instructs script to create the
                        catalogues using only the _exome regions_. Whole genome
                        context by default. This parameter is valid only for
                        the "vcf" input.        

### EXECUTION 
#### USING MULTIPROCESSING
To execute a job using SigProfilerExtractor, user need to run the sigpro.py file from the source directory inside SigProfilerExtractor. Users need to pass the parameters previously described argurments (above) for the sigpro.py file. 

Example 1: 

``` python3 sigpro.py -t vcf -o results -p projectA -r GRCh37 -s 1 -e 10 -n 500 -c 8 -m "96","192","INDEL" -l --indel ```

The above command will extract the mutational processes from a "vcf" _input type_ , from "projectA", using _referrence genomes_ "GRCh37". The _start processes_ will be 1, _end processes_ 10, the _number of iteration_ will be 500 and _number of cpus_ perticipating in the computaion will be 8. Processes will be extracted _hierarchically_ (since -l is there) for the _mutational contexts_ "96", "192 and "INDEL" (since --indel is there). Finally, the output information will stored in the "results" folder in the SigProfilerExtractor directory.    

#### USING MESSAGE PASSING INTERFACE (MPI)
To execute a job using SigProfilerExtractor, user need to run the sigpro.py file from the source directory inside SigProfilerExtractor. Users need to pass the parameters previously described argurments for the mpi_sigpro.py file. However, in the command section, users have to add _mpiexec_ command and its parameter _-n_ before the original command to run the python file. Here, _n_ is the number of cpus to be used. The "-c/--cpu" and "-l/--layer" arguments are **NOT applicable** for mpi_sigpro.py.  

Example 2: 

``` mpiexec -n 8 python3 mpi_sigpro.py -t vcf -o results -p projectA -r GRCh37 -s 1 -e 10 -n 500 -m "96","192","INDEL" --indel ```

The above command will perform the similam task as Example one except extracting the processes in a hierarchical manner since _"-l/--layer"_ argument is not applicable for mpi_sigpro.py.  


### OUTPUT
After SigProfilerExtractor is successfully executed, a output directory will be generated according to the name of the parameter of the (-o/--output) argument. The directory will be created inside the SigProfilerExtractor directory. In the "output" directory there will be subfolder for each type of mutational contexts. Inside each mutational context subdirectory, there will be subdirectories for the each layer (L) if the (-l/--layer) parameters are used to extract the matutational processes in a hierarchical manner. Inside each layer (L) subdirectories, there will be two subdirectories "All solution" and "Selected solution", one image file showing the "reconstruction error vs stability plot" for each number of processes and csvfile listing reconstruction errors and stabilities for each number of processes. Inside the "All solutions" subdirectory, there will be subdirectories for each number of processes. Each of the subdirectories will contain the processes files, exposures file, probability file (the probability of occuring processes in each mutations for a given sample) and a plot for the signatures. The "Selected solution" directories will contain the files for selected number of mutational processes automatically by the SigProfilerExtractor. 

## COPYRIGHT
This software and its documentation are copyright 2018 as a part of the sigProfiler project. The SigProfilerExtractor framework is free software and is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

## CONTACT INFORMATION
Please address any queries or bug reports to S M Ashiqul Islam (Mishu) at m0islam.ucsd.edu
