# terra

a file containing a set of functions that uses [dalmatian](github.com/broadinstitute/dalmatian) to interact with the [GCP](https://cloud.google.com/storage/docs/gsutil) powered genomics HPC platform: [Terra](www.terra.bio). 
They contain a list of additional functions to do more than what is available in dalmatian

The goal is to improve reproducibility and productionalization of pipelines working with Terra.

#### Available functions:

- createManySubmissions: allows you to create many terra jobs in parallel
- waitForSubmission: an await function on Terra jobs
- removeSamples: a function that removes samples on a workspace and takes care of more edge cases (linked sample sets and pair sets..).
- uploadFromFolder: uploads fastq samples from a folder into a Terra workspace with the right namings etc..
- updateAllSampleSet: updates a sample set with all samples
- addToSampleSet: updates a sample set with some new samples
- addToPairSet: updates a pair set with some new pairs
- saveOmicsOutput: *WIP*
- changeGSlocation: Function to move data around from one workspace to a bucket or to another workspace. can also work on dataframes containing lists of paths
- renametsvs: *WIP*
- findBackErasedDuplicaBamteFromTerraBucket:   If you have erased bam files in gcp with bai files still present and the bam files are stored elsewhere and their location is in a terra workspace. Will find them back by matching bai sizes and copy them back to their original locations.
- shareTerraBams: will share some files from gcp with a set of users using terra as metadata repo. Only works with files that are listed on a terra workspace tsv but actually point to a regular google bucket and not a terra bucket.
- shareCCLEbams: same as shareTerraBams but is completed to work with CCLE bams from the CCLE sample tracker.
- saveConfigs: will save everything about a workspace into a csv and json file
- cleanWorkspace: removes all processing folder in a terra workspace easily
- changeToBucket: moves all bam/bai files in a sampleList from Terra, to another gs bucket and rename them in the sample list
- delete_job:  removes files generated by a job on Terra.
- removeFromFailedWorkflows: Lists all files from all jobs that have failed and deletes them.
- deleteHeavyFiles: deletes all files above a certain size in a workspace (that are used or unused).
- findFilesInWorkspaces: given All your terra workspaces, find a given gs filename

## highly recommanded

*This package won't contain anything that overlap with those and might use those packages for what it is doing.*
- firecloud-dalmatian (python)
- gsutil
- nextflow (better than terra)
