Metadata-Version: 2.1
Name: moovai
Version: 0.0.4
Summary: A package for working with GCS and BigQuery
Home-page: https://github.com/moovai/Toolbox/
Author: Antsa Randriamihaja
Author-email: antsa.randriamihaja@moov.ai
License: UNKNOWN
Project-URL: Documentation, https://github.com/moovai/Toolbox/blob/master/README.md
Project-URL: Source Code, https://github.com/moovai/Toolbox/tree/master/MoovAI
Description: ## MOOVAI TOOLBOX
        
        This repository contains reusable code to expedite development. 
        To use repository, ensure you are using Python 3.6. 
        
        ### Installation:
        To use this package, install using pip:
        
        ```python
        pip install moovai
        ```
        
        ### Google Cloud:
        This folder contains code to handle GCP resources. 
        Prior to using the methods here, you need to be authenticated to GCP.
        If you are using a service account, and have the key JSON file:
        - On Linux or MacOS
        ```python
        export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
        ```
        - On Windows:
        
        ```python
        set GOOGLE_APPLICATION_CREDENTIALS=[PATH]
        ```
        
        #### cloud-storage
        Before using the code here, first make sure that you have a bucket on GCS. If you don't, create one.
        
        ##### upload_to_gcs: 
        Uploads a local file to a folder on google cloud storage. The file is then DELETED locally.
        ###### Parameters:
        - file: REQUIRED: [STRING] Local file path to upload to google cloud storage.
        - bucket_name: REQUIRED: [STRING]  Name of your bucket.
        - folder: REQUIRED: [STRING] Folder path to where you want your file to be uploaded to on GCS.
        
        returns: GCS URI where the file was uploaded
        ###### Sample usage:
        
        ```python
        from moovai.google_cloud import cloud_storage
        
        file = "/path/to/my/file.txt"
        bucket_name = "my_bucket"
        folder = "Folder/Subfolder/"
        
        cloud_storage.upload_to_gcs(file, bucket_name, folder)
        # returns "gs://my_bucket/Folder/Subfolder/file.txt"
        
        ```
        
        ##### download_file_gcs:
        Downloads file from google cloud storage to local disk. 
        ###### Parameters:
        - gcs_uri: REQUIRED. [STRING] URI of file located on google cloud storage to be downloaded
        
        returns: None. GCS file is downloaded locally.
        ###### Sample usage:
        
        ```python
        from moovai.google_cloud import cloud_storage
        
        gcs_uri = "gs://my_bucket/Folder/Subfolder/my_file.txt"
        
        cloud_storage.download_file_gcs(gcs_uri)
        # file "my_file.txt" downloaded locally
        
        ```
        
        #### bigquery
        ##### get_schema_from_json:
        Takes a schema.json file and converts it into a schema file compatible with BigQuery.
        ###### Parameters:
        - schema_path: REQUIRED. [STRING] Path to your_schema.json file.
        
        returns: schema to plug into BigQuery upload job.
        ###### Sample usage:
        
        ```python
        from moovai.google_cloud import bigquery
        
        schema_file = "/path/to/my/schema.json"
        bigquery.get_schema_from_json(schema_file)
        
        ```
        
        ##### get_schema_from_csv:
        Takes a csv file containing your data and extracts the schema from your file.
        It is recommended to simply use BigQuery's auto-detect schema feature. Use this as a last resort.
        ###### Parameters:
        - csv_file_path: REQUIRED. [STRING] Path to csv file.
        
        returns: schema to plug into BigQuery upload job.
        ###### Sample usage:
        
        ```python
        from moovai.google_cloud import bigquery
        
        csv_file = "/path/to/my/file.csv"
        bigquery.get_schema_from_csv(csv_file)
        
        ```
        
        ##### upload_local_file_to_bq:
        Uploads local file to BigQuery. schema_path and schema are optional arguments. They are exclusive of one another, provide only one if you want to.
        ###### Parameters:
        - file: REQUIRED. [STRING] Path to local CSV file to upload
        - dataset_id: REQUIRED. [STRING] Name of your BigQuery dataset.
        - table_id: REQUIRED. [STRING] Name of your BigQuery table to query.
        - schema_path: OPTIONAL. [STRING] Path to schema.json file. 
        - schema: OPTIONAL. [STRING] schema compatible with BigQuery. 
        - overwrite: OPTIONAL. [BOOL] Defaults to False. if set to True, BigQuery will overwrite table, else, it will append new data to table.
        
        returns: None.
        ###### Sample usage:
        ```python
        from moovai.google_cloud import bigquery
        
        file = "/path/to/my_file.csv"
        dataset_id = "my_dataset_id"
        table_id = "my_table_id"
        bigquery.upload_local_file_to_bq(file, dataset_id, table_id)
        
        ```
        
        ##### upload_gcs_file_to_bq:
        Uploads file from cloud storage to BigQuery. schema_path and schema are optional arguments. They are exclusive of one another, provide only one if you want to.
        ###### Parameters:
        - gcs_uri: REQUIRED. [STRING] Path to CSV file located on cloud storage to upload to BigQuery.
        - dataset_id: REQUIRED. [STRING] Name of your BigQuery dataset.
        - table_id: REQUIRED. [STRING] Name of your BigQuery table to query.
        - schema_path: OPTIONAL. [STRING] Path to schema.json file. 
        - schema: OPTIONAL. [STRING] schema compatible with BigQuery. 
        - overwrite: OPTIONAL. [BOOL] Defaults to False. if set to True, BigQuery will overwrite table, else, it will append new data to table.
        
        returns: None.
        ###### Sample usage:
        ```python
        from moovai.google_cloud import bigquery
        
        gcs_uri = "gs://my_bucket/Path/To/my_file.csv"
        dataset_id = "my_dataset_id"
        table_id = "my_table_id"
        bigquery.upload_local_file_to_bq(gcs_uri, dataset_id, table_id)
        
        ```
        
        ##### generate_sql_query:
        Generates a SQL query string to use to query a BigQuery table.
        ###### Parameters:
        - project: REQUIRED. [STRING] Project ID
        - dataset_id: REQUIRED. [STRING] Name of your BigQuery dataset.
        - table_id: REQUIRED. [STRING] Name of your BigQuery table to query.
        - columns: OPTIONAL. [ARRAY] List of column names you want to select.
        - conditions: OPTIONAL. [ARRAY] List of conditions to satisfy.
        
        returns: STRING. Standard SQL query string.
        ###### Sample usage:
        
        ```python
        from moovai.google_cloud import bigquery
        
        project = "my_project_id"
        dataset_id = "my_dataset_id"
        table_id = "my_table_id"
        bigquery.generate_sql_query(project, dataset_id, table_id)
        #returns "SELECT * FROM `project.dataset_id.table_id`" (return everything from table)
        
        columns = ["col1", "col2"]
        conditions = ["Date >= TIMESTAMP('2019-05-01')", "col3 < 3"]
        
        bigquery.generate_sql_query(project, dataset_id, table_id, columns=columns, conditions=conditions)
        #returns "SELECT col1, col2 FROM `project.dataset_id.table_id` WHERE Date >= TIMESTAMP('2019-05-01') AND col3 < 3" (returns col1 and col2 that meet the specified conditions)
        
        ```
        
        ##### get_data_from_bq:
        Takes a SQL query string (Standard SQL) and returns pandas dataframe
        ###### Parameters:
        - sql_query: REQUIRED. [STRING] string representing the query you want to make.
        
        returns: pandas Dataframe containg the result of your query.
        ###### Sample usage:
        
        ```python
        from moovai.google_cloud import bigquery
        
        sql_query = "SELECT * FROM `my_project.my_dataset.my_table`"
        bigquery.get_data_from_bq(sql_query)
        
        ```
        
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3.6
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
