Metadata-Version: 2.1
Name: pyplatform-datalake
Version: 2020.11.1
Summary: Pyplatform-datalake package provides functions for Google Cloud Storage and Microsoft Storage services.
Home-page: https://github.com/mhadi813/pyplatform
Author: Muhammad Hadi
Author-email: mhadi813@gmail.com
License: BSD
Keywords: google bigquery cloud functions storage jupyterlab python SQL
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Office/Business
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Description-Content-Type: text/markdown
Requires-Dist: google-cloud-storage (==1.31.0)
Requires-Dist: gcsfs (==0.7.0)
Requires-Dist: azure-storage-blob (==1.5.0)
Requires-Dist: xlrd (==1.2.0)
Requires-Dist: XlsxWriter (==1.3.3)
Requires-Dist: openpyxl (==3.0.5)
Requires-Dist: pyplatform-common

### Pyplatform-datalake package provides functions for Google Cloud Storage and Microsoft Storage services.

## Installation
```python
pip install pyplatform-datalake
```

## Authentication and environment variables
Refer to [main page](https://github.com/mhadi813/pyplatform) for documentation on authentication

## Exploring the modules
```python

from pyplatform.datalake import *
show_me()
```

## Usage
### uploading in-memory data to Google Cloud Storage
```python
import pyplatform.datalake as dlk
from io import BytesIO
# df #TODO create dataframe as source
in_mem_file = BytesIO()
df.to_excel(in_mem_file,index=False)
in_mem_file.seek(0)
dlk.gcs_upload_blob(in_mem_file,bucket_id='bucket',blobname='test.xlsx')

```

### downloading and loading trained forecasting model from Google Cloud Storage
```python
import pyplatform.datalake as dlk
from io import BytesIO
import pickle

gcs_uri = 'gs://bucket/folder_name/model_name'
model_name="fbprophet_forecast_daily"
dlk.gcs_download_blob(gcs_uri,filepath=model_name)
with open(model_name,"rb") as model:
    my_model=pickle.load(model)

```

