# encoding: utf-8
# THIS FILE IS AUTOGENERATED!
from __future__ import unicode_literals
from setuptools import setup
setup(
    author='Kyle Lahnakoski',
    author_email='kyle@lahnakoski.com',
    classifiers=["Development Status :: 4 - Beta","Topic :: Software Development :: Libraries","Topic :: Software Development :: Libraries :: Python Modules","License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)","Programming Language :: Python :: 3.6","Programming Language :: Python :: 3.7","Programming Language :: Python :: 3.8"],
    description='jx-bigquery - JSON Expressions for BigQuery',
    include_package_data=True,
    install_requires=["google-cloud-bigquery","jx-python>=3.57.20089","jx-python>=3.57.20089","mo-dots>=3.53.20059","mo-future>=3.51.20059","mo-json>=3.57.20089","mo-kwargs>=3.54.20059","mo-logs>=3.57.20089","mo-sql>=3.55.20074","mo-times>=3.57.20089","mo-times>=3.57.20089"],
    license='MPL 2.0',
    long_description='# jx-bigquery\n\nJSON Expressions for BigQuery\n\n\n## Status\n\nMarch 2020 - Active but incomplete:  Can insert [tidy](https://en.wikipedia.org/wiki/Tidy_data) JSON documents into BigQuery while managing the schema.  Queries are not supported yet.\n\n## Overview\n\nThe library is intended to manage multiple BigQuery tables to give the illusion of one table with a dynamically managed schema. \n\n\n## Definitions\n\n* `partition` - Big data is split into separate containers based on age. This allows queries on recent data to use less resources, and allows old data to be dropped quickly\n* `cluster` - another name for the sorted order of the data in a partition. Sorting by the most common used lookup will make queries faster\n* `id` - The set of columns that identifies the document \n\n\n## Configuration\n\n* `table` - Any name you wish to give to this table series\n* `top_level_fields` - BigQuery demands that control columns are top-level.  Define them here.\n* `partition` - \n  * `field` - The dot-delimited field used to partition the tables (must be time datatype)\n  * `expire` - When BigQuery will automatically drop your data. \n* `id` - The identification of documents \n  * `field` - the set of columns to uniquely identify this document\n  * `version` - column used to determine age of a document; replacing newer with older\n* `cluster` - Columns used to sort the partitions \n* `schema` - {name: type} dictionary - needed when there is no data; BigQuery demands column definitions\n* `sharded` - *boolean* - set to `true` if you allow this library to track multiple tables. It allows for schema migration (expansion only), and for faster insert from a multitude of machines  \n* `account_info` - The information BigQuery provides to connect \n\n### Example\n\nThis is a complicated example. See [tests/config.json](https://github.com/klahnakoski/jx-bigquery/blob/dev/tests/config.json) for a minimal example.\n\n```json\n{\n    "table": "my_table_name",\n    "top_level_fields": {},\n    "partition": {\n        "field": "submit_time",\n        "expire": "2year"\n    },\n    "id": {\n        "field": "id",\n        "version": "last_modified"\n    },\n    "cluster": [\n        "id",\n        "last_modified"\n    ],\n    "schema": {\n        "id": "integer",\n        "submit_time": "time",\n        "last_modified": "time"\n    },\n    "sharded": true,\n    "account_info": {\n        "private_key_id": {\n            "$ref": "env://BIGQUERY_PRIVATE_KEY_ID"\n        },\n        "private_key": {\n            "$ref": "env://BIGQUERY_PRIVATE_KEY"\n        },\n        "type": "service_account",\n        "project_id": "my-project-id",\n        "client_email": "me@my_project.iam.gserviceaccount.com",\n        "client_id": "12345",\n        "auth_uri": "https://accounts.google.com/o/oauth2/auth",\n        "token_uri": "https://oauth2.googleapis.com/token",\n        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",\n        "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/my-project.iam.gserviceaccount.com"\n    }\n}\n```\n\n## Usage\n\nSetup `Dataset` with an application name\n\n```python\n    dataset = bigquery.Dataset(\n        dataset=application_name, \n        kwargs=settings\n    )\n```\n\nCreate a table\n\n```python\n    destination = dataset.get_or_create_table(settings.destination)\n```\n\nInsert documents as you please\n\n\n```python\n    destination.extend(documents)\n```\n\nRequest a merge when done\n\n```python\n    destination.merge()\n```\n\n## Running tests\n\n\nFork and clone this repo. \n\n```\ngit clone https://github.com/klahnakoski/jx-bigquery.git \ncd jx-bigquery\npip install -r requirements.txt\n```\n\nYou will require a Google API key to run tests. The website will allow you to generate one and download a JSON file with the key.  Update the [tests/config.json](https://github.com/klahnakoski/jx-bigquery/blob/dev/tests/config.json) to point to that file: \n\n\n```\n# contents of tests/config.json\n{\n  "destination": {\n    "account_info": {\n      "$ref": "file:///e:/moz-fx-dev-ekyle-treeherder-a838a7718652.json"\n    }\n  },\n  "constants": {},\n  "debug": {\n    "trace": true\n  }\n}\n```\n\nThen you can run the tests\n\n```\npython -m unittest discover tests\n```\n\n> **NOTE** - the tests will create a `testing` dataset and generate/drop tables \n',
    long_description_content_type='text/markdown',
    name='jx-bigquery',
    packages=["jx_bigquery/expressions","jx_bigquery"],
    url='https://github.com/klahnakoski/jx-bigquery',
    version='3.61.20093'
)