Metadata-Version: 2.1
Name: vinyl-core
Version: 0.0.3
Summary: A python framework for building next-gen analytics infrastructure
License: LICENSE.md
Author: Ian Tracey
Author-email: ian@turntable.so
Requires-Python: >=3.10,<4.0
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: boto3 (>=1.34.13,<2.0.0)
Requires-Dist: fsspec (>=2024.2.0,<2025.0.0)
Requires-Dist: geopandas (>=0.14.2,<0.15.0)
Requires-Dist: google-auth (>=2.28.0,<3.0.0)
Requires-Dist: lets-plot (>=4.2.0,<5.0.0)
Requires-Dist: netext (>=0.2.1,<0.3.0)
Requires-Dist: pydantic (>=2.5.3,<3.0.0)
Requires-Dist: python-dotenv (>=1.0.0,<2.0.0)
Requires-Dist: rustworkx (>=0.13.2,<0.14.0)
Requires-Dist: sqlglot[rs] (>=20.7.1,<21.0.0)
Requires-Dist: textual (>=0.49.0,<0.50.0)
Requires-Dist: toml (>=0.10.2,<0.11.0)
Requires-Dist: tqdm (>=4.66.1,<5.0.0)
Requires-Dist: turntable-spoonbill[bigquery,duckdb,postgres,snowflake] (>=8.0.0,<9.0.0)
Requires-Dist: typer[all] (>=0.9.0,<0.10.0)
Requires-Dist: watchdog[watchmedo] (>=3.0.0,<4.0.0)
Description-Content-Type: text/markdown

# Vinyl

## Sources

Sources are data models that can't be modified. Sources act as the bridge between external systems (SaaS tools, databases, other Vinyl projects, etc.) and Vinyl. Sources are the only way to get data into Vinyl. Sources are comprised of 3 parts: (1) a source definition which contains fields, typing and other metadata (2) a connector which is an adapter to the system where the data is stored and (3) a fetcher which tells Vinyl how to fetch data from the connector and at what frequency.

## Most Used Commands

`vinyl sources list` - list all sources available in the project
`vinyl sources cache` - cache data from all sources into a local repository (default is .turntable/sources)

## How to load sources

```python
import vinyl

conn = vinyl.connect('local')
```

The return of connect() is an ibis connection. The first time this may be slow as each table needs to be created. Subsequent loads are cached.

