Metadata-Version: 2.1
Name: quiltcore
Version: 0.5.0
Summary: low-level plubming to read/write Quilt packages
Author: Ernest Prabhakar
Author-email: ernest@quiltdata.io
Requires-Python: >=3.10,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: jsonlines (==1.2.0)
Requires-Dist: multiformats (>=0.2.1,<0.3.0)
Requires-Dist: pandas (>=2.0.2,<3.0.0)
Requires-Dist: pandas-stubs (>=2.0.2.230605,<3.0.0.0)
Requires-Dist: pyarrow (>=13.0.0,<14.0.0)
Requires-Dist: quilt3 (>=5.3.1,<6.0.0)
Requires-Dist: s3fs (>=2023.6.0,<2024.0.0)
Requires-Dist: typing-extensions (>=4.6.3,<5.0.0)
Requires-Dist: tzlocal (>=5.0.1,<6.0.0)
Requires-Dist: un-yaml (>=0.3.1)
Requires-Dist: universal-pathlib (>=0.1.3,<0.2.0)
Description-Content-Type: text/markdown

# QuiltCore

QuiltCore is a library for building and running [Quilt](https://quiltdata.com) data packages.
It is designed to leverage standard open source technology and YAML configuration files
so that it can easily be ported to other languages and platforms.

This initial implementation is in Python.

## Key Technologies

- Apache [Arrow](https://arrow.apache.org/) for reading, writing, and representing manifests
  - [PyArrow](https://arrow.apache.org/docs/python/) for Python bindings to Arrow
- fsspec [filesystems](https://filesystem-spec.readthedocs.io/en/latest/)
  for reading and writing files from various sources
- [PyYAML](https://pyyaml.org/) for reading and writing YAML configuration files

## Example

```bash
poetry install
```

```python
from quiltcore import Registry
from tempfile import TemporaryDirectory
from upath import UPath

TEST_BKT = "s3://quilt-example"
TEST_PKG = "akarve/amazon-reviews"
TEST_TAG = "1570503102"
TEST_HASH = "ffe323137d0a84a9d1d6f200cecd616f434e121b3f53a8891a5c8d70f82244c2"
TEST_KEY = "camera-reviews"
```

### Get Manifest

<!--pytest-codeblocks:cont-->
```python
path = UPath(TEST_BKT)
registry = Registry(path)
named_package = registry.get(TEST_PKG)
manifest = named_package.get(TEST_TAG)
entry = manifest.get(TEST_KEY)
```

### Get Object

<!--pytest.mark.skip-->
```python
with TemporaryDirectory() as tmpdir:
  dest = UPath(tmpdir)
  outfile = dest / TEST_KEY
  entry.install(tmpdir)
  print(outfile.resolve())
  assert outfile.exists()
  local_bytes = outfile.read_bytes()
  assert entry.verify(local_bytes)
```

