Metadata-Version: 2.2
Name: memory-foam
Version: 0.0.2
Summary: Read the contents of files from S3 into memory
Author-email: Matt Seddon <mattseddon@hotmail.com>
License: Apache-2.0
Project-URL: Documentation, https://github.com/mattseddon/memory-foam
Project-URL: Issues, https://github.com/mattseddon/memory-foam/issues
Project-URL: Source, https://github.com/mattseddon/memory-foam
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Development Status :: 2 - Pre-Alpha
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fsspec>=2024.2.0
Requires-Dist: s3fs>=2024.2.0
Requires-Dist: gcsfs>=2024.2.0
Requires-Dist: adlfs>=2024.2.0
Provides-Extra: tests
Requires-Dist: pytest<9,>=8; extra == "tests"
Requires-Dist: pytest-sugar>=0.9.6; extra == "tests"
Requires-Dist: pytest-cov>=4.1.0; extra == "tests"
Requires-Dist: pytest-mock>=3.12.0; extra == "tests"
Requires-Dist: pytest-servers[all]>=0.5.9; extra == "tests"
Requires-Dist: hypothesis; extra == "tests"
Provides-Extra: dev
Requires-Dist: memory_foam[tests]; extra == "dev"
Requires-Dist: mypy==1.15.0; extra == "dev"
Provides-Extra: examples
Requires-Dist: dlt[duckdb]; extra == "examples"
Requires-Dist: pillow; extra == "examples"
Requires-Dist: ultralytics; extra == "examples"
Requires-Dist: tqdm; extra == "examples"

# memory-foam

`memory-foam` is a Python package that provides a set of iterators to load the contents of files from s3 cloud storage into memory for easy processing.

## Features

- **Unified Interface**: Seamlessly interact with files stored in S3.
- **Asynchronous Support**: Efficiently load files using asynchronous iterators.
- **Version Awareness**: Handle different versions of files with ease.

## Installation

You can install `memory-foam` using pip:

```bash
pip install memory-foam
```

## Example usage

```python
from io import BytesIO
from memory_foam import iter_files

...

for pointer, contents in iter_files(uri, client_config):
        results = process(contents)
        data = pointer.to_dict_with(results)
        save(data)
```
