Metadata-Version: 2.1
Name: fsmirror
Version: 0.1
Summary: A metadata management package based on filesystem mirroring.
Home-page: https://github.com/wesmadrigal/fsmirror
Author: Wes Madrigal
Author-email: wes@kurve.ai
License: MIT
Project-URL: Source, http://github.com/wesmadrigal/fsmirror
Project-URL: Issue Tracker, https://github.com/wesmadrigal/fsmirror/issues
Keywords: metadata management,filesystems
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Information Technology
Description-Content-Type: text/markdown
Requires-Dist: exceptiongroup==1.2.0
Requires-Dist: iniconfig==2.0.0
Requires-Dist: packaging==23.2
Requires-Dist: pluggy==1.4.0
Requires-Dist: pytest==8.0.2
Requires-Dist: PyYAML==6.0.1
Requires-Dist: tomli==2.0.1
Provides-Extra: base
Requires-Dist: exceptiongroup==1.2.0; extra == "base"
Requires-Dist: iniconfig==2.0.0; extra == "base"
Requires-Dist: packaging==23.2; extra == "base"
Requires-Dist: pluggy==1.4.0; extra == "base"
Requires-Dist: pytest==8.0.2; extra == "base"
Requires-Dist: PyYAML==6.0.1; extra == "base"
Requires-Dist: tomli==2.0.1; extra == "base"

# fsmirror

## Installation
```python
pip install fsmirror
```

### Functionality
Mirror project filesystems for metadata tracking.  It can be useful to have 
a direct path mirror between code that generates data and the location in a filesystem
or object store that stores the data / artifacts it generates.

### Usage

* Create a configuration file like the one in `examples/example_config.yml`
* Set the config path:
```bash
export FSMIRROR_CONFIG_PATH=/your/project/path/config.yml`
```

Use `fsmirror` for managing where to store artifacts, the following pseudocode is
an example of how it should be used:

```python
from some_data_pipeline import task
from storage_client import Client

from fsmirror.fsmirror import FSMirror, config

fm = FSMirror(config=config)

data = task()
storage_path = fm.mirror_relative(task, with_id=True)
Client.save_to_s3(storage_path)
```


