Metadata-Version: 2.3
Name: wids
Version: 0.1.1
Summary: A Python library for managing and processing dataset archives.
License: BSD-3-Clause
Author: Thomas Breuel
Author-email: tmbdev@gmail.com
Requires-Python: >=3.10
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Dist: braceexpand
Requires-Dist: numpy
Requires-Dist: pyyaml
Project-URL: Homepage, https://github.com/webdataset/wids
Project-URL: Repository, https://github.com/webdataset/wids
Description-Content-Type: text/markdown

# The Web Indexed Datasets Library

The Web Indexed Datasets Library (`wids`) is a library for reading
WebDataset datasets in PyTorch with indexed (random) access.  WebDataset
datasets are simply collections of standard POSIX tar files containing
training samples.

The `wids` library is a drop-in replacement for other indexed datasets
in Python.  Despite being indexed and random access, it can work
with datasets larger than local disk storage and performs incremental
downloads.  Under the covers, it uses a very fast memory mapped tar
file reader.

