---
title: Spool
execute:
  warning: false
---

Spools are containers/managers of [patches](patch.qmd). Spools come in a few varieties which
can manage a group of patches loaded into memory, archives of local files,
and (in the future) a variety of clients for accessing remote resources.

# Data Sources

The simplest way to get the appropriate spool for a specified input is to use
the `spool` function, which should work in the vast majority of cases.

## Patches (in-memory)

```{python}
import dascore as dc

patch_list = [dc.get_example_patch()]

spool1 = dc.spool(patch_list)
```

## A Single File

```{python}
import dascore as dc
from dascore.utils.downloader import fetch

path_to_das_file = fetch("terra15_das_1_trimmed.hdf5")

spool2 = dc.spool(path_to_das_file)
```

## A Directory of DAS Files

```{python}
import dascore as dc
from dascore.utils.downloader import fetch

# we use the fetch to make sure the file is downloaded. You would
# just need to replace directory_path with your data directory path.
path_to_das_file = fetch("terra15_das_1_trimmed.hdf5")
directory_path = path_to_das_file.parent

# update will create an index of the contents for fast querying/access
spool3 = dc.spool(directory_path).update()
```

If you want the index file to exist somewhere else, for example if you can't
write to the data directory, you can specify an index path.

```{python}
#| warning: false
#| output: false
import tempfile
from pathlib import Path

index_path = Path(tempfile.mkdtemp()) / "index.h5"

# update will create an index of the contents for fast querying/access
spool = dc.spool(directory_path, index_path=index_path).update()

# new spools will also need to specify the index path
spool_new = dc.spool(directory_path, index_path=index_path).update()
```

:::{.callout-warning}
If you remove files from a directory that has already been indexed, you should delete the index file and then call `update` again on the spool like this:

```python
spool.indexer.index_path.unlink()
spool.update()
```

It is best not to delete files once added to a directory managed by a spool.
:::

Despite some implementation differences, all spools have common behavior/methods.

# Accessing patches

Patches are extracted from the spool via simple iteration or indexing. New
spools are returned via slicing.

```{python}
import dascore as dc

spool = dc.get_example_spool()

patch = spool[0]  # extract first patch

# iterate patchs
for patch in spool:
    ...

# slice spool to create new spool which excludes first patch.
new_spool = spool[1:]
```

# get_contents

Returns a dataframe listing contents. This method may not be supported on all
spools, especially those interfacing with vast remote resources.

```{python}
#| output: false
import dascore as dc
spool = dc.get_example_spool()

# Return dataframe with contents of spool (each row has metadata of a patch)
contents = spool.get_contents()
print(contents)
```

```{python}
#| echo: false
from IPython.display import display

display(contents.drop(columns=['patch']))
```

# select

Selects a subset of a spool and returns a new spool. `get_contents` will now
reflect a subset of the original data requested by the select operation.

```{python}
import dascore as dc
spool = dc.get_example_spool()

# select a spool with
subspool = spool.select(time=('2020-01-03T00:00:09', None))
```

In addition to trimming the data along a specified dimension (as shown above),
select can be used to filter patches that meet a specified criteria.


```{python}
import dascore as dc
# load a spool which has many diverse patches
spool = dc.get_example_spool('diverse_das')

# Only include patches which are in network 'das2' or 'das3'
subspool = spool.select(network={'das2', 'das3'})

# only include spools which match some unix-style query on their tags.
subspool = spool.select(tag='some*')
```

# chunk

Chunk controls how data are grouped together in patches. It can be used to
merge contiguous patches together, specify the size of patches for processing,
specify overlap with previous segments, etc.

```{python}
import dascore as dc
spool = dc.get_example_spool()

# chunk spool for 3 second increments with 1 second overlaps
# and keep any segements that don't have full 3600 seconds
subspool = spool.chunk(time=3, overlap=1, keep_partial=True)

# merge all contiguous segments along time dimension
merged_spool = spool.chunk(time=None)
```
