# gwframe Upstream Feature Requests

This document tracks features and improvements identified during sgn-gwframe development that could be contributed upstream to the gwframe library.

**Date:** 2026-02-12
**sgn-gwframe Version:** Current main branch
**gwframe Version:** (to be determined)

---

## 1. Masked Array Support for Multi-Segment Data

**Priority:** High
**Status:** Needed for production feature
**Related Code:** `src/sgn_gwframe/sinks/frame.py:384-386`

### Current Limitation

GWFrameSink currently only handles the first buffer in a TSFrame:

```python
# TODO: fix this indexing to handle multiple buffers as multiple segments
# This requires converting frame data to masked arrays using
# the data_valid flag. Needs: sgn-ts MR 171 + gwframe masked array support
data = frame.buffers[0]
```

### Requested Feature

Support for writing masked arrays to GWF files to represent multi-segment data with gaps. When a TSFrame contains multiple buffers (segments), we need to:

1. Convert the segmented data to a masked array
2. Mark gap regions using the mask
3. Write the masked array to the frame file preserving gap information

### Use Case

Real-time pipelines may produce TSFrames with multiple segments due to:
- Data quality flags causing gaps
- Network interruptions
- Detector downtime

Currently, we can only write the first segment, losing the rest of the data.

### Proposed API

```python
import numpy as np
import gwframe

# Create masked array with gaps
data = np.ma.array([1, 2, 3, 4, 5], mask=[0, 0, 1, 1, 0])

frame = gwframe.Frame(t0=1000000000, duration=5, name="H1")
frame.add_channel("H1:TEST", data, sample_rate=1)
frame.write("output.gwf")

# On read, preserve mask
frame_data = gwframe.read("output.gwf", channel=["H1:TEST"])
assert isinstance(frame_data["H1:TEST"].array, np.ma.MaskedArray)
```

### References

- sgn-ts MR 171 (add data_valid tracking to buffers)
- GWF format specification for gap/invalid data representation

---

## 2. Frame History Metadata Improvements

**Priority:** Medium
**Status:** Works but could be enhanced
**Related Code:** `src/sgn_gwframe/sinks/frame.py:418-419`

### Current Implementation

We can add history metadata using:

```python
gwf.add_history(name, comment)
```

### Requested Enhancement

More structured history metadata API:

```python
gwf.add_history_entry(
    name="creator",
    comment="sgn-gwframe pipeline",
    timestamp=1000000000,  # Optional GPS time
    version="1.0.0",        # Optional version
    user=None,              # Optional username
)
```

### Use Case

Better tracking of data provenance and processing history for debugging and reproducibility.

---

## 3. Atomic Write Support in gwframe

**Priority:** Low
**Status:** Working around in sgn-gwframe
**Related Code:** `src/sgn_gwframe/sinks/frame.py:335-352`

### Current Workaround

We implement atomic writes ourselves:

```python
# Write to temporary file
temp_file = tmpdir / f".{filename}.tmp"
writer = gwframe.FrameWriter(str(temp_file))
writer.write_frame(frame)
writer.__exit__()

# Move to final location
shutil.move(temp_file, final_file)
```

### Requested Feature

Built-in atomic write support in FrameWriter:

```python
with gwframe.FrameWriter(filename, atomic=True, tmpdir=None) as writer:
    writer.write_frame(frame)
    # Automatically commits to final location on successful exit
    # Automatically cleans up temp file on exception
```

### Benefits

- Simpler user code
- Consistent atomic write behavior across all gwframe users
- Better error handling (automatic temp file cleanup)

---

## 4. Compression Control Documentation

**Priority:** Low
**Status:** Works but underdocumented
**Related Code:** `src/sgn_gwframe/sinks/frame.py:104-105`

### Current State

We use compression settings but documentation is unclear:

```python
compression: int = gwframe.Compression.ZERO_SUPPRESS_OTHERWISE_GZIP
compression_level: int = 6
```

### Requested Documentation

Clear documentation of:
- Available compression schemes (enum values)
- Compression level ranges and effects on file size/speed
- Recommended settings for different use cases (streaming vs archival)
- Performance benchmarks

### Example Documentation Needed

```
Compression Schemes:
- ZERO_SUPPRESS_OTHERWISE_GZIP (default): Fast for sparse data
- GZIP: General purpose compression
- NONE: No compression (fastest write, largest files)

Compression Levels (0-9):
- 0: No compression
- 1-3: Fast compression, lower ratio (recommended for streaming)
- 4-6: Balanced (default: 6)
- 7-9: Maximum compression, slower (recommended for archival)

Benchmarks:
- Level 1: ~50 MB/s write, 2:1 compression ratio
- Level 6: ~20 MB/s write, 3:1 compression ratio
- Level 9: ~10 MB/s write, 3.5:1 compression ratio
```

---

## 5. TimeSeries Validation

**Priority:** Low
**Status:** Working around in sgn-gwframe
**Related Code:** `src/sgn_gwframe/sources/framewatch.py:468-494`

### Current Workaround

We manually validate frame data after reading:

```python
frame = gwframe.read(filepath, channel=channels)

# Manual validation
for channel, series in frame.items():
    if int(series.sample_rate) != expected_rate:
        raise ValueError(f"Sample rate mismatch: {series.sample_rate} != {expected_rate}")
    if series.duration != expected_duration:
        raise ValueError(f"Duration mismatch: {series.duration} != {expected_duration}")
    if series.t0 != expected_t0:
        raise ValueError(f"Time mismatch: {series.t0} != {expected_t0}")
```

### Requested Feature

Built-in validation in gwframe.read():

```python
frame = gwframe.read(
    filepath,
    channel=channels,
    validate={
        "sample_rate": {channel: rate for channel, rate in expected_rates.items()},
        "duration": expected_duration,
        "t0": expected_t0,
    }
)
# Raises gwframe.ValidationError if any check fails
```

### Benefits

- Catch corrupted files earlier
- Clearer error messages from gwframe
- Reduce boilerplate validation code

---

## 6. Multi-Frame File Random Access

**Priority:** Low
**Status:** Potential future feature

### Use Case

When reading from multi-frame files (duration > frame_duration), ability to:

1. List all frames in a file without reading data
2. Read specific frames by index or time range
3. Seek to specific frames efficiently

### Proposed API

```python
# List frames in file
info = gwframe.inspect("multi_frame.gwf")
# Returns: [{"t0": 1000000000, "duration": 1, "channels": [...]}, ...]

# Read specific frame by index
frame = gwframe.read("multi_frame.gwf", frame_index=5, channel=["H1:TEST"])

# Read frame by time
frame = gwframe.read("multi_frame.gwf", t0=1000000005, channel=["H1:TEST"])
```

---

## 7. Channel Name Validation

**Priority:** Low
**Status:** No validation in gwframe

### Current Behavior

gwframe accepts any string as a channel name, including invalid ones.

### Requested Feature

Optional validation of channel names against LIGO T050017 format:

```python
gwf.add_channel(
    "H1:GDS-CALIB_STRAIN",
    data,
    sample_rate=16384,
    validate_name=True  # Check IFO:SUBSYSTEM-CHANNEL format
)
```

---

## Summary Table

| Feature | Priority | Status | Blocker? |
|---------|----------|--------|----------|
| Masked Array Support | High | Needed | Yes (for multi-segment) |
| History Metadata | Medium | Works | No |
| Atomic Writes | Low | Workaround OK | No |
| Compression Docs | Low | Underdocumented | No |
| TimeSeries Validation | Low | Workaround OK | No |
| Multi-Frame Random Access | Low | Future | No |
| Channel Name Validation | Low | No validation | No |

---

## Next Steps

1. ✅ Document feature requests (this document)
2. Check gwframe repository for existing issues/PRs related to these features
3. Prioritize: Start with masked array support (High priority, blocking feature)
4. Prepare proof-of-concept implementations
5. Submit upstream PRs with test coverage

---

## Contact

For questions about these feature requests, contact the sgn-gwframe maintainers.
