# Oscura

**Python framework for hardware reverse engineering: signal analysis, protocol decoding, and automated dissector generation.**

Unified workflows from oscilloscope captures to Wireshark dissectors. IEEE-compliant measurements, native protocol decoders, differential analysis, and CRC recovery—all in Python without tool juggling.

[![CI](https://github.com/oscura-re/oscura/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/oscura-re/oscura/actions/workflows/ci.yml)
[![Code Quality](https://github.com/oscura-re/oscura/actions/workflows/code-quality.yml/badge.svg?branch=main)](https://github.com/oscura-re/oscura/actions/workflows/code-quality.yml)
[![codecov](https://codecov.io/gh/oscura-re/oscura/graph/badge.svg)](https://codecov.io/gh/oscura-re/oscura)
[![PyPI version](https://img.shields.io/pypi/v/oscura)](https://pypi.org/project/oscura/)
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

---

## What Oscura Provides

Unified Python framework for hardware reverse engineering workflows:

- **File Format Loading** - Oscilloscopes (Tektronix, Rigol), logic analyzers (Sigrok, VCD), network captures (PCAP), automotive (BLF), scientific instruments (HDF5, TDMS, Touchstone)
- **Protocol Decoding** - Serial (UART, SPI, I2C), automotive (CAN, LIN, FlexRay), debug (JTAG, SWD), industrial (Modbus, BACnet, EtherCAT, OPC UA, PROFINET), audio (I2S), wireless (BLE)
- **Signal Analysis** - Waveform measurements (IEEE 181), spectral analysis (IEEE 1241), jitter analysis (IEEE 2414), power analysis (IEEE 1459), eye diagrams, signal integrity
- **Protocol Inference** - CRC recovery, state machine extraction, field detection, format identification, magic byte discovery
- **Export Automation** - Wireshark dissectors, Scapy layers, DBC files, Kaitai Struct, 010 Editor templates, HTML/JSON reports

---

## Installation

```bash
pip install oscura
```

For development setup or detailed installation options, see [Installation Guide](docs/getting-started/installation.md).

**Requirements**: Python 3.12+ | Dependencies: numpy, scipy, matplotlib, scikit-learn, cantools (all peer-reviewed, industry-standard libraries)

---

## Usage Approaches

Oscura provides 5 usage patterns. **[→ Choose your approach](docs/getting-started/quick-start.md)**

Quick summary:

- **Comprehensive Script** - Immediate analysis ([showcase](demos/showcase/analyze_data_capture.py))
- **Convenience API** - Python applications ([examples](examples/))
- **Expert API** - Full control ([API docs](docs/api/))
- **YAML Pipeline** - Config-driven ([guide](docs/user-guide/integrations/yaml-pipelines.md))
- **CLI** - Command-line ([reference](docs/reference/cli.md))

---

## Example: Unknown Protocol Reverse Engineering

```python
from oscura.sessions import BlackBoxSession

# Differential analysis to identify protocol structure
session = BlackBoxSession(name="Unknown Device")
session.add_recording("idle", "idle.bin")
session.add_recording("active", "active.bin")
diff = session.compare("idle", "active")

# Generate Wireshark dissector from inferred structure
session.export_results("dissector", "protocol.lua")
```

---

## Capabilities Reference

### File Format Support

Unified loader with automatic format detection via `oscura.load(path)`:

- **Oscilloscopes**: Tektronix WFM/TSS, Rigol WFM
- **Logic Analyzers**: VCD, Sigrok .sr
- **Network**: PCAP/PCAPNG
- **Scientific**: HDF5, TDMS, Touchstone S-parameters
- **Generic**: WAV, CSV, NumPy, binary dumps
- **Side-Channel**: ChipWhisperer traces

### Waveform Measurements (IEEE 181-2011)

Timing, amplitude, and quality measurements with sub-sample interpolation:

- **Timing**: rise/fall time, period, frequency, pulse width, duty cycle, phase, delay, slew rate
- **Amplitude**: peak-to-peak, RMS, mean, overshoot, undershoot, top, base
- **Advanced**: area, burst width, extinction ratio, quality factor

### Spectral Analysis (IEEE 1241-2010)

FFT, PSD, quality metrics:

- **Frequency Domain**: FFT, PSD (Welch method), spectrogram, coherence
- **Quality Metrics**: THD, SNR, SINAD, SFDR, ENOB
- **Advanced**: Cepstrum, bispectrum, cross-spectrum, transfer function

### Digital Signal Analysis

- **Conversion**: Threshold detection with hysteresis, edge detection, clock extraction
- **Timing**: Clock/baud rate detection, pulse width measurement, frame sync
- **Logic Levels**: TTL, CMOS, LVCMOS, LVDS, ECL detection
- **Quality**: Glitch detection, timing violations, signal quality metrics

### Eye Diagram Analysis

Eye diagram generation and measurements: height, width, area, crossing percentage, Q-factor, BER estimation, mask testing.

### Jitter Analysis (IEEE 2414-2020)

RJ/DJ decomposition using dual-Dirac model:

- **Components**: Random jitter (RJ), deterministic jitter (DJ), periodic jitter (PJ), data-dependent jitter (DDJ)
- **Measurements**: TIE, period jitter, cycle-to-cycle jitter
- **Visualization**: Bathtub curves for BER extrapolation

### Power Analysis (IEEE 1459-2010)

- **Basic**: Instantaneous, average, peak, RMS power, energy
- **AC Power**: Active, reactive, apparent power, power factor
- **Harmonics**: THD, harmonic content, crest factor
- **Quality**: Ripple, sag/swell detection, flicker, unbalance

### Statistical Analysis

- **Descriptive**: Mean, median, std dev, quartiles, percentiles
- **Distribution**: Histograms, skewness, kurtosis, normality tests
- **Outliers**: Z-score, IQR, MAD, isolation forest, DBSCAN
- **Correlation**: Auto/cross-correlation, Pearson, Spearman
- **Time Series**: Periodicity detection, trend analysis, change points

### Pattern Recognition

- **Period Detection**: FFT-based, autocorrelation, multi-method
- **Sequences**: Repeating patterns, n-grams, Lempel-Ziv complexity
- **Entropy**: Shannon entropy, byte frequency, encrypted/compressed region detection
- **Structure**: Magic bytes, headers, delimiters, sync patterns, alignment

### Protocol Decoders

Native state machine implementations:

- **Serial**: UART, SPI, I2C, 1-Wire
- **Automotive**: CAN, CAN-FD, LIN, FlexRay
- **Debug**: JTAG (IEEE 1149.1), SWD (ARM CoreSight)
- **Industrial**: Modbus, BACnet, EtherCAT, OPC UA, PROFINET, HDLC
- **Audio**: I2S
- **Wireless**: BLE advertising packets
- **Parallel**: Centronics, GPIB (IEEE 488.2)

### Protocol Inference

- **CRC Recovery**: Brute-force parameter search, algorithm detection
- **Field Detection**: Statistical boundary inference, type classification
- **Format Analysis**: Magic bytes, framing, escaping detection
- **Protocol Detection**: Multi-heuristic with confidence scoring
- **State Machines**: RPNI algorithm for state extraction
- **Dissector Generation**: Wireshark (Lua), Scapy (Python), Kaitai Struct, 010 Editor

### Automotive Analysis

- **CAN**: Frame decoding, ID identification, statistical analysis, differential analysis
- **DBC Generation**: Signal inference, automatic naming, scaling detection
- **Diagnostics**: OBD-II, UDS, J1939, DTC lookup
- **Security**: Replay detection, fuzzing, anomaly detection

### Signal Integrity

TDR analysis, S-parameters (Touchstone), insertion/return loss, VSWR, impedance calculation, crosstalk detection.

### Machine Learning

- **Classification**: Signal type, protocol identification, modulation detection (Random Forest/SVM)
- **Feature Extraction**: Timing, spectral, statistical features
- **Clustering**: K-means, DBSCAN, hierarchical
- **Anomaly Detection**: Isolation Forest, One-Class SVM

### Visualization

Waveforms, FFT/PSD spectra, spectrograms, histograms, logic analyzer views, eye diagrams, bathtub curves, constellation diagrams, Smith charts, Bode/Nyquist plots, protocol timelines.

### Export Formats

JSON, CSV, HTML reports, Wireshark dissectors (Lua), Scapy dissectors (Python), Kaitai Struct (YAML), DBC (CAN database), 010 Editor templates, WAV, VCD, NumPy, HDF5, MATLAB.

---

## When to Use Oscura

**Best suited for:**

- End-to-end Python workflows (capture → analysis → documentation)
- Unknown protocol reverse engineering with differential analysis
- DBC generation from CAN captures (open-source CANalyzer alternative)
- Automatic Wireshark dissector generation from inferred protocols
- Unified API across 14 oscilloscope/logic analyzer formats
- Reproducible research with hypothesis tracking
- Scripted multi-tool RE pipelines

**Oscura's strength:** Workflow automation combining multiple RE steps with hypothesis-driven analysis.

---

## Standards Compliance

Oscura implements measurements per IEEE specifications.

| Standard | Coverage | Implementation |
|----------|----------|----------------|
| IEEE 181-2011 | Pulse timing, rise/fall, overshoot, duty cycle | Native algorithms |
| IEEE 1241-2010 | SNR, SINAD, THD, SFDR, ENOB | Native + scipy.signal |
| IEEE 1459-2010 | Active/reactive power, harmonics, power factor | Native + scipy.signal |
| IEEE 2414-2020 | TIE, RJ/DJ decomposition, bathtub curves | Native + scipy.stats |

**Validation**: 475 test files with 80%+ code coverage, property-based testing via Hypothesis.

---

## Quality Metrics

**Testing**: 475 test files, 80%+ coverage with branch coverage, property-based testing (Hypothesis), nightly stress tests

**CI/CD**: Pre-commit hooks (format, lint, type check), merge queue validation, security scanning (Bandit, Safety)

**Type Safety**: MyPy strict mode, comprehensive type hints

**Documentation**: Google-style docstrings, 95% documentation coverage

View metrics: [CI Dashboard](https://github.com/oscura-re/oscura/actions) | [Coverage](https://codecov.io/gh/oscura-re/oscura)

---

## Demonstrations

**Comprehensive working demonstrations** covering all framework capabilities. See [demos/README.md](demos/README.md) for complete catalog.

**Categories**: Waveform Analysis, File I/O, Custom DAQ, Serial Protocols, Protocol Decoding, UDP Analysis, Protocol Inference, Automotive, Timing, Mixed Signal, Spectral, Jitter, Power, Signal Integrity, EMC, Signal RE, Advanced Inference, Complete Workflows

**Learning Paths**:

- Beginner (2-4 hours): File loading, basic measurements, export
- Intermediate (6-10 hours): Protocol decoding (UART, SPI, I2C, PCAP)
- Advanced (12-20 hours): CRC recovery, state machines, Wireshark dissectors
- Expert (20-40 hours): Complete RE workflows with ML inference

```bash
./scripts/setup.sh
python demos/00_getting_started/00_hello_world.py
```

---

## Documentation

**User Guides**:

- [Quick Start](docs/getting-started/quick-start.md) - Installation and first steps
- [Black-Box Analysis](docs/user-guide/workflows/blackbox-analysis.md) - Unknown protocol RE
- [Side-Channel Analysis](docs/user-guide/workflows/side-channel-analysis.md) - ChipWhisperer integration
- [Hardware Acquisition](docs/user-guide/workflows/hardware-acquisition.md) - Instrument control
- [Complete Workflows](docs/user-guide/workflows.md) - End-to-end pipelines

**API Reference**: [docs/api/](docs/api/) - Complete function reference

**Development**: [Architecture](docs/developer-guide/architecture.md) | [Testing](docs/testing/) | [CHANGELOG](CHANGELOG.md)

---

## Contributing

```bash
git clone https://github.com/oscura-re/oscura.git
cd oscura
./scripts/setup.sh                    # Setup with hooks
./scripts/check.sh                    # Quality checks
./scripts/test.sh                     # Test suite
python3 .claude/hooks/validate_all.py # Must pass 5/5
```

**Needed Contributions**:

- Workflow automation (new pipelines, export formats, integrations)
- File format loaders (oscilloscope/LA formats not yet supported)
- Inference algorithms (better state machine learning, field detection)
- Protocol decoders (proprietary protocols you've reversed)
- Hardware integration (DAQ systems, instrument drivers)
- Real-world validation (test on your captures, report issues)

[Contributing Guide](CONTRIBUTING.md) | [Architecture](docs/developer-guide/architecture.md)

---

## Project Status

**Version**: [0.12.0](https://github.com/oscura-re/oscura/releases/latest) (2026-02-03)

**Active Development**: Hypothesis-driven RE workflows, automotive protocol analysis (CAN-FD, J1939, OBD-II, UDS), unknown protocol inference (state machines, field detection, CRC recovery), multi-format loading, export automation

**Stability**: Production-ready for security research, right-to-repair, academic use. API evolution documented in [CHANGELOG](CHANGELOG.md).

[Release History](https://github.com/oscura-re/oscura/releases) | [Roadmap](https://github.com/oscura-re/oscura/discussions)

---

## Legal

**License**: [MIT License](LICENSE) - Permissive use, modification, distribution

**Disclaimer**: Intended for legitimate security research, right-to-repair, academic study, and authorized testing. Users responsible for compliance with applicable laws. Unauthorized access is illegal and unethical.

**Dependencies**: Python, NumPy, SciPy, Matplotlib, scikit-learn, cantools, Hypothesis. See [pyproject.toml](pyproject.toml).

---

## Citation

```bibtex
@software{oscura2026,
  title = {Oscura: Hardware Reverse Engineering Framework},
  author = {Oscura Contributors},
  year = {2026},
  url = {https://github.com/oscura-re/oscura},
  version = {0.12.0}
}
```

Machine-readable: [CITATION.cff](CITATION.cff)

---

**Oscura** - Hardware reverse engineering framework providing unified workflows from signal capture to protocol specification. Built for security research, right-to-repair, academic study, and defense applications.
