# nblite

**Notebook-driven Python development made simple.**

nblite is a tool for developing Python packages using Jupyter notebooks, and/or plaintext notebooks (e.g. [percent](https://jupytext.readthedocs.io/en/latest/formats-scripts.html#the-percent-format) format `.pct.py` files) as the source of truth. Write your code in notebooks, add export directives, and nblite generates clean Python modules automatically.

**Note:** `nblite` was inspired by the excellent [nbdev](https://github.com/AnswerDotAI/nbdev), with some adjustments to make it more lightweight and quality-of-life additions. Full credit of the concept and implementation of notebook-driven development using Jupyter notebooks should go to the creators of [nbdev](https://github.com/AnswerDotAI/nbdev).

## Features

- **Notebook-first development**: Write code in Jupyter notebooks with full interactivity
- **Automatic module generation**: Export marked cells to Python modules
- **Multiple formats**: Support for `.ipynb`, percent-style `.pct.py`, and standard `.py` modules
- **Smart execution**: Fill notebook outputs with parallel execution and change detection
- **Git integration**: Pre-commit hooks for auto-cleaning and validation
- **Documentation generation**: Build docs with MkDocs, Jupyter Book, or Quarto
- **Flexible pipelines**: Define custom export pipelines between code locations

## Installation

```bash
pip install nblite
```

## Quick Start

### 1. Initialize a project

```bash
mkdir myproject && cd myproject
nbl init --name mylib
```

This creates:
```
myproject/
├── nblite.toml     # Configuration file
├── nbs/            # Notebooks directory
└── mylib/          # Python package
    └── __init__.py
```

### 2. Create a notebook

```bash
nbl new nbs/core.ipynb --title "Core Module"
```

### 3. Add code with export directives

In your notebook, mark cells for export:

```python
#|default_exp core

#|export
def greet(name: str) -> str:
    """Return a greeting message."""
    return f"Hello, {name}!"

#|export
class Calculator:
    """A simple calculator."""

    def add(self, a: int, b: int) -> int:
        return a + b
```

### 4. Export to Python modules

```bash
nbl export
```

This generates `mylib/core.py`:
```python
# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core.ipynb

__all__ = ['greet', 'Calculator']

def greet(name: str) -> str:
    """Return a greeting message."""
    return f"Hello, {name}!"

class Calculator:
    """A simple calculator."""

    def add(self, a: int, b: int) -> int:
        return a + b
```

### 5. Fill notebook outputs

```bash
nbl fill
```

Executes all notebooks and saves their outputs. Uses smart change detection to skip unchanged notebooks.

## Configuration

nblite is configured via `nblite.toml`:

```toml
# Export pipeline: notebooks -> modules
export_pipeline = "nbs -> lib"

# Code locations
[cl.nbs]
path = "nbs"
format = "ipynb"

[cl.lib]
path = "mylib"
format = "module"

# Git hooks (optional)
[git]
auto_clean = true
auto_export = true

# Notebook cleaning (optional)
[clean]
remove_outputs = false
remove_execution_counts = false
```

See the [Configuration Guide](docs/configuration.md) for all options.

## Export Directives

Control what gets exported with special comments:

| Directive | Description |
|-----------|-------------|
| `#\|default_exp module_name` | Set the default export module |
| `#\|export` | Export this cell to the default module |
| `#\|exporti` | Export as internal (not in `__all__`) |
| `#\|export_to module_name` | Export to a specific module |
| `#\|hide` | Hide cell from documentation |
| `#\|eval: false` | Skip cell during execution |

See the [Directives Reference](docs/directives.md) for all directives.

## CLI Commands

| Command | Description |
|---------|-------------|
| `nbl init` | Initialize a new project |
| `nbl new` | Create a new notebook |
| `nbl export` | Run the export pipeline |
| `nbl fill` | Execute notebooks and fill outputs |
| `nbl test` | Test notebooks execute without errors |
| `nbl clean` | Clean notebooks (remove outputs/metadata) |
| `nbl convert` | Convert between notebook formats |
| `nbl from-module` | Convert Python modules to notebooks |
| `nbl prepare` | Run export, clean, fill, and readme |
| `nbl render-docs` | Generate documentation |
| `nbl preview-docs` | Preview documentation |
| `nbl info` | Show project information |
| `nbl list` | List files in code locations |
| `nbl install-hooks` | Install git hooks |
| `nbl validate` | Validate git staging state |

Use `nbl <command> --help` for detailed options.

See the [CLI Reference](docs/cli-reference.md) for complete documentation.

## Multi-Stage Pipelines

Define complex export pipelines:

```toml
# notebooks -> percent scripts -> modules
export_pipeline = """
nbs -> pcts
pcts -> lib
"""

[cl.nbs]
path = "nbs"
format = "ipynb"

[cl.pcts]
path = "pcts"
format = "percent"

[cl.lib]
path = "mylib"
format = "module"
```

This creates an intermediate representation in percent format, useful for:
- Code review (percent files are plain Python)
- Debugging export issues
- Version control of notebook content

## Git Integration

Install git hooks for automatic cleaning and export:

```bash
nbl install-hooks
```

The pre-commit hook will:
1. Clean notebooks (remove outputs if configured)
2. Run the export pipeline
3. Validate staging state

Configure in `nblite.toml`:

```toml
[git]
auto_clean = true      # Clean notebooks before commit
auto_export = true     # Run export on commit
validate_staging = true # Warn about staging issues
```

## Documentation Generation

Generate documentation from notebooks:

```bash
# Build documentation
nbl render-docs

# Preview with live reload
nbl preview-docs
```

Supported generators:
- **MkDocs** (default): `pip install mkdocs mkdocs-material mkdocs-jupyter`
- **Jupyter Book**: `pip install jupyter-book`
- **Quarto**: Install from https://quarto.org/

Configure in `nblite.toml`:

```toml
docs_cl = "nbs"           # Code location to document
docs_title = "My Project" # Documentation title
docs_generator = "mkdocs" # Generator to use

[docs]
output_folder = "_docs"
```

## Notebook Execution

Fill notebooks with outputs:

```bash
# Execute all notebooks
nbl fill

# Execute specific notebooks
nbl fill nbs/core.ipynb nbs/utils.ipynb

# Parallel execution
nbl fill --workers 8

# Test without saving (dry run)
nbl test
```

Control execution with directives:

```python
#|eval: false
# This cell is skipped during execution
expensive_computation()

#|skip_evals
# All following cells are skipped
...

#|skip_evals_stop
# Execution resumes here
```

## Converting Existing Code

Convert Python modules to notebooks:

```bash
# Single file
nbl from-module utils.py nbs/utils.ipynb

# Entire directory
nbl from-module src/ nbs/ --recursive
```

## Project Structure

A typical nblite project:

```
myproject/
├── nblite.toml          # Configuration
├── nbs/                 # Source notebooks
│   ├── 00_index.ipynb
│   ├── 01_core.ipynb
│   └── 02_utils.ipynb
├── mylib/               # Generated Python package
│   ├── __init__.py
│   ├── core.py
│   └── utils.py
├── _docs/               # Generated documentation
└── README.md            # Generated from notebook
```

## Documentation

- [Getting Started](docs/getting-started.md) - Tutorial for new users
- [Configuration Guide](docs/configuration.md) - Complete `nblite.toml` reference
- [CLI Reference](docs/cli-reference.md) - All commands and options
- [Directives Reference](docs/directives.md) - Notebook directives
- [Export Pipeline](docs/export-pipeline.md) - How export works
- [Git Integration](docs/git-integration.md) - Hooks and workflows
- [Documentation Generation](docs/documentation-generation.md) - Building docs

## Philosophy

nblite follows the **literate programming** philosophy: code and documentation live together. Notebooks are the source of truth, and Python modules are generated artifacts.

Key principles:
1. **Notebooks first**: Write and test code interactively
2. **Explicit exports**: Only marked cells become part of your library
3. **Clean separation**: Keep exploration separate from production code
4. **Reproducibility**: Fill outputs to ensure notebooks run correctly

## Contributing

Contributions are welcome! Please see our contributing guidelines.

## License

MIT License - see LICENSE file for details.

## Acknowledgments

nblite is inspired by [nbdev](https://nbdev.fast.ai/) from fast.ai, reimagined as a lightweight, focused tool for notebook-driven development.
