# Export Pipeline

The export pipeline is the core feature of nblite, transforming notebooks into Python modules. This guide explains how it works in detail.

## Concept

The export pipeline converts code from notebooks into Python modules:

```
Notebook (ipynb) → Python Module (.py)
```

Only cells marked with export directives are included in the output.

## How Export Works

### Step 1: Parse Notebooks

nblite reads each notebook and identifies:
1. The default export module (`#|default_exp`)
2. Cells with export directives (`#|export`, `#|exporti`, `#|export_to`)

### Step 2: Extract Code

For each exported cell:
1. Remove the directive comments
2. Extract the Python code
3. Identify function/class names for `__all__`

### Step 3: Generate Module

Create the Python module with:
1. Autogenerated warning header
2. `__all__` list with public names
3. Exported code in order

## Pipeline Definition

Define the pipeline in `nblite.toml`:

### Simple Pipeline (String)

```toml
export_pipeline = "nbs -> lib"
```

This exports from code location `nbs` to code location `lib`.

### Multi-Stage Pipeline

```toml
export_pipeline = """
nbs -> pcts
pcts -> lib
"""
```

This creates an intermediate representation:
1. `nbs` (ipynb) → `pcts` (percent format)
2. `pcts` (percent) → `lib` (Python modules)

### List Format

```toml
export_pipeline = [
    { from_key = "nbs", to_key = "pcts" },
    { from_key = "pcts", to_key = "lib" },
]
```

## Code Locations

Code locations define where files live and their format:

```toml
[cl.nbs]
path = "nbs"
format = "ipynb"

[cl.pcts]
path = "pcts"
format = "percent"

[cl.lib]
path = "mylib"
format = "module"
export_mode = "percent"  # or "py"
```

### Format Types

| Format | Extension | Description |
|--------|-----------|-------------|
| `ipynb` | `.ipynb` | Jupyter notebook |
| `percent` | `.pct.py` | Percent-style Python |
| `module` | `.py` | Standard Python module |

### Export Modes

When exporting to `module` format, choose the style:

**Percent mode** (default):
```python
# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core.ipynb

__all__ = ['my_func']

# %% ../nbs/core.ipynb 0
def my_func():
    pass
```

**Py mode** (clean Python):
```python
# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core.ipynb

__all__ = ['my_func']

def my_func():
    pass
```

Set with `export_mode = "py"` or `export_mode = "percent"`.

## Export Examples

### Basic Export

**Notebook (`nbs/core.ipynb`):**

```python
#|default_exp core
```

```python
#|export
def greet(name: str) -> str:
    """Return a greeting."""
    return f"Hello, {name}!"
```

```python
#|export
class Calculator:
    """A simple calculator."""

    def add(self, a: int, b: int) -> int:
        return a + b
```

**Result (`mylib/core.py`):**

```python
# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core.ipynb

__all__ = ['greet', 'Calculator']

# %% ../nbs/core.ipynb 1
def greet(name: str) -> str:
    """Return a greeting."""
    return f"Hello, {name}!"

# %% ../nbs/core.ipynb 2
class Calculator:
    """A simple calculator."""

    def add(self, a: int, b: int) -> int:
        return a + b
```

### Export with Imports

```python
#|export
import json
from pathlib import Path
from typing import Dict, List
```

Imports are exported just like other code and appear at the top of the module.

### Internal Exports

```python
#|exporti
def _helper():
    """Internal function."""
    pass
```

The function is exported but not added to `__all__`.

### Export to Different Module

```python
#|export_to mylib.helpers
def helper_func():
    """Goes to helpers module."""
    pass
```

This exports to `mylib/helpers.py` instead of the default module.

### Ordered Exports

```python
#|export_to mylib.core 10
CONSTANT = 42

#|export_to mylib.core 20
def uses_constant():
    return CONSTANT * 2
```

Lower order numbers appear first in the output.

## Module Path Resolution

The `#|default_exp` directive sets the module path:

| Directive | Output File |
|-----------|-------------|
| `#\|default_exp core` | `<lib>/core.py` |
| `#\|default_exp utils` | `<lib>/utils.py` |
| `#\|default_exp io.readers` | `<lib>/io/readers.py` |
| `#\|default_exp sub.module` | `<lib>/sub/module.py` |

Nested directories are created automatically.

## `__all__` Generation

nblite automatically generates `__all__` by analyzing exported code:

```python
#|export
def public_func():  # Added to __all__
    pass

#|export
class PublicClass:  # Added to __all__
    pass

#|export
PUBLIC_CONSTANT = 42  # Added to __all__

#|exporti
def _private():  # NOT added to __all__
    pass
```

**Result:**
```python
__all__ = ['public_func', 'PublicClass', 'PUBLIC_CONSTANT']
```

### What Gets Added

- Function definitions (`def name`)
- Class definitions (`class Name`)
- Top-level assignments (`NAME = value`)

### What Doesn't Get Added

- Names starting with `_` (by convention)
- Anything in `#|exporti` cells
- Import statements

## Cell References

Exported code includes cell references as comments:

```python
# %% ../nbs/core.ipynb 5
def my_function():
    pass
```

This helps you find the source cell when debugging.

### Reference Styles

Configure in `nblite.toml`:

```toml
[export]
cell_reference_style = "relative"  # or "absolute"
```

**Relative** (default):
```python
# %% ../nbs/core.ipynb 5
```

**Absolute:**
```python
# %% /home/user/project/nbs/core.ipynb 5
```

## Autogenerated Warning

By default, exported files include a warning:

```python
# AUTOGENERATED! DO NOT EDIT! File to edit: ../nbs/core.ipynb
```

Disable with:

```toml
[export]
include_autogenerated_warning = false
```

## Function Export (`export_as_func`)

Export an entire notebook as a callable function:

**Notebook:**

```python
#|default_exp workflows.process
#|export_as_func true
```

```python
#|top_export
import pandas as pd
```

```python
#|set_func_signature
def process(input_path: str, output_path: str) -> dict:
    ...
```

```python
#|export
df = pd.read_csv(input_path)
result = df.describe()
result.to_csv(output_path)
```

```python
#|func_return
{"status": "success", "rows": len(df)}
```

**Result (`workflows/process.py`):**

```python
# AUTOGENERATED! DO NOT EDIT!

import pandas as pd

def process(input_path: str, output_path: str) -> dict:
    df = pd.read_csv(input_path)
    result = df.describe()
    result.to_csv(output_path)
    return {"status": "success", "rows": len(df)}
```

## Running Export

### Export All

```bash
nbl export
```

### Export Specific Notebooks

```bash
nbl export nbs/core.ipynb nbs/utils.ipynb
```

### Dry Run

```bash
nbl export --dry-run
```

Shows what would be exported without making changes.

### Custom Pipeline

Override the configured pipeline with `--export-pipeline`:

```bash
# Use a different pipeline
nbl export --export-pipeline "nbs -> lib"

# Multiple rules (comma-separated)
nbl export --export-pipeline "nbs -> pts, pts -> lib"

# Reverse direction (percent to ipynb)
nbl export --export-pipeline "pts -> nbs"
```

This is useful for:
- **Running a subset**: Export only certain stages
- **Reversing direction**: Convert percent files back to ipynb
- **Testing**: Try different pipeline configurations without modifying config

**Example: Converting percent back to ipynb**

If you've edited `.pct.py` files directly, you can convert them back to `.ipynb`:

```bash
# Ensure you have a target code location for ipynb output
nbl export --export-pipeline "pts -> nbs"
```

**Note:** The code locations referenced in the custom pipeline must exist in your `nblite.toml` configuration.

## Best Practices

### 1. One Module Per Notebook

Keep notebooks focused on a single module:

```
nbs/core.ipynb    → mylib/core.py
nbs/utils.ipynb   → mylib/utils.py
nbs/io.ipynb      → mylib/io.py
```

### 2. Use Descriptive Module Names

```python
#|default_exp mylib.data.loaders  # Good
#|default_exp stuff                # Bad
```

### 3. Keep Non-Export Code Separate

```python
#|export
def production_code():
    """This goes to the module."""
    pass

# Testing/exploration (not exported)
result = production_code()
print(result)
```

### 4. Order Imports First

```python
#|export
import pandas as pd
import numpy as np

#|export
def uses_imports():
    return pd.DataFrame()
```

### 5. Document as You Go

```python
#|export
def complex_function(data: list) -> dict:
    """
    Process data with the XYZ algorithm.

    This function implements the algorithm described in
    Smith et al. (2023).

    Args:
        data: Input data list

    Returns:
        Processed results as a dictionary

    Example:
        >>> complex_function([1, 2, 3])
        {'sum': 6, 'mean': 2.0}
    """
    pass
```

## Troubleshooting

### "No default_exp found"

Every notebook needs a `#|default_exp` directive:

```python
#|default_exp mymodule
```

### Exports Not Appearing

Check that:
1. Cell has `#|export` directive
2. Directive is at the top of the cell
3. Notebook is in a code location in the pipeline

### Wrong Output Location

Verify your configuration:

```toml
export_pipeline = "nbs -> lib"

[cl.nbs]
path = "nbs"      # Source directory
format = "ipynb"

[cl.lib]
path = "mylib"    # Output directory
format = "module"
```

### Circular Dependencies

If module A imports from B and B imports from A:
1. Consider combining into one module
2. Use `#|export_to` to control ordering
3. Move shared code to a third module
