Metadata-Version: 2.4
Name: bitp
Version: 1.1.18
Summary: Tools for BitBake/Yocto projects
Author-email: Bruce Ashfield <bruce.ashfield@gmail.com>
License: GPL-2.0-only
Project-URL: Homepage, https://gitlab.com/bruce-ashfield/bitbake-project
Project-URL: Repository, https://gitlab.com/bruce-ashfield/bitbake-project
Project-URL: Issues, https://gitlab.com/bruce-ashfield/bitbake-project/-/issues
Keywords: bitbake,yocto,openembedded,layers,git
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU General Public License v2 (GPLv2)
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: Software Development :: Embedded Systems
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: COPYING
Provides-Extra: completion
Requires-Dist: argcomplete>=2.0; extra == "completion"
Provides-Extra: dev
Requires-Dist: argcomplete>=2.0; extra == "dev"
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Requires-Dist: pytest-mock>=3.0; extra == "dev"
Dynamic: license-file

# bitp

A CLI tool for managing BitBake/Yocto/OpenEmbedded layer repositories.

The command is `bit` for quick typing.

## Overview

`bit` helps developers working with Yocto/OpenEmbedded projects by providing tools to:

- **Update** git repos backing BitBake layers from a `bblayers.conf`
- **Explore** commits interactively with fzf-based navigation
- **Export** patches with cover letters for upstream submission
- **Manage branches** across multiple layer repositories
- **Search** the OpenEmbedded Layer Index for layers
- **Bootstrap** new projects by cloning core repositories
- **Manage projects** - switch between multiple Yocto builds from anywhere

## Features

- **Automatic layer discovery** - finds layers by searching for `conf/layer.conf` files
- **Multi-layer repo support** - handles repos containing multiple layers (e.g., meta-openembedded)
- **Interactive fzf menus** - fast navigation with preview panes and keyboard shortcuts
- **Tab completion** - bash completion via argcomplete
- **Background refresh** - upstream status checks run in background for fast startup
- **Per-repo configuration** - custom display names, update defaults, push targets

## Quick Start

```bash
# Create a new Yocto project
bit create -b scarthgap --execute
bit setup

# Check status of all layer repos
bit status

# Interactively explore repos and commits
bit explore

# Update all repos (interactive)
bit update

# Update all repos non-interactively (apply saved defaults)
bit update -y

# Search for layers in the OE Layer Index
bit layer-index security
```

## Screenshots

### Project Dashboard

Manage multiple Yocto/OE builds from a single interface. Switch between projects, launch explore/update/config, or open a build shell.

![Dashboard](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/dashboard.png)

### Explore Repos

Browse layer repositories with upstream status, branch info, and dirty/clean indicators. Yocto projects get extra keybindings for recipes, deps, fragments, and build info.

![Explore](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/explore.png)

Expand multi-layer repos to see individual layers:

![Explore Expanded](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/explore-expanded.png)

### Recipe Browser

Search and browse BitBake recipes across layers. Expand recipes to see dependencies inline.

![Recipes](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/recipes.png)

![Recipe Dependencies](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/recipes-deps.png)

### Commit Browser

Drill into a repo to browse commits with preview panes, mark ranges for export, and launch git tools.

![Commits](https://gitlab.com/bruce-ashfield/bitbake-project/-/raw/main/docs/images/commits.png)

## Commands

| Command | Alias | Description |
|---------|-------|-------------|
| `explore` | `x` | Interactively explore commits in layer repos |
| `update` | `u` | Update git repos referenced by layers |
| `status` | - | Show local commit summary for layer repos |
| `info` | `i` | Show build configuration and layer status |
| `config` | `c` | View and configure repo/layer settings |
| `branch` | `b` | View and switch branches across repos |
| `export` | - | Export patches from layer repos |
| `repos` | - | List layer repos |
| `create` | - | Create a new Yocto/OE project (clone core repos) |
| `setup` | - | Show OE/Yocto build environment setup command |
| `setup shell` | - | Start shell with OE build environment pre-sourced |
| `setup clone` | - | Clone repos from a `.conf.json` config or registry |
| `setup apply` | - | Apply layers and fragments from saved config |
| `setup configs` | - | Browse and manage saved build configurations |
| `setup export` | - | Export current build state as `.conf.json` |
| `setup registry-copy` | - | Copy a `.conf.json` file to the user registry |
| `b4` | - | Mail-based patch management using b4 and lore.kernel.org |
| `patches` | - | Explore and manage patches across layers |
| `deps` | - | Show layer and recipe dependencies |
| `recipes` | - | Search and browse BitBake recipes |
| `fragments` | - | Browse and manage OE configuration fragments |
| `layer-index` | - | Search OpenEmbedded Layer Index for layers |
| `projects` | `p` | Manage and switch between multiple project directories |
| `serve` | - | Start browser-based web dashboard server |

Run `bit` with no arguments for an interactive command menu, or `bit --help` for detailed usage.

## Common Workflows

### Starting a New Project

There are several ways to create a new project, depending on whether you have a saved build configuration.

#### Basic clone (no config file)

Clone the three core repos (bitbake, openembedded-core, meta-yocto) and set up the build environment:

```bash
# Clone core repos at a release branch
bit setup clone -b scarthgap --doit

# Show the environment setup command
bit setup

# Or jump straight into a sourced shell
bit setup shell
```

#### Clone from a saved configuration

Build configurations (`.conf.json` files) capture repos, layers, branches, and OE fragments. Clone from one to reproduce a build:

```bash
# Clone from a specific config file
bit setup clone --config poky.conf.json --doit

# Or browse the config registry to pick one interactively
bit setup clone --config --doit

# Apply layers and fragments to the build directory
bit setup apply
```

When `--config` is passed without a filename, a **registry picker** opens showing all available configs from the built-in and user registries (`~/.config/bit/registry/`). Each entry shows its type (`[Built-in]`, `[User]`) and a preview of included repos, layers, and fragments.

`setup apply` creates `build/conf/` if missing, adds layers to `bblayers.conf`, and applies OE configuration fragments. For one-of fragment categories (e.g., machine, distro where only one choice makes sense), an interactive picker prompts you to choose which fragment to enable.

#### From the dashboard

The dashboard detects empty or new directories and offers to create a project:

1. Open `bit projects`
2. Press `+` to add a directory (browse, enter a path, or use current directory)
3. If the directory is empty or doesn't exist, a **config registry picker** opens:
   - Shows all saved configs with `[Built-in]`, `[User]`, and `[Basic]` labels
   - Preview pane shows config details (repos, layers, fragments)
   - Select a config to clone its repos into the new project
   - Or choose "Basic clone" to get the three core repos (bitbake, oe-core, meta-yocto) without a config file
   - Esc skips config selection and registers an empty project
4. After cloning, `setup apply` chains automatically:
   - Creates `build/conf/` if missing (with confirmation prompt)
   - Adds layers from the config to `bblayers.conf`
   - **Fragment selection**: OE configuration fragments from the config are applied. For one-of categories (e.g., machine, distro), an interactive picker prompts you to choose which fragment to enable
   - Validates the setup and marks the config as applied
5. If apply is declined or interrupted, the project shows `⚠ setup` in the dashboard — expand it and select "setup" to resume later

If the directory already has content, it's added as an existing project (no config picker).

### Saving and Sharing Build Configurations

#### Export current build state

```bash
# Export to a .conf.json file in the current directory
bit setup export

# Export with a custom filename
bit setup export my-build.conf.json

# Export and save to the user registry for reuse
bit setup export --registry

# Export a different registered project
bit setup export --project my-other-build
```

The export captures git remotes, branches, layer paths, and enabled OE fragments.

The dashboard also supports export: expand a project and select "export" to open an inline dialog with fragment checkboxes (`[x]`/`[ ]`) to toggle which fragments are included, an optional description field, and actions to export to a file or save to the registry.

#### Manage the config registry

```bash
# Copy a config file into the registry
bit setup registry-copy poky.conf.json

# Copy with a custom name
bit setup registry-copy poky.conf.json --as my-custom-build

# Browse and manage all saved configs
bit setup configs
```

The registry browser (`bit setup configs`) provides:
- Preview pane showing config details (repos, layers, fragments)
- Actions: Clone, Apply, Copy, Describe, Edit, Rename, Delete
- Built-in configs are read-only (Clone, Apply, Copy only)
- User configs support all actions including inline rename and describe

### Build Environment Shell

```bash
# Show the source command for oe-init-build-env
bit setup

# Start a shell with the build environment pre-sourced
bit setup shell
```

`setup shell` starts a new shell with the OE build environment already sourced, so `bitbake` and related tools are immediately available.

### Daily Development

```bash
# Check what's changed upstream
bit status

# Update repos (interactive)
bit update

# Update all repos non-interactively (apply saved defaults)
bit update -y

# Or update a specific repo
bit update OE-core
```

### Exploring Commits

```bash
# Interactive two-level browser
bit explore

# Jump directly to a repo
bit x OE-core
```

Keybindings in explore mode (repo list):
- `Enter` - explore commits in selected repo
- `u` - pull --rebase
- `m` - pull (merge)
- `r` - refresh repo (fetch)
- `R` - refresh all repos
- `B` - switch all repos to branch (common branches first, partial dimmed)
- `t` - launch git history viewer (tig/lazygit/gitui/gitk)
- `v` - toggle verbose display (HEAD commit per repo/layer)
- `\` - expand/collapse all multi-layer repos
- `c` - open config menu
- `P` - pull patches from remote project (browse remote commits, import)
- `X` - prune all backup branches in selected repo (with confirmation)
- `M` - open message log viewer (see [Status Messages](#status-messages))
- `q` - quit

Yocto-specific keybindings (explore mode, Yocto projects only):
- `i` - show build info (same as `bit info`)
- `Ctrl-R` - browse recipes (scoped to selected repo's layers, or single layer if expanded)
- `D` - show dependencies (scoped to selected layer)
- `f` - browse configuration fragments

Keybindings in commit browser:
- `Tab` - mark commit for selection
- `Space` - select range of commits
- `?` - toggle preview pane
- `d` - toggle diff view in preview (stat ↔ full patch)
- `g` - toggle graph view in preview (git log --graph branch topology)
- `a` - toggle blame view in preview
- `f` - switch to file/tree view (browse files, expand commit history per file)
- `c` - copy commit hash
- `e` - export selected commits
- `i` - interactive rebase (select range, then squash/reword/reorder)
- `t` - launch git history viewer
- `l` - browse commits grouped by layer (multi-layer repos)
- `p` - find patch source on lore.kernel.org
- `PgUp`/`PgDn` - scroll commit list
- `Alt-Up`/`Alt-Down` - scroll preview page
- `Ctrl-U`/`Ctrl-D` - scroll preview half-page
- `←` or `b` - go back
- `q` - quit

Keybindings in file/tree view (`f` from commit browser):
- `Enter`/`→` - expand directory or show file's commit history
- `←` - collapse directory/file or go to parent
- `\` - toggle expand/collapse directory or file commits
- `d` - toggle diff view in preview
- `a` - toggle blame view
- `f` - toggle raw file content view
- `?` - toggle preview pane
- `b`/`Esc` - go back to commit browser
- `q` - quit

### Exporting Patches for Upstream

```bash
# Prepare commits (reorder for upstream)
bit export prep --branch zedd/feature

# Export patches
bit export --target-dir ~/patches

# Export with version number
bit export --target-dir ~/patches -v 2
```

### Searching for Layers

```bash
# Interactive search
bit layer-index

# Search with query
bit layer-index virtualization

# Get layer info for scripting
bit layer-index -i meta-virtualization

# Clone a layer directly
bit layer-index -c meta-security -b scarthgap
```

### Managing Branches

```bash
# Interactive branch management
bit branch

# Switch all repos to a release branch
bit branch --all scarthgap
```

Keybindings in branch management (`bit branch`):
- `Enter`/`→` - expand repo to show branches / switch to selected branch
- `←` - collapse expanded repo
- `B` - switch all repos to branch (common branches first, partial dimmed)
- `q` - quit

### Viewing Build Info

```bash
# Show build configuration (like BitBake summary)
bit info

# Show only layer info with branch:commit
bit info layers

# Show only key variables
bit info vars
```

Output format matches BitBake's build configuration:
```
Build Configuration:
  MACHINE = qemuarm64
  DISTRO  = poky

Layers:
  meta                = "master:01a65e8d5f73..."
  meta-selftest
  meta-poky           = "master:de4abc0a175a..."
```

The `info` entry is also available in the explore repo list menu for interactive browsing.

### Managing Multiple Projects

```bash
# Open project manager (interactive)
bit projects

# Add a project directory
bit projects add /path/to/yocto-build

# List known projects
bit projects list

# Remove a project from tracking
bit projects remove /path/to/old-project
```

Keybindings in projects menu:
- `Space` - activate selected project (all commands will operate on it)
- `Enter`/`→` - expand project; on already-expanded project, runs configurable default command (default: explore)
- `?`/`m` - open commands menu
- `+` - browse for a new directory to add
- `-` - remove selected project from tracking
- `n` - rename project
- `u` - update repos in project
- `x` - explore repos in project
- `c` - open config menu with inline sub-pickers for dir browser, git viewer, preview layout, recipe scan, default action, and backup prune age
- `S` - open shell in project
- `M` - open message log viewer (see [Status Messages](#status-messages))
- `q` - quit

Note: When no valid project context exists (no bblayers.conf found), the projects picker is automatically shown. Persona is auto-detected when adding projects (Yocto projects detected by bblayers.conf, layer.conf files, or poky directory; generic projects detected by .git directories without Yocto markers).

### Web Dashboard

```bash
# Start the web dashboard (opens browser automatically)
bit serve

# Custom port and host
bit serve --port 9000 --host 0.0.0.0

# Don't open browser (e.g. for remote servers)
bit serve --no-browser
```

The web dashboard provides a browser-based interface with the same functionality as the terminal UI: projects, repos, branches, updates, config, patches, recipes, and more. It also includes an integrated terminal with tmux session sharing — the web terminal and TUI share the same tmux session, so you can seamlessly switch between them.

The web server can also be launched from the TUI dashboard, where it runs as a background daemon thread.

## Configuration

Per-repo settings are stored in `.bit.defaults` (JSON):

```json
{
  "/path/to/poky": "rebase",
  "/path/to/meta-oe": "skip",
  "__extra_repos__": ["/path/to/bitbake"],
  "__hidden_repos__": ["/path/to/unwanted-repo"],
  "__push_targets__": {
    "/path/to/oe-core": {
      "push_url": "ssh://git@push.openembedded.org/oe-core-contrib",
      "branch_prefix": "yourname/"
    }
  },
  "fzf_theme": "dark",
  "fzf_text_color": "light-gray",
  "fzf_custom_colors": {"pointer": "green"}
}
```

Configure interactively with `bit config`.

### Theme Customization

The tool supports customizable options via Settings menu (`bit config` -> Settings):

**Colors** submenu:
- **Mode** - Color source: global (`~/.config/bit/colors.json`), per-project (`.bit.defaults`), or built-in defaults
- **Theme** - Base color scheme (default, dark, light, dracula, nord, etc.)
- **Individual** - Per-element color overrides with live preview

New projects default to **global** mode — changing colors in any project updates `~/.config/bit/colors.json`, shared by all projects in global mode. Switch to **custom** for per-project overrides.

A color preview panel shows all themed elements (pointer, header, prompt, etc.) with current colors applied.

**Directory Browser** - Choose preferred file browser for project selection:
- broot (recommended)
- ranger
- nnn
- fzf (fallback)

**Git Viewer** - Choose preferred git history viewer:
- auto (detect first available: tig > lazygit > gitui > gitk)
- tig - ncurses git interface
- lazygit - terminal UI for git
- gitui - blazing fast terminal git UI (Rust)
- gitk - graphical git browser

**Preview Layout** - Configure commit browser preview pane position:
- Bottom (default) - preview below commit list
- Right (side-by-side) - preview beside commit list
- Top - preview above commit list

**Default Action** - Configure what Enter/→ does on expanded projects:
- Explore (default), Shell, Recipes, Update, Config

**Backup Prune Age** - Auto-prune old backup branches on commit browser entry:
- Always (0), 1/3/7/14/30 days, or Never (default: 7 days)
- Backup branches (`*-backup-YYYYMMDD-HHMMSS`) are created by the drop commits feature

**Terminal Colors** - Configure output colors for:
- Upstream indicator (commits to pull)
- Local commit count
- Dirty/clean status
- Repo names (configured, discovered, external)

## Visual Indicators

Repos are color-coded by source:
- **Green** - from bblayers.conf
- **Magenta** with `(?)` - discovered layers (not in bblayers.conf)
- **Cyan** with `(ext)` - external repos (git repos without conf/layer.conf)

Status indicators:
- `5 local` - commits ahead of upstream tracking ref
- `↓ 3` - commits to pull from upstream
- `→ contrib` - tracking a non-origin remote (shown when not tracking origin)
- `[clean]` / `[DIRTY]` - working tree status

### Status Messages

Actions like activating a project, refreshing repos, or running updates produce
status messages (e.g. `✓ Activated: poky`, `Refreshing OE-core... done`). These
are captured silently — they never print to the terminal — and are available in
two ways:

- **Header indicator**: The most recent message appears above the keybinding
  help for one fzf iteration, then disappears on the next redraw.
- **Message log** (`M` key): Opens a scrollable popup showing all messages from
  the current session with timestamps (newest first). Available in both the
  dashboard and explore menus.

### Remote SSH Projects

Remote projects on SSH-accessible build machines can be managed alongside local
ones:

- Add via manage menu (`+` → Remote SSH) with `user@host:/path` URI format
- Dashboard shows remote projects with `⇄ hostname` indicator
- Remote summary (branch, dirty, ahead/behind) gathered via single batched SSH call
- Actions dispatch to remote bit (auto-deployed if not installed) with fallback to raw git
- SSH connection multiplexing (`ControlMaster=auto`, `ControlPersist=600`) avoids repeated auth
- All SSH calls suppress askpass prompts (`SSH_ASKPASS_REQUIRE=never`, `BatchMode=yes`) to
  prevent password dialogs from interfering with the TUI
- Interactive SSH sessions (explore, shell) are preceded by a connection pre-check;
  if auth fails, a clean error message is shown instead of hanging
- **Pull patches** (`P` key in explore repos view): browse commits on a remote build machine and import them
  - Automatically finds matching repo on remote by basename
  - Commit browser shows provenance tags: `[build4]` (remote-only), `[upstream][build4]` (shared upstream), `[upstream][build4][local]` (shared base)
  - Mark commits with Tab/Space, press `e` to export
  - `t` launches history viewer on local repo, `g` toggles remote graph in preview
  - Import methods: apply patches (git am), fetch branch (git fetch), or save .patch files

## Requirements

- Python 3.9+
- Git
- fzf (optional, but recommended for interactive features)
- argcomplete (optional, for tab completion)
- xclip or xsel (optional, for clipboard support)

## Development

### Installation for Development

```bash
# Clone the repository
git clone https://gitlab.com/bruce-ashfield/bitbake-project.git
cd bitbake-project

# Create a virtual environment
python3 -m venv .venv
source .venv/bin/activate

# Install with dev dependencies
pip install -e .[dev]
```

### Running Tests

The project uses pytest with pytest-mock and pytest-cov:

```bash
# Activate virtual environment first
source .venv/bin/activate

# Run all tests
pytest tests/ -v

# Run with coverage report
pytest tests/ --cov=bitbake_project --cov-report=term-missing

# Run only unit tests
pytest tests/unit/ -v

# Run only integration tests
pytest tests/integration/ -v

# Run tests matching a pattern
pytest tests/ -k "test_colors" -v
```

### Test Structure

```
tests/
├── conftest.py                    # Shared fixtures
├── unit/
│   ├── test_colors.py            # Colors class ANSI formatting
│   ├── test_helpers.py           # Pure functions (clean_title, dedupe, sort)
│   ├── test_state.py             # State management (defaults, prep, export)
│   ├── test_gitrepo.py           # GitRepo class
│   ├── test_fzf_menu.py          # FzfMenu class (mocked)
│   ├── test_bblayers_parser.py   # BblayersParser
│   ├── test_cli_parser.py        # CLI argument parsing
│   ├── test_info.py              # Info command parsing and formatting
│   └── test_update.py            # Update command (upstream tracking)
├── integration/
│   ├── test_cli_dispatch.py      # Command dispatch, --help, aliases
│   └── test_projects_command.py  # Projects add/remove/list
└── data/
    └── bblayers/                  # Test bblayers.conf files
```

## Project Structure

The project is organized as a Python package with a commands subpackage:

```
bitbake_project/
├── core.py           # Core abstractions (Colors, GitRepo, FzfMenu)
├── cli.py            # Argument parsing and main()
├── tmux.py           # Shared tmux helpers (session naming, attach command)
├── commands/         # Command implementations
│   ├── common.py     # Shared utilities (BblayersParser, etc.)
│   ├── explore.py    # explore, status commands
│   ├── export.py     # export, export prep commands
│   ├── config.py     # config command
│   ├── branch.py     # branch command
│   ├── info.py       # info command (build configuration)
│   ├── update.py     # update command (upstream tracking support)
│   ├── b4.py         # b4 command (lore, mail-based patch management)
│   ├── patches.py    # patches command (patch browser, Upstream-Status)
│   ├── search.py     # layer-index command
│   ├── setup.py      # create, setup commands
│   ├── repos.py      # repos command
│   ├── projects.py   # projects command, dashboard, directory browser
│   └── ssh_remote.py # SSH transport for remote projects
└── web/              # Browser-based dashboard (stdlib http.server)
    ├── server.py     # HTTP server with WebSocket upgrade
    ├── api.py        # REST API endpoints
    ├── terminal.py   # PTY-to-WebSocket bridge (tmux sessions)
    └── static/       # Frontend (HTML, CSS, JavaScript)
```

Distribution options:
- **Standalone**: Single-file zipapp (`dist/bit`)
- **Pip install**: `pip install -e .` for development

## License

This project is licensed under the GNU General Public License v2.0 (GPL-2.0-only).

See [COPYING](COPYING) for the full license text.

## See Also

- [INSTALL.md](INSTALL.md) - Installation instructions
- [Yocto Project](https://www.yoctoproject.org/)
- [OpenEmbedded](https://www.openembedded.org/)
- [OpenEmbedded Layer Index](https://layers.openembedded.org/)
