Metadata-Version: 2.1
Name: mass_driver
Version: 0.17.1
Summary: Send bulk repo change requests
Home-page: https://github.com/OverkillGuy/mass-driver
License: GPL-3.0-or-later
Keywords: repo-automation
Author: Jb Doyon
Author-email: jb@jiby.tech
Requires-Python: >=3.11,<4.0
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Software Development :: Code Generators
Requires-Dist: GitPython
Requires-Dist: PyGithub (==2.1.1)
Requires-Dist: pydantic (==1.*)
Project-URL: Documentation, https://jiby.tech/mass-driver
Project-URL: Repository, https://github.com/OverkillGuy/mass-driver
Description-Content-Type: text/markdown

# Mass Driver

![PyPI](https://img.shields.io/pypi/v/mass-driver)
![PyPI - License](https://img.shields.io/pypi/l/mass-driver)

Send bulk repo change requests.

This repository is on Github:
[Overkillguy/mass-driver](https://github.com/OverkillGuy/mass-driver/).

Requires Python 3.11.

See also the docs at [jiby.tech/mass-driver/](https://jiby.tech/mass-driver/)

## Installation

Install the package:

```none
pip install mass-driver
```

We recommend you install CLIs via [pipx](https://pypa.github.io/pipx/), for
dependency isolation:

```none
pipx install mass-driver
```

If you want to install from a git branch rather than Pypi:

```none
pipx install https://github.com/OverkillGuy/mass-driver
```

See pipx docs: <https://pypa.github.io/pipx/#running-from-source-control>

## Running the tool

Use the help menu to start with:

```none
mass-driver --help
```

## Preparing a change

Let's prepare for doing a change over dozens of repositories. We'll need to find
a `PatchDriver` that suits our needs, and configure it accordingly.

List available `PatchDriver`s via:

```none
mass-driver drivers --list
# The docs for a single driver:
mass-driver driver --info counter
```

Remember, `PatchDriver`s are exposed via a python plugin system, which means
anyone can package their own!

Once you've got a driver, you should create a Migration file, in TOML:

```toml
# Saved as "fix_teamname.toml"
[mass-driver.migration]
# As seen in 'git log':
commit_message = """Change team name

Team name XYZ is wrong, we should be called ABC instead.
See JIRA-123[1].

[1]: https://example.com/tickets/JIRA-123
"""

# Override the local git commit author
commit_author_name = "John Smith"
commit_author_email = "smith@example.com"

branch_name = "fix-team-name"

# PatchDriver class to use.
# Selected via plugin name, from "massdriver.drivers" entrypoint
driver_name = "teamname-changer"

# Config given to the PatchDriver instance
driver_config = { filename = "catalog.yaml", team_name = "Core Team" }

# Note: No "forge" section = no forge activity to pursue (no PR will be created)
```

With this file named `fix_teamname.toml` in hand, we can apply the change
locally, either against a local repo we've already cloned:

```shell
mass-driver run fix_teamname.toml --repo-path ~/workspace/my-repo/
```

Or against a repo being cloned from URL:

```shell
mass-driver run fix_teamname.toml --repo-path 'git@github.com:OverkillGuy/sphinx-needs-test.git'
```

The cloned repo will be under `.mass_driver/repos/USER/REPONAME/`.
We should expect a branch named `fix-team-name` with a single commit.

To apply the change over a list of repositories, create a file with relevant
repos:

```shell
cat <<EOF > repos.txt
git@github.com:OverkillGuy/sphinx-needs-test.git
git@github.com:OverkillGuy/speeders.git
EOF

mass-driver run fix_teamname.toml --repo-filelist repos.txt
```

## Creating PRs

Once the commits are done locally, let's send them up as PR a second step. For
this, we'll be creating a second activity file containing a Forge definition.

Similarly, forges can be listed and detailed:

```shell
mass-driver forges --list
# The docs for a single forge:
mass-driver forge --info counter
```

Consider using the `forge_name = "github"`.
Create a new Activity with a Forge:

``` toml
# An Activity made up of just a forge
[mass-driver.forge]
forge_name = "github"

base_branch = "main"

head_branch = "fix-teamname"
draft_pr = true
pr_title = "[JIRA-123] Bump counter.txt to 1"
pr_body = """Change team name

Team name XYZ is wrong, we should be called ABC instead.
See JIRA-123[1].

[1]: https://example.com/tickets/JIRA-123
"""

# Do you need to git push the branch before PR?
git_push_first = true
```

Now run mass-driver, remembering to set the `FORGE_TOKEN` envvar for a
Github/other auth token.

``` shell
export FORGE_TOKEN="ghp_supersecrettoken"
mass-driver run fix_teamname_forge.toml --repo-filelist repos.txt
```

## Combining migration then forge

Sometimes, we wish to expedite both the committing and the PR creation in a
single move.

The Activity file can contain both sections:

``` toml
# An activity made up of first a Migration, then a Forge
[mass-driver.migration]
# As seen in 'git log':
commit_message = """Change team name

Team name XYZ is wrong, we should be called ABC instead.
See JIRA-123[1].

[1]: https://example.com/tickets/JIRA-123
"""

# Override the local git commit author
commit_author_name = "John Smith"
commit_author_email = "smith@example.com"

branch_name = "fix-team-name"

# PatchDriver class to use.
# Selected via plugin name, from "massdriver.drivers" entrypoint
driver_name = "teamname-changer"

# Config given to the PatchDriver instance
driver_config = { filename = "catalog.yaml", team_name = "Core Team" }

# And a forge = PR creation after Migration
[mass-driver.forge]
forge_name = "github"

base_branch = "main"

head_branch = "fix-teamname"
draft_pr = true
pr_title = "[JIRA-123] Bump counter.txt to 1"
pr_body = """Change team name

Team name XYZ is wrong, we should be called ABC instead.
See JIRA-123[1].

[1]: https://example.com/tickets/JIRA-123
"""

# Do you need to git push the branch before PR?
git_push_first = true

```

## Discovering repos using a Source

Sometimes, the repos we want to apply patches to is a dynamic thing, coming from
tooling, like a Github repository search, some compliance tool report, service
catalogue, etc.

To address this, mass-driver can use a Source plugin to discover repositories to
apply activities to.

``` toml
# An Activity file with a file-list Source
[mass-driver.source]
source_name = "repo-list"
# Source 'repo-list' takes a 'repos' list of cloneable URLs:
[mass-driver.source.source_config]
repos = [
  "git@github.com:OverkillGuy/mass-driver.git",
  "git@github.com:OverkillGuy/mass-driver-plugins.git",
]
```

Because we included a `Source`, we can omit the CLI flags `--repo-path` or
`--repo-filelist`, to instead rely on the activity's config to discover the
repos.

``` shell
mass-driver run activity.toml
```

Smarter Sources can use more elaborate parameters, maybe even secret parameters
like API tokens.

Note that to pass secrets safely at runtime, config parameters passed via
`source_config` in file format can be passed as envvar, using prefix `SOURCE_`.
So we could have avoided the `repos` entry in file, by providing a
`SOURCE_REPOS` envvar instead. This feature works because the Source class
derives from `Pydantic.BaseSettings`.

As a `Source` developer, though, you should really look into usage of
`Pydantic.SecretStr` to avoid leaking the secret when config or result is
stored. See [Pydantic docs on Secret
fields](https://docs.pydantic.dev/1.10/usage/types/#secret-types).

## Using the scanners

Before doing any actual migration, we might want to explore existing
repositories to see what kind of change is required.

Mass-driver provides for this usecase via the scanners plugin system, enabling a
simple python function to be run against many repos, with the purpose of
gathering information.

<!-- scanner-activity -->
Let's define an Activity file specifying a list of scanners to run:

``` toml
# An Activity file for scanning
[mass-driver.scan]
scanner_names = ["root-files", "dockerfile-from"]
```

This can be run just like a migration:

``` shell
mass-driver run scan.toml --repo-filelist repos.txt
```

## Reviewing bulk PR status

Have a look at the `view-pr` subcommand for reviewing the status of many PRs at
once.

It requires specifying a forge like `github`, along with setting any required
tokens, such as via `FORGE_TOKEN` envvar for `github` forge.

``` shell
export FORGE_TOKEN=xyz
mass-driver view-pr github \
    --pr \
    https://github.com/OverkillGuy/mass-driver/pull/1 https://github.com/OverkillGuy/mass-driver/pull/2
# Can specify multiple PRs as a args list
```

Equivalently via a file full of newline-delimited PR URLs

``` shell
export FORGE_TOKEN=xyz
mass-driver view-pr github --pr-filelist prs.txt
```

With sample result:

```none
> Pull request review mode!
[001/004] Fetching PR status...
[002/004] Fetching PR status...
[003/004] Fetching PR status...
[004/004] Fetching PR status...

Merged:
https://github.com/OverkillGuy/mass-driver/pull/1
https://github.com/OverkillGuy/sphinx-needs-test/pull/1

Closed (but not merged):
https://github.com/OverkillGuy/mass-driver/pull/2
https://github.com/OverkillGuy/sphinx-needs-test/pull/2

In summary: 4 unique PRs, of which...
- 002 (50.0%) merged
- 002 (50.0%) closed (but not merged)
```

## Development

### Python setup

This repository uses Python3.11, using
[Poetry](https://python-poetry.org) as package manager to define a
Python package inside `src/mass_driver/`.

`poetry` will create virtual environments if needed, fetch
dependencies, and install them for development.

For ease of development, a `Makefile` is provided, use it like this:

``` shell
make  # equivalent to "make all" = install lint docs test build
# run only specific tasks:
make install
make lint
make test
# Combine tasks:
make install test
```

Once installed, the module's code can now be reached through running
Python in Poetry:

``` shell
$ poetry run python
>>> from mass_driver import main
>>> main("blabla")
```

This codebase uses [pre-commit](https://pre-commit.com) to run linting
tools like `ruff`. Use `pre-commit install` to install git
pre-commit hooks to force running these checks before any code can be
committed, use `make lint` to run these manually. Testing is provided
by `pytest` separately in `make test`.

### Documentation

Documentation is generated via [Sphinx](https://www.sphinx-doc.org/en/master/),
using the cool [myst_parser](https://myst-parser.readthedocs.io/en/latest/)
plugin to support Markdown files like this one.

Other Sphinx plugins provide extra documentation features, like the recent
[sphinx-autodoc2](https://sphinx-autodoc2.readthedocs.io/en/latest/index.html)
to generate API reference without headaches, and with myst-markdown support in
docstrings too!

To build the documentation, run

``` shell
# Requires the project dependencies provided by "make install"
make docs
# Generates docs/build/html/
```

To browse the website version of the documentation you just built, run:

``` shell
make docs-serve
```

And remember that `make` supports multiple targets, so you can generate the
documentation and serve it:

``` shell
make docs docs-serve
```

## License

This project is released under GPLv3 or later. See `COPYING` file for GPLv3
license details.

### Templated repository

This repository was created by the copier template available at
gh:OverkillGuy/python-template, using version v1.3.1.

