# ACC CLI — Databricks Project Generator

[![PyPI version](https://img.shields.io/pypi/v/acc-cli.svg)](https://pypi.org/project/acc-cli/)
[![Python](https://img.shields.io/pypi/pyversions/acc-cli.svg)](https://pypi.org/project/acc-cli/)

A command-line tool to scaffold production-ready Databricks pipeline projects from templates. Built with [Typer](https://typer.tiangolo.com/) and [Cookiecutter](https://cookiecutter.readthedocs.io/).

---

## Table of Contents

- [Features](#features)
- [Project Structure](#project-structure)
- [Installation](#installation)
- [Usage](#usage)
- [Available Templates](#available-templates)
- [Storage Options](#storage-options)
- [Development Setup](#development-setup)
- [Publishing a New Version](#publishing-a-new-version)
- [Versioning Convention](#versioning-convention)

---

## Features

- 🚀 Scaffold a Databricks ETL pipeline project in seconds
- 📁 Generates a fully structured Python package with `src/` layout
- ☁️ Supports multiple storage backends: **UC**, **DBFS**, **S3**, **ADLS**
- 🔧 Includes `Makefile`, `pyproject.toml`, configs, Databricks job JSON, and tests out of the box
- 📦 Installable via `pip` — works as a global CLI tool

---

## Project Structure

```
python-deployment-package/
├── .gitignore
├── README.md
├── pyproject.toml           # Package metadata & entry point
├── requirements.txt         # Runtime dependencies
├── dist/                    # Built distributions (wheel + sdist)
├── docs/
│   ├── cli-spec.md
│   ├── project-structure.md
│   └── tech-decisions.md
├── src/
│   └── acc_cli/             # Main CLI package
│       ├── __init__.py
│       ├── cli.py           # Typer app & entry point
│       ├── config.py        # Templates & storage config
│       ├── commands/
│       │   ├── __init__.py
│       │   └── init.py      # `acc init` command
│       ├── utils/
│       │   ├── __init__.py
│       │   └── utils.py     # Input collection, cookiecutter wrapper
│       └── templates/
│           └── etl/         # ETL pipeline cookiecutter template
│               ├── cookiecutter.json
│               └── {{cookiecutter.project_slug}}/
│                   ├── Makefile
│                   ├── pyproject.toml
│                   ├── README.md
│                   ├── requirements.txt
│                   ├── requirements-dev.txt
│                   ├── .gitignore
│                   ├── configs/
│                   │   ├── dev.yaml
│                   │   └── prod.yaml
│                   ├── jobs/
│                   │   └── databricks_job.json
│                   ├── src/{{cookiecutter.project_slug}}/
│                   │   ├── main.py
│                   │   ├── pipelines/
│                   │   ├── tasks/
│                   │   └── utils/
│                   └── tests/
└── tests/
    ├── __init__.py
    └── test_cli.py
```

---

## Installation

```bash
pip install acc-cli
```

> **Note (macOS):** If `acc` is not found after install, add Python's bin directory to your PATH:
> ```bash
> echo 'export PATH="/Library/Frameworks/Python.framework/Versions/3.12/bin:$PATH"' >> ~/.zshrc
> source ~/.zshrc
> ```

---

## Usage

### Start the scaffolder (default command)

```bash
acc
```

### Explicit init subcommand

```bash
acc init
```

### Help

```bash
acc --help
acc init --help
```

### Interactive prompts

When you run `acc init`, you will be prompted for:

| Prompt | Description | Default |
|--------|-------------|---------|
| Project name | Name of your new project | — |
| Version | Initial version | `0.1.0` |
| Author | Your name | — |
| Template | Project template to use | `etl` |
| Storage | Cloud storage backend | `uc` |
| Databricks workspace URL | Your workspace URL | — |
| Output directory | Where to create the project | current directory |

### Example session

```
ACC Project Scaffolder

Available templates:
  etl        — Extract-Transform-Load pipeline
  ml         — Machine learning pipeline
  utility    — Utility / helper library

Project name: my-sales-pipeline
Version [0.1.0]:
Author: Jane Doe
Template [etl / ml / utility] [etl]:
Storage [uc / dbfs / s3 / adls] [uc]: s3
Databricks workspace URL: https://adb-xxxx.azuredatabricks.net
Output directory [...]: /Users/jane/projects

Project created successfully!

Location:   /Users/jane/projects/my_sales_pipeline
Storage:    s3://bucket/wheels
Wheel path: s3://bucket/wheels/my-sales-pipeline-0.1.0-py3-none-any.whl

Next steps:
  cd /Users/jane/projects/my_sales_pipeline
  make install && make test
```

---

## Available Templates

| Key | Description | Status |
|-----|-------------|--------|
| `etl` | Extract-Transform-Load pipeline | ✅ Available |
| `ml` | Machine learning pipeline | 🚧 Coming soon |
| `utility` | Utility / helper library | 🚧 Coming soon |

---

## Storage Options

| Key | Resolved path |
|-----|---------------|
| `uc` | `dbfs:/Volumes/catalog/schema/volume` |
| `dbfs` | `dbfs:/FileStore/wheels` |
| `s3` | `s3://bucket/wheels` |
| `adls` | `abfss://container@storageaccount.dfs.core.windows.net/wheels` |

---

## Development Setup

### 1. Clone the repository

```bash
git clone https://github.com/your-org/python-deployment-package.git
cd python-deployment-package
```

### 2. Create a virtual environment

```bash
python3 -m venv .venv
source .venv/bin/activate
```

### 3. Install in editable mode

```bash
pip install -e .
```

### 4. Verify the CLI works

```bash
acc --help
```

### 5. Run tests

```bash
pytest tests/ -v
```

---

## Publishing a New Version

> ⚠️ **PyPI does not allow re-uploading the same version.** Always bump the version before building.

### Step 1 — Bump the version in `pyproject.toml`

```toml
[project]
version = "0.1.2"   # ← change this
```

### Step 2 — Clean previous builds

```bash
rm -rf dist/ build/ src/acc_cli.egg-info
```

### Step 3 — Build the distribution

```bash
python3 -m build
```

This creates two files inside `dist/`:
- `acc_cli-<version>-py3-none-any.whl` — binary wheel (fast install)
- `acc_cli-<version>.tar.gz` — source distribution

### Step 4 — Upload to PyPI

```bash
python3 -m twine upload dist/*
```

Twine will use credentials from `~/.pypirc`. If that file doesn't exist, it will prompt for your API token.

### Setting up `~/.pypirc` (one-time)

```ini
[distutils]
index-servers = pypi

[pypi]
username = __token__
password = pypi-YOUR_API_TOKEN_HERE
```

> Get your API token at [https://pypi.org/manage/account/token/](https://pypi.org/manage/account/token/)
> The username must always be `__token__` (literally).

### Step 5 — Verify on PyPI

```
https://pypi.org/project/acc-cli/
```

---

## Versioning Convention

This project follows [Semantic Versioning](https://semver.org/) (`MAJOR.MINOR.PATCH`):

| Change type | Example |
|-------------|---------|
| Bug fix / small restructure | `0.1.1` → `0.1.2` |
| New feature / new template | `0.1.2` → `0.2.0` |
| Breaking change | `0.2.0` → `1.0.0` |

---

## Dependencies

| Package | Purpose |
|---------|---------|
| `typer>=0.12` | CLI framework |
| `rich>=13` | Terminal formatting |
| `cookiecutter>=2.6` | Project scaffolding from templates |

---

## License

Proprietary — All rights reserved.


