---
name: python-library-making
description: "Build and distribute reusable Python packages following Synapsis standards. Use this skill when creating new Python libraries, publishing packages (internal or public), managing library versions, setting up Git dependencies, or establishing modular code structures. Covers init, src layout, Git-based distribution (recommended), PyPI publishing, versioning with git tags, and comprehensive troubleshooting. Best practice for moving code from copy-paste across projects into maintainable, installable libraries."
risk: safe
source: original
date_added: 2026-03-04
category: python
compatibility: "uv (modern Python package manager), Git/GitLab/GitHub, python 3.8+"
---

# Python Library Making Skill

## Core Philosophy

> **Modularity over Duplication.** Code that's used across multiple projects (logging connectors, standard response formats, validation utilities) should be modular, versioned libraries—not copy-pasted `utils.py` files scattered across 10 different codebases.

### Why This Matters
- **Maintenance Burden:** Every copy requires independent bug fixes and updates.
- **Testing Complexity:** Inconsistent behavior across projects when libraries diverge.
- **Team Scalability:** New team members must learn from redundant patterns, not standard ones.
- **Deployment Safety:** Version control via git tags and semantic versioning prevents silent breakage.

---

## When to Use This Skill

Use this skill when you encounter **any** of these scenarios:

### ✅ **Creating New Libraries**
- User asks: *"Create a new Python library"*, *"Make a package"*, *"Init a library project"*
- Setting up a new library for shared code (logging, responses, connectors, validators)
- Establishing proper `src/` layout with `pyproject.toml` and git initialization

### ✅ **Advanced Distribution & Publishing**
- User asks: *"How do I publish to PyPI?"*, *"Distribute my package"*, *"Release a library"*
- Publishing an open-source package to the public Python package index
- Setting up CI/CD for automatic publishing, managing publish tokens

### ✅ **Managing Library Versions & Updates**
- User asks: *"Update library version"*, *"Release next version"*, *"Add feature to library"*
- Bumping semantic versions, creating git tags, managing changelog
- Syncing version updates across dependent projects

### ✅ **Internal Dependencies & Git-Based Distribution**
- User asks: *"Install library from Git"*, *"Use private package"*, *"Add internal dependency"*
- Installing libraries via SSH git URLs (recommended for internal Synapsis packages)
- Managing authentication, handling git tags for versioning

### ✅ **Library Best Practices & Troubleshooting**
- Code reviews: checking for proper `src/` layout, `__init__.py` exports, pyproject.toml compliance
- Debugging: `ModuleNotFoundError` after installation, authentication failures, migration from old package manager

---

## Quick Start (5 minutes)

### For the Impatient
```bash
# 1. Create library
uv init --lib my-utils
cd my-utils

# 2. Add code to src/my_utils/
# Edit src/my_utils/__init__.py and export functions

# 3. Push to Git
git commit -m "feat: initial commit"
git push

# 4. Install in another project
uv add git+ssh://git@gitlab.com/synapsis/my-utils.git
```

---

## Pattern 1: Initialize & Structure a Library

### Step 1a: Project Initialization
Use `uv` (not pip, not poetry) for consistency across the Synapsis ecosystem:

```bash
uv init --lib my-library-name
cd my-library-name
```

This creates:
```
my-library-name/
├── pyproject.toml       # Package metadata & dependencies
├── README.md            # Auto-generated
├── src/
│   └── my_library_name/
│       └── __init__.py  # Main entry point
└── .gitignore           # Auto-configured
```

**Critical:** Use **src layout** (`src/package_name/`), NOT flat layout. This prevents import ambiguity.

### Step 1b: Validate the Structure

```bash
# Run validation script (optional but recommended)
# See scripts/validate_structure.py for automated checks
python scripts/validate_structure.py .
```

Required structure:
- ✅ `src/` folder exists
- ✅ `src/package_name/` matches `name` in pyproject.toml (underscores, not hyphens)
- ✅ `__init__.py` exists in src/package_name/
- ✅ `pyproject.toml` has correct metadata

---

## Pattern 2: Organize Code & Expose Public API

### Define Your Package Structure

```
src/my_utils/
├── __init__.py           # ← EXPORT PUBLIC API HERE
├── logger.py             # Internal modules
├── response_handler.py
├── validators.py
└── _internal/
    └── helpers.py        # Private modules (prefix with _)
```

### Export Public Functions in `__init__.py`

```python
# src/my_utils/__init__.py
from .logger import setup_logging, LogConfig
from .response_handler import StandardResponse
from .validators import validate_email

__all__ = [
    "setup_logging",
    "LogConfig",
    "StandardResponse",
    "validate_email",
]

__version__ = "0.1.0"
```

### User Import Simplicity

Once published:
```python
# Users can import cleanly
from my_utils import setup_logging, StandardResponse
```

NOT:
```python
# Avoid deep imports (bad practice)
from my_utils.logger import setup_logging
from my_utils.response_handler import StandardResponse
```

---

## Pattern 3: Distribution Options (Choose One)

### Option 3A: **INTERNAL (Git Dependency)** — *Recommended for Synapsis*

For proprietary company code (not to be published publicly).

**Step 1: Initialize Git & Push to GitLab**
```bash
git init
git remote add origin git@gitlab.com:synapsis/my-utils.git
git add .
git commit -m "feat: initial library setup"
git push -u origin main
```

**Step 2: Install in Another Project**
```bash
cd /path/to/consumer-project
uv add git+ssh://git@gitlab.com/synapsis/my-utils.git
```

**Step 3: (Optional) Use Git Tags for Versions**
```bash
# In my-utils repo, after editing
git tag v0.1.1
git push origin v0.1.1

# In consumer project, install specific version
uv add git+ssh://git@gitlab.com/synapsis/my-utils.git@v0.1.1
```

**Troubleshooting Git Dependency:**
- SSH key not configured? → Run `ssh -T git@gitlab.com` to verify
- Permission denied? → Check SSH key permissions (`chmod 600 ~/.ssh/id_rsa`)
- Want to update dependency? → Run `uv lock --upgrade-package my-utils`

### Option 3B: **PUBLIC (PyPI)** — *For Open Source*

For packages you want to share with the Python community.

**Step 1: Ensure Working Directory is Clean**
```bash
git status  # All changes committed
```

**Step 2: Build Distribution Artifacts**
```bash
uv build
# Generates: dist/my_library_name-0.1.0-py3-none-any.whl + .tar.gz
```

**Step 3: Publish to PyPI**
```bash
# Requires PyPI token (create at https://pypi.org/account/tokens/)
uv publish --token pypi-AgEIcHlwaS5vcmc...
# OR use environment variable
export PYPI_TOKEN=pypi-AgEIcHlwaS5vcmc...
uv publish --token $PYPI_TOKEN
```

**Step 4: Install Anywhere (No Git Required)**
```bash
uv add my-library-name
# Automatically fetches from PyPI
```

**Verification:**
```bash
python -c "import my_utils; print(my_utils.__version__)"
```

---

## Pattern 4: Versioning & Updates

Semantic Versioning: `MAJOR.MINOR.PATCH` (e.g., `0.1.0`)

### Workflow for Library Updates

**Step 1: Make Changes**
```bash
# Edit src/my_utils/logger.py, add new features
# Write tests (see references/test_workflow.md)
```

**Step 2: Update Version in `pyproject.toml`**
```toml
# pyproject.toml
[project]
name = "my-utils"
version = "0.1.1"  # ← Bumped from 0.1.0
```

**Step 3: Commit & Tag (For Git Dependency)**
```bash
git add -A
git commit -m "feat: add robust logging"
git tag v0.1.1
git push origin main
git push origin v0.1.1
```

**Step 4: Update Consumer Projects**
```bash
cd /path/to/consumer-project
uv lock --upgrade-package my-utils
# Fetches latest version from git tag
```

### Version Strategy
- `0.1.0` → `0.1.1`: Bug fixes, internal improvements (patch)
- `0.1.0` → `0.2.0`: New features, backward compatible (minor)
- `0.1.0` → `1.0.0`: Breaking changes (major)

---

## Common Pitfalls & Anti-Patterns

| Pitfall | Why It's Bad | Solution |
| :--- | :--- | :--- |
| **Flat Layout** (`my_lib.py` at root) | Ambiguous imports, conflicts with project code | Use **src layout**: `src/my_lib/my_lib.py` |
| **Mixed Export Points** | Users import from `my_lib.internals.helpers` instead of clean API | Expose **only public API** in `__init__.py`, use `__all__` |
| **No `__init__.py`** | Python doesn't recognize folder as package | Always create `src/package_name/__init__.py`, even if empty |
| **Hardcoded Versions** | Manual version bumping scattered across files | Single source of truth: **pyproject.toml** only |
| **Forgotten Git Tags** | Dependency versions unpredictable, hard to roll back | **Always tag** with `git tag vX.Y.Z` before pushing |
| **Copy-Paste Dependencies** | User assumes library code is stable, but it's a local copy | Use **Git dependency or PyPI**, never local file paths |
| **SSH Keys Not Configured** | Git dependency install fails silently | Verify: `ssh -T git@gitlab.com` works before distributing |

---

## Verification Checklist

Use this before declaring a library "production-ready":

- [ ] **Project Structure**
  - [ ] `src/package_name/` folder exists (not flat layout)
  - [ ] `__init__.py` present and exports public API
  - [ ] `pyproject.toml` has correct `name` field (underscores match src folder)
  - [ ] `.gitignore` includes `__pycache__/`, `.pyc`, `dist/`, `build/`

- [ ] **Code Quality**
  - [ ] Public functions documented with docstrings
  - [ ] Type hints added to at least public functions
  - [ ] Tests written (see [test_workflow.md](references/test_workflow.md))

- [ ] **Distribution Setup**
  - [ ] **Git Dependency:** Remote configured (`git remote -v`), pushed to main branch
  - [ ] **Git Tags** (if applicable): `git tag v0.1.0` created for version commitment
  - [ ] **PyPI** (if public): Built with `uv build`, verified `dist/` artifacts exist
  - [ ] **Token Stored Safely:** PyPI token in environment variable, NOT committed to repo

- [ ] **Installation Test**
  - [ ] Create empty test directory
  - [ ] Install library: `uv add git+ssh://...` or `uv add package-name`
  - [ ] Import test: `python -c "import package_name; print(package_name.__version__)"`
  - [ ] Runs without `ModuleNotFoundError`

- [ ] **Documentation**
  - [ ] README.md written with usage examples
  - [ ] CHANGELOG.md or version history documented
  - [ ] Dependencies clearly listed in `pyproject.toml`

---

## References & Tools

See detailed walkthroughs in the bundled resources:

- **[references/pyproject_template.md](references/pyproject_template.md)** — Annotated `pyproject.toml` templates (internal vs public)
- **[references/troubleshooting.md](references/troubleshooting.md)** — Expanded error solutions for ModuleNotFoundError, auth issues, versioning problems
- **[references/git_workflow.md](references/git_workflow.md)** — Deep dive into git tagging, dependency updates, multi-version testing
- **[scripts/validate_structure.py](scripts/validate_structure.py)** — Automated folder structure checker
- **[scripts/gen_checklist.py](scripts/gen_checklist.py)** — Generate per-project setup checklist
- **[assets/.pyproject.toml.template](assets/.pyproject.toml.template)** — Copy-paste config template

---

## Summary

**Key Decisions:**
1. **Always use `uv`** for package manager consistency (10-100x faster than pip)
2. **Always use src layout** for clarity and to prevent import conflicts
3. **Prefer Git dependencies** for internal Synapsis packages (version control + SSH security)
4. **Use PyPI only for public open-source packages**
5. **Tag with git tags** for version commits, enabling reproducible installs

**Next Steps:**
- Check [references/pyproject_template.md](references/pyproject_template.md) for your specific use case (internal vs public)
- Run `python scripts/validate_structure.py` to ensure compliance
- Review [references/troubleshooting.md](references/troubleshooting.md) if errors occur
