# nxCypher Open-Source Publishing Instructions

Complete guide for publishing nxCypher as an open-source project. These instructions can be followed manually or by an AI assistant in a new session.

---

## Status: What Has Been Done

All automated/code tasks are **complete**. The following were implemented:

### Files Created
| File | Purpose |
|------|---------|
| `NOTICE` | Apache 2.0 Section 4(d) attribution — derivative work of GrandCypher |
| `DCO` | Developer Certificate of Origin v1.1 |
| `CONTRIBUTING.md` | Contribution guidelines with DCO sign-off instructions |
| `CODE_OF_CONDUCT.md` | Contributor Covenant v2.1 |
| `.github/SECURITY.md` | Vulnerability reporting policy |
| `.github/ISSUE_TEMPLATE/bug_report.md` | Structured bug report template |
| `.github/ISSUE_TEMPLATE/feature_request.md` | Feature request template |
| `.github/PULL_REQUEST_TEMPLATE.md` | PR checklist with DCO sign-off |
| `.github/workflows/dco.yml` | CI workflow enforcing DCO sign-off on PRs |
| `nxcypher/test_count_star.py` | Tests for COUNT(*) and COUNT(DISTINCT) |

### Files Modified
| File | Change |
|------|--------|
| `LICENSE` | Added APPENDIX with IBM copyright notice |
| `NOTICE` | Created with GrandCypher attribution and Cypher trademark notice |
| `pyproject.toml` | Added `license = "Apache-2.0"` and `license-files` |
| `MANIFEST.in` | Added `NOTICE` to included files |
| `README.md` | Added license/CI badges, COUNT(*)/DISTINCT rows in feature table, Contributing and License sections |
| `nxcypher/__init__.py` | Copyright header + COUNT(*)/COUNT(DISTINCT) feature |
| `nxcypher/struct.py` | Copyright header added |
| `nxcypher/hinter.py` | Copyright header added |
| `nxcypher/indexer.py` | Copyright header added |
| `nxcypher/kb_loader.py` | Copyright header added |

### Verification
- **544 tests pass** (`uv run pytest nxcypher/ -v`)
- **No new lint errors** (`uv run ruff check nxcypher/`)
- All governance files verified present

---

## Remaining Steps (Manual / Human Actions)

These steps require human judgment and cannot be automated:

### Step 1: Update README Badge URLs

The CI badge URL in `README.md` currently points to `github.com/ibm/nxcypher`. Update it to match the actual target repository:

```
Find:    https://github.com/ibm/nxcypher/actions/workflows/python-package.yml
Replace: https://github.com/<OWNER>/<REPO>/actions/workflows/python-package.yml
```

(Both in the badge image URL and the badge link.)

### Step 2: Update SECURITY.md Maintainer Contact

In `.github/SECURITY.md`, the email placeholder is `maintainers@example.com`. Replace with the actual maintainer email for security reports, or remove the email option and rely solely on GitHub Security Advisories.

### Step 3: Remove IBM Internal Git Remote

The repository has an IBM internal remote. Before pushing to a public repo:

```bash
# Check current remotes
git remote -v

# Remove the IBM internal remote
git remote remove origin
# or if it's named differently:
git remote remove ibm
```

### Step 4: Audit Git History

Review commit messages for any IBM internal URLs, credentials, or sensitive references:

```bash
# Search for internal hostnames
git log --all --oneline | head -50
git log --all --grep="github.ibm.com" --oneline
git log --all --grep="ibm.com" --oneline
```

If sensitive references are found, consider:
- Squashing commits before publishing
- Using `git filter-repo` to clean history
- Starting fresh with an initial commit

### Step 5: Choose Target Repository

Decide where to publish:
- **GitHub.com personal**: `github.com/<username>/nxcypher`
- **GitHub.com IBM org**: `github.com/ibm/nxcypher`
- Other hosting platform

### Step 6: Obtain IBM Management Approval

Per IBM Open Innovation Handbook:
- Get executive/management approval for the open-source release
- Ensure legal has reviewed the Apache 2.0 licensing approach
- Confirm the derivative work attribution in `NOTICE` is acceptable

### Step 7: Create Public Repository and Push

```bash
# Add the new public remote
git remote add origin https://github.com/<OWNER>/nxcypher.git

# Push
git push -u origin master
```

### Step 8: Enable GitHub Features

After pushing to the public repo:
1. Enable **GitHub Security Advisories** in repo Settings > Security
2. Verify the **DCO Check** workflow runs on PRs
3. Verify issue and PR templates appear correctly
4. (Optional) Add branch protection rules requiring DCO check to pass

### Step 9: Publish to PyPI (Optional)

```bash
uv build
uv publish
# or: twine upload dist/*
```

Ensure `pyproject.toml` has the correct package name (`nxcypher`) and version before publishing.

---

## Architecture Reference

For anyone continuing development work:

### Key Files
- `nxcypher/__init__.py` — Main engine (~4150 lines): grammar, parser, executor
- `nxcypher/struct.py` — Data structures
- `nxcypher/hinter.py` — Query hint processing
- `nxcypher/indexer.py` — Attribute indexing for performance
- `nxcypher/kb_loader.py` — Knowledge base JSONL loader

### Development Commands
```bash
# Install dependencies
uv sync

# Run all tests
uv run pytest nxcypher/ -v

# Run specific test file
uv run pytest nxcypher/test_count_star.py -v

# Lint
uv run ruff check nxcypher/

# Auto-fix lint issues
uv run ruff check nxcypher/ --fix
```

### Recent Feature Additions
- **COUNT(*)**: Aggregate counting all matched rows (with grouping support)
- **COUNT(DISTINCT x)**: Distinct counting in aggregates
- **COLLECT(DISTINCT x)**: Distinct values in COLLECT aggregation
- **DISTINCT in SUM/AVG**: Distinct values in numeric aggregates

### Missing Neo4j Functions (Future Work)
See the plan file at `.claude/plans/` for a full inventory. Key gaps include:
- Type conversion: `toString()`, `toInteger()`, `toFloat()`, `toBoolean()`
- Utility: `coalesce()`, `properties()`, `exists()`
- Math: `abs()`, `round()`, `ceil()`, `floor()`, `sqrt()`
- String: `replace()`, `substring()`, `split()`, `left()`, `right()`
- List: `range()`, `reverse()`, `last()`, `length()` (for paths)
- Temporal: `date()`, `datetime()`, `timestamp()`, `duration()`
- Advanced aggregates: `percentileCont()`, `percentileDisc()`, `stDev()`

---

## License Compliance Checklist

Before publishing, verify:

- [x] `LICENSE` file present with Apache 2.0 full text + copyright appendix
- [x] `NOTICE` file with GrandCypher attribution (Apache 2.0 Section 4d)
- [x] Copyright headers in all source files (Apache 2.0 Section 4b)
- [x] `pyproject.toml` has `license = "Apache-2.0"` (PEP 639)
- [x] `MANIFEST.in` includes both `LICENSE` and `NOTICE`
- [x] `DCO` file present
- [x] `CONTRIBUTING.md` with DCO sign-off instructions
- [x] `CODE_OF_CONDUCT.md` (Contributor Covenant v2.1)
- [x] `.github/SECURITY.md` vulnerability reporting policy
- [x] GitHub issue/PR templates present
- [x] DCO enforcement CI workflow present
- [x] `README.md` has license badge, attribution, contributing section
- [x] All dependencies are permissively licensed (Apache 2.0, MIT, BSD 3-clause)
- [ ] README badge URLs match actual target repo
- [ ] SECURITY.md has real maintainer contact
- [ ] IBM internal git remote removed
- [ ] Git history audited for sensitive content
- [ ] Management approval obtained
- [ ] Public repo created and pushed
