# stitch-proj

Some tools for building a Translator BigKG.  
This software project is experimental and under active development.

---

## Installation

### From PyPI

```bash
pip install stitch-proj
````

### For development

```bash
pip install stitch-proj[dev]
```

### From source

```bash
git clone https://github.com/Translator-CATRAX/stitch-proj.git
cd stitch-proj
pip install -e .[dev]
```

---

## Overview

There are two primary intended users of `stitch-proj`:

1. **Ingester**
   A developer who wants to ingest the
   [Babel concept identifier normalization database](https://github.com/TranslatorSRI/Babel)
   into a local SQLite database.

2. **Querier**
   A developer building an application (e.g., a BigKG build system) who wants to
   programmatically query a local Babel SQLite database.

---

## Package Structure

This project uses a `src/` layout:

```
src/
  stitch_proj/
    ingest_babel.py
    local_babel.py
    row_counts.py
    stitchutils.py
```

Import the package as:

```python
import stitch_proj
```

---

## Tools

* `stitch_proj.ingest_babel`
  Downloads and ingests the Babel database into a local SQLite database.

* `stitch_proj.local_babel`
  Provides functions for querying a local Babel SQLite database.

* `stitch_proj.row_counts`
  Prints table row counts for a local Babel SQLite database.

---

## Running the Ingest

After installation, the console script is available:

```bash
ingest-babel --help
```

Or invoke via module:

```bash
python -m stitch_proj.ingest_babel --help
```

A full ingest requires:

* CPython 3.12
* At least 32 GiB RAM
* ~600 GiB temporary disk space
* ~200 GiB for the final SQLite database

A full ingest may take 30–40 hours depending on hardware.

---

## Downloading a Pre-Built Babel Database

A pre-built SQLite file is available from S3:

```
https://rtx-kg2-public.s3.us-west-2.amazonaws.com/babel-20250331-p1.sqlite
```

Place it in a directory such as:

```
db/babel.sqlite
```

You can then use `stitch_proj.local_babel` to query it.

---

## Running Tests

Ensure a valid `babel.sqlite` file exists locally, then run:

```bash
pytest -v
```

Some tests require internet connectivity.

---

## Systems Tested

`ingest_babel.py` has been tested on:

* Ubuntu 24.04 (x86_64, Intel Xeon)
* Ubuntu 24.04 (ARM64, AWS Graviton3)
* macOS 14 (Apple Silicon)

The package is pure Python and platform-independent, but large ingests require
substantial memory and storage.

---

## Development Workflow

Run linting, typing, and tests with:

```bash
pytest
ruff check .
mypy src
```

Or install development dependencies:

```bash
pip install -e .[dev]
```

---

## License

MIT License. See `LICENSE`.

---

## Citation

Please see the Babel project's `CITATION.cff`:

[https://github.com/TranslatorSRI/Babel/blob/master/CITATION.cff](https://github.com/TranslatorSRI/Babel/blob/master/CITATION.cff)

