Database - Snailz Docs

Skip to content

# Database

Save data in SQLite database.

## `database_generate(root, db_file)`

Create a SQLite database from CSV files.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`root` | `Path` | Path to directory containing CSV files. | required
`db_file` | `str | None` | Filename for database file or None. | required

Returns:

Type | Description
--- | ---
`Connection | None` | sqlite3.Connection: Database connection if database is in-memory or None otherwise

Source code in `src/snailz/database.py`

     57
     58
     59
     60
     61
     62
     63
     64
     65
     66
     67
     68
     69
     70
     71
     72
     73
     74
     75
     76
     77
     78
     79
     80
     81
     82
     83
     84
     85
     86
     87
     88
     89
     90
     91
     92
     93
     94
     95
     96
     97
     98
     99
    100
    def database_generate(root: Path, db_file: str | None) -> sqlite3.Connection | None:
        """Create a SQLite database from CSV files.

        Parameters:
            root: Path to directory containing CSV files.
            db_file: Filename for database file or None.

        Returns:
            sqlite3.Connection: Database connection if database is in-memory or None otherwise
        """
        if db_file is None:
            conn = sqlite3.connect(":memory:")
        else:
            db_path = root / db_file  # pragma: no cover
            Path(db_path).unlink(missing_ok=True)  # pragma: no cover
            conn = sqlite3.connect(db_path)  # pragma: no cover

        cursor = conn.cursor()

        _import_single_files(root, cursor)
        _import_assay_files(
            root,
            cursor,
            "*_treatments.csv",
            _make_create("treatments", TREATMENTS),
            _make_insert("treatments", TREATMENTS),
            lambda v: v,
        )
        _import_assay_files(
            root,
            cursor,
            "*_readings.csv",
            _make_create("readings", READINGS),
            _make_insert("readings", READINGS),
            lambda v: float(v),
        )

        conn.commit()

        if db_file is None:
            return conn
        else:
            conn.close()  # pragma: no cover
            return None  # pragma: no cover

## `_import_assay_files(root, cursor, pattern, create, insert, convert)`

Import data from all clean assay files.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`root` | `Path` | path to root directory | required
`cursor` | `Cursor` | database cursor | required
`pattern` | `str` | filename pattern | required
`create` | `str` | SQL table creation statement | required
`insert` | `str` | SQL insertion statement | required
`convert` | `Callable` | text-to-value conversion function | required

Source code in `src/snailz/database.py`

    103
    104
    105
    106
    107
    108
    109
    110
    111
    112
    113
    114
    115
    116
    117
    118
    119
    120
    121
    122
    123
    124
    125
    126
    127
    128
    129
    130
    131
    132
    def _import_assay_files(
        root: Path,
        cursor: sqlite3.Cursor,
        pattern: str,
        create: str,
        insert: str,
        convert: Callable,
    ) -> None:
        """Import data from all clean assay files.

        Parameters:
            root: path to root directory
            cursor: database cursor
            pattern: filename pattern
            create: SQL table creation statement
            insert: SQL insertion statement
            convert: text-to-value conversion function
        """
        cursor.execute(create)
        for filename in (root / utils.ASSAYS_DIR).glob(pattern):
            with open(filename, "r") as stream:
                rows = [r for r in csv.reader(stream)]
                assert rows[0][0] == "id"
                ident = rows[0][1]
                data = [r[1:] for r in rows[NUM_ASSAY_HEADER_ROWS:]]
                temp = []
                for i, row in enumerate(data):
                    for j, val in enumerate(row):
                        temp.append((ident, i + 1, chr(ord("A") + j), convert(val)))
                cursor.executemany(insert, temp)

## `_import_single_files(root, cursor)`

Import single CSV files into database.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`root` | `Path` | path to root directory | required
`cursor` | `Cursor` | database cursor | required

Source code in `src/snailz/database.py`

    135
    136
    137
    138
    139
    140
    141
    142
    143
    144
    145
    146
    147
    148
    149
    150
    151
    152
    153
    def _import_single_files(root: Path, cursor: sqlite3.Cursor) -> None:
        """Import single CSV files into database.

        Parameters:
            root: path to root directory
            cursor: database cursor
        """
        for filename, table, spec in (
            (utils.ASSAYS_CSV, "assays", ASSAYS),
            (utils.MACHINES_CSV, "machines", MACHINES),
            (utils.PERSONS_CSV, "persons", PERSONS),
            (utils.SPECIMENS_CSV, "specimens", SPECIMENS),
        ):
            filepath = root / filename
            with open(filepath, "r") as stream:
                data = [row for row in csv.reader(stream)]
                assert data[0] == _make_header(spec)
                cursor.execute(_make_create(table, spec))
                cursor.executemany(_make_insert(table, spec), data[1:])

## `_make_create(table, spec)`

Generate SQL table creation statement.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`table` | `str` | database table name | required
`spec` | `tuple` | tuples of (field name, field properties) | required

Returns:

Type | Description
--- | ---
`str` | SQL table creation statement

Source code in `src/snailz/database.py`

    156
    157
    158
    159
    160
    161
    162
    163
    164
    165
    166
    167
    def _make_create(table: str, spec: tuple) -> str:
        """Generate SQL table creation statement.

        Parameters:
            table: database table name
            spec: tuples of (field name, field properties)

        Returns:
            SQL table creation statement
        """
        fields = ", ".join(f"{name} {props}" for name, props in spec)
        return f"create table {table}({fields})"

## `_make_header(spec)`

Generate expected CSV header row.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`spec` | `tuple` | tuples of (field name, field properties) | required

Returns:

Type | Description
--- | ---
`list[str]` | Expected first row of CSV file.

Source code in `src/snailz/database.py`

    170
    171
    172
    173
    174
    175
    176
    177
    178
    179
    def _make_header(spec: tuple) -> list[str]:
        """Generate expected CSV header row.

        Parameters:
            spec: tuples of (field name, field properties)

        Returns:
            Expected first row of CSV file.
        """
        return [name for name, _ in spec]

## `_make_insert(table, spec)`

Generate SQL insertion statement.

Parameters:

Name | Type | Description | Default
--- | --- | --- | ---
`table` | `str` | database table name | required
`spec` | `tuple` | tuples of (field name, field properties) | required

Returns:

Type | Description
--- | ---
`str` | SQL record insertion statement

Source code in `src/snailz/database.py`

    182
    183
    184
    185
    186
    187
    188
    189
    190
    191
    192
    def _make_insert(table: str, spec: tuple) -> str:
        """Generate SQL insertion statement.

        Parameters:
            table: database table name
            spec: tuples of (field name, field properties)

        Returns:
            SQL record insertion statement
        """
        return f"insert into {table} values ({', '.join('?' * len(spec))})"