{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b2b2afd81e6c452db843acfdf9bd533c",
   "metadata": {},
   "source": [
    "# CSV Ingestion\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lakelogic/LakeLogic/blob/main/examples/03_data_sources/files/csv/csv_demo.ipynb) \n",
    "[![GitHub Repo](https://img.shields.io/badge/GitHub-Repo-blue?logo=github)](https://github.com/lakelogic/LakeLogic/blob/main/examples/03_data_sources/files/csv/csv_demo.ipynb)\n",
    "\n",
    "## Business Scenario\n",
    "\n",
    "CSV drops are common for partner feeds and exports. You need schema validation before loading them.\n",
    "\n",
    "## Value Proposition\n",
    "\n",
    "- Define CSV schema in YAML\n",
    "- Validate rows on ingestion\n",
    "- Produce clean outputs\n",
    "\n",
    "---\n",
    "\n",
    "## Goals\n",
    "\n",
    "1. Load a CSV file\n",
    "2. Validate against contract\n",
    "3. Inspect results\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1159ba8816de43e7bd256fa9e9fd04ab",
   "metadata": {},
   "source": [
    "## Step 2: Review the Contract\n",
    "\n",
    "Open `csv_contract.yaml` to see the schema, source config, and quality rules.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80f30d7499774d6e8ab2756fe0b54ca1",
   "metadata": {},
   "source": [
    "## Step 3: Run the Contract\n",
    "\n",
    "Run LakeLogic using the YAML contract. Update credentials as needed.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8eaca72e18cf4579bbe608c8192c5e4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import csv\n",
    "from lakelogic import DataProcessor\n",
    "\n",
    "BASE = Path.cwd()\n",
    "contract_path = BASE / \"csv_contract.yaml\"\n",
    "if not contract_path.exists():\n",
    "    candidate = BASE / \"examples\" / \"03_data_sources\" / \"files\" / \"csv\" / \"csv_contract.yaml\"\n",
    "    if candidate.exists():\n",
    "        BASE = candidate.parent\n",
    "        contract_path = candidate\n",
    "\n",
    "data_dir = BASE / \"data\"\n",
    "data_dir.mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "csv_path = data_dir / \"sample.csv\"\n",
    "if not csv_path.exists():\n",
    "    rows = [\n",
    "        {\"id\": 1, \"email\": \"alice@example.com\"},\n",
    "        {\"id\": 2, \"email\": \"\"},\n",
    "        {\"id\": 3, \"email\": \"bob@example.com\"},\n",
    "    ]\n",
    "    with open(csv_path, \"w\", newline=\"\", encoding=\"utf-8\") as handle:\n",
    "        writer = csv.DictWriter(handle, fieldnames=[\"id\", \"email\"])\n",
    "        writer.writeheader()\n",
    "        writer.writerows(rows)\n",
    "\n",
    "processor = DataProcessor(contract=contract_path)\n",
    "result = processor.run_source(csv_path)\n",
    "\n",
    "print(result)\n",
    "print(f\"Good: {len(result.good)} | Bad: {len(result.bad)}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}