{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "# Tutorial 1: ACS PUMS Microdata Analysis\n",
                "\n",
                "This tutorial demonstrates analyzing person-level microdata from the American Community Survey (ACS) Public Use Microdata Sample (PUMS).\n",
                "\n",
                "**Goal:** Get age and sex data for adults in California and Texas, then create weighted frequency tables stratified by state."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Setup"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import os\n",
                "from cendat import CenDatHelper\n",
                "from dotenv import load_dotenv\n",
                "\n",
                "# Load your API key from environment\n",
                "load_dotenv()\n",
                "cdh = CenDatHelper(years=[2022], key=os.getenv(\"CENSUS_API_KEY\"))"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Step 1: Find and Select the PUMS Product"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Search for the ACS 1-year PUMS product\n",
                "# The \\b ensures we match the exact endpoint, not subpaths\n",
                "cdh.list_products(patterns=r\"acs/acs1/pums\\b\")\n",
                "cdh.set_products()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Step 2: Select Geography and Variables"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# For PUMS, geography is simpler—we just need \"state\"\n",
                "cdh.set_geos(values=\"state\", by=\"desc\")\n",
                "\n",
                "# Select the variables we need:\n",
                "# - SEX: Person's sex\n",
                "# - AGEP: Person's age\n",
                "# - ST: State code\n",
                "# - PWGTP: Person weight (crucial for microdata!)\n",
                "cdh.set_variables(names=[\"SEX\", \"AGEP\", \"ST\", \"PWGTP\"])"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Step 3: Get Data"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Fetch data for California (06) and Texas (48)\n",
                "response = cdh.get_data(\n",
                "    within={\"state\": [\"06\", \"48\"]}\n",
                ")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Step 4: Analyze with Tabulate\n",
                "\n",
                "The `tabulate()` method creates Stata-style frequency tables with proper weighting:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Age distribution by sex, stratified by state\n",
                "# Only adults (AGEP > 17), using person weights\n",
                "response.tabulate(\n",
                "    \"SEX\", \"AGEP\",\n",
                "    strat_by=\"ST\",\n",
                "    weight_var=\"PWGTP\",\n",
                "    where=\"AGEP > 17\"\n",
                ")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## Step 5: Convert to DataFrame"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# For further analysis, convert to a DataFrame\n",
                "df = response.to_polars(concat=True, destring=True)\n",
                "print(df.head())"
            ]
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "name": "python",
            "version": "3.11.0"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 4
}