{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5f3084b1-2bc3-4a9a-a63e-0bb66fdfc76b",
   "metadata": {},
   "source": [
    "# Model\n",
    "\n",
    "This notebook loads your assembled and cleaned data and runs modeling and analysis on it.\n",
    "\n",
    "- First, we run predictive models\n",
    "- Then, we analyze your predictions as well as existing ones (such as the assessor's)\n",
    "- Finally, we generate automated statistical reports assessing their quality"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "initial_id",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Change these as desired\n",
    "\n",
    "# The slug of the locality you are currently working on\n",
    "locality = \"us-nc-guilford\"\n",
    "\n",
    "# Whether to print out a lot of stuff (can help with debugging) or stay mostly quiet\n",
    "verbose = True\n",
    "\n",
    "# Clear previous state for this notebook and start fresh\n",
    "clear_checkpoints = True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "024f9ec3-40cb-4bc8-92d2-a5bd04be30a2",
   "metadata": {},
   "source": [
    "# 1. Basic setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1635a044-ae12-445e-ad37-13b947ca40b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import init_notebooks\n",
    "init_notebooks.setup_environment()\n",
    "locality = init_notebooks.check_for_different_locality(locality)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aecc0e73-51d4-4355-8113-30ecc087fef6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import a bunch of stuff\n",
    "from openavmkit.pipeline import (\n",
    "    init_notebook,\n",
    "    load_settings,\n",
    "    load_cleaned_data_for_modeling,\n",
    "    examine_sup,\n",
    "    write_canonical_splits,\n",
    "    try_variables,\n",
    "    try_models,\n",
    "    finalize_models,\n",
    "    run_and_write_ratio_study_breakdowns,\n",
    "    enrich_sup_spatial_lag,\n",
    "    from_checkpoint,\n",
    "    delete_checkpoints,\n",
    "    identify_outliers,\n",
    "    write_parquet\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc48ed32-1b6a-469e-bec2-7d7de42f528a",
   "metadata": {},
   "outputs": [],
   "source": [
    "init_notebook(locality)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0638e861-e637-486f-8950-24e299d7a60d",
   "metadata": {},
   "outputs": [],
   "source": [
    "if clear_checkpoints:\n",
    "    delete_checkpoints(\"3-model\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dbff7085-5d79-41a7-8cb9-9f19fe1f45ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "settings = load_settings()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd9d583e-bc6e-470e-8aad-c5ba806c0ef1",
   "metadata": {},
   "source": [
    "# 2. Prepare"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "347fec55-d97f-4d1e-9b6d-405a1d85379a",
   "metadata": {},
   "source": [
    "We load the cleaned data from the last checkpoint:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3320b5bb-a269-409a-ae8c-ff92a42fea30",
   "metadata": {},
   "outputs": [],
   "source": [
    "# load the data\n",
    "sales_univ_pair = load_cleaned_data_for_modeling(settings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c47d7c38-85a5-4aec-b5e0-03d5ed19f511",
   "metadata": {},
   "outputs": [],
   "source": [
    "examine_sup(sales_univ_pair, load_settings())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0ab610f-1a29-4c38-8b73-2bac2d583852",
   "metadata": {},
   "source": [
    "We separate our test set from our training set.  \n",
    "This guarantees we have one durable source of truth for test/train set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21a2b4be-2b54-4270-bf8f-ce17b3d00ded",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "write_canonical_splits(\n",
    "    sales_univ_pair,\n",
    "    load_settings(),\n",
    "    verbose=verbose\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fc957a0f-e959-43a3-ae64-a08819e7be6e",
   "metadata": {},
   "outputs": [],
   "source": [
    "sales_univ_pair = from_checkpoint(\"3-model-00-enrich-spatial-lag\", enrich_sup_spatial_lag,\n",
    "    {\n",
    "        \"sup\": sales_univ_pair,\n",
    "        \"settings\": load_settings(),\n",
    "        \"verbose\": verbose\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88db81cc-d24e-4c16-9d58-f7b42df387c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "write_parquet(sales_univ_pair.universe, \"out/look/3-spatial-lag-universe.parquet\")\n",
    "write_parquet(sales_univ_pair.sales, \"out/look/3-spatial-lag-sales.parquet\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ebe935a-5a00-403b-95ba-f5972c4434be",
   "metadata": {},
   "outputs": [],
   "source": [
    "examine_sup(sales_univ_pair, load_settings())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83e92a7e-8a21-4470-bec2-f4d4f4ddf92a",
   "metadata": {},
   "source": [
    "# 3. Experiment\n",
    "\n",
    "Try out variables and models before running the real thing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73340d45-2cdb-4e55-a81a-ac39c5e93b89",
   "metadata": {},
   "outputs": [],
   "source": [
    "try_variables(\n",
    "    sales_univ_pair,\n",
    "    load_settings(),\n",
    "    verbose,\n",
    "    plot = False\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "df9f9a22-39b5-4d82-b984-ad0719e4bda2",
   "metadata": {},
   "outputs": [],
   "source": [
    "try_models(\n",
    "    sup=sales_univ_pair,\n",
    "    settings=load_settings(),\n",
    "    save_params=True,\n",
    "    verbose=verbose,\n",
    "    run_main=True,\n",
    "    run_vacant=False,\n",
    "    run_hedonic=False,\n",
    "    run_ensemble=True,\n",
    "    do_shaps=False,\n",
    "    do_plots=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55a648f7-b8c3-4872-959a-3fbc78af280b",
   "metadata": {},
   "source": [
    "# 4. Identify Outliers\n",
    "\n",
    "Look at the predictions that you scored badly against. Answer these questions:\n",
    "\n",
    "- Does the sale make sense? Is it possibly invalid?\n",
    "- Is there a pattern to missed predictions? Are you missing a key variable?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65edffd2-f81a-4d07-815c-a124b1f067ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "identify_outliers(\n",
    "    sup=sales_univ_pair,\n",
    "    settings=load_settings()\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0904868f-508a-47ac-b575-1d919906f882",
   "metadata": {},
   "source": [
    "# 5. Finalize models\n",
    "\n",
    "Once we've locked in good values, we finalize our results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c08db046-f5e1-4439-b636-c6b353ad9336",
   "metadata": {},
   "outputs": [],
   "source": [
    "results = from_checkpoint(\"3-model-02-finalize-models\", finalize_models,\n",
    "    {\n",
    "        \"sup\": sales_univ_pair,\n",
    "        \"settings\": load_settings(),\n",
    "        \"save_params\": True,\n",
    "        \"use_saved_params\": True,\n",
    "        \"verbose\": verbose\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e064bae-c7e3-47f1-81d5-1eca67743722",
   "metadata": {},
   "source": [
    "# 6. Generate reports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e4a9f31f-3a9e-4f3f-9fcb-17a87f270166",
   "metadata": {},
   "outputs": [],
   "source": [
    "# run ratio study reports\n",
    "run_and_write_ratio_study_breakdowns(load_settings())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6223591b-3266-4b1a-bc6a-b00a0ce29469",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
