{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2997d370c8be",
   "source": "**Note:** This notebook is designed for **Google Colab**.\n\nIf you see the Colab logo <span style='vertical-align:bottom;'><img src='https://colab.research.google.com/img/colab_favicon_256px.png' width='40' alt='Colab logo'></span> in the top-left corner, you're all set! Please **continue**.\n\nIf you don't see the logo (e.g., you are on GitHub), please click the button below to open it in the correct environment:\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mparrott-at-wiris/aimodelshare/blob/master/notebooks/qmss_climate_competition_notebook.ipynb)",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "id": "b8effcee3fc4",
   "source": "# **QMSS Climate Change ML Competition: Build & Submit Custom Models**\n\nWelcome to the **QMSS Climate Change ML Competition**.\n\n**Why this matters:**\nBuildings account for nearly 40% of global energy consumption and contribute roughly 30% of global greenhouse gas emissions. Identifying which buildings are high energy users enables targeted retrofits and meaningful reductions in carbon emissions. Your models can help pinpoint the buildings where intervention will have the greatest climate impact.\n\n**The Goal:**\nTrain a model to predict which buildings are high energy users (`high_energy_usage = 1`) using building characteristics, location, and climate features.\n\n**Who is this for?**\nThis notebook is designed for participants with Python experience (e.g., Scikit-Learn, TensorFlow, PyTorch). You will build, train, and submit your own machine learning models directly to the competition leaderboard.",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "id": "b8686aed8a00",
   "source": "## Competition Awards & Rules\n\n### Individual & Team Tracks\nThis competition has **two separate award tracks**:\n- **Individual Track** — Your modelshare.ai username is automatically captured when you log in. Every submission is linked to your individual account.\n- **Team Track** — If you are working as a team, set the `\"Team\"` field in your submission to your team name. All team members should use the **same team name** when submitting.\n\nYou can participate in both tracks simultaneously. Digital awards are given separately for individuals and teams.\n\n### Digital Awards\nAll awards are **digital awards**. Winners will be publicized by the QMSS Program.\n- **1st, 2nd, 3rd Place** — Top three submissions on the final leaderboard receive digital podium awards (awarded separately for individuals and teams).\n- **Gold Medal** — Top 1% of all submissions\n- **Silver Medal** — Top 5% of all submissions\n- **Bronze Medal** — Top 10% of all submissions\n\nMedals are awarded in addition to podium prizes (e.g., a 1st-place finisher also earns a Gold Medal).\n\n### Scoring\nYour submissions are scored on the held-out test set using the following classification metrics:\n- **F1 Score** — harmonic mean of precision and recall\n- **Accuracy** — overall proportion of correct predictions\n- **Precision** — proportion of predicted positives that are truly positive\n- **Recall** — proportion of actual positives correctly identified\n\nThe leaderboard displays all four metrics so you can evaluate your model's performance in detail. **Final award winners will be ranked by F1 Score.**\n\n### Tie-Breaker\nIf two submissions achieve the same F1 score, the submission with the **earliest timestamp** wins.\n\n### Submission Rules\n- Each submission must be a 1D array of **25,000 binary predictions** (0 or 1), one per test sample.\n- You may submit **multiple times** — the leaderboard tracks all submissions.\n- **Submitting as an individual?** Leave the `\"Team\"` field empty (`\"\"`) or remove it entirely. Your username is captured automatically.\n- **Submitting as a team?** Set the `\"Team\"` field to your team name. All team members should use the same name.",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "id": "996a31dd50da",
   "source": "## Quick Start Guide\n\nTo participate in the competition, complete these 5 steps:\n\n1.  **Install Libraries:** Run the setup cell to install `aimodelshare`.\n2.  **Get the Data:** Run the data loading cell to retrieve the pre-split training and testing data.\n3.  **Train Your Model:** Use the provided Scikit-Learn Pipeline example or write your own custom training code.\n4.  **Connect:** Link this notebook to the QMSS Climate Competition Leaderboard.\n5.  **Submit:** Send your predictions to the leaderboard to see your score.\n\n**Ready? Click the Play Button on the first code cell below to get started.**",
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "id": "d23fe439b1b7",
   "source": "---\n# **Step 1: Installation**\n\nWe need to install the `aimodelshare` library to connect to the competition backend.",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "5902154ae29b",
   "source": "# Install the aimodelshare library\nprint(\"Installing required libraries...\")\n!pip install aimodelshare --upgrade -q --no-warn-script-location > /dev/null 2>&1\nprint(\"Installation complete!\")",
   "metadata": {},
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "366b9af22982",
   "source": "---\n# **Step 2: Load Data**\n\nWe will load the pre-split competition data directly from the competition URLs.\n\n* **train.csv** — 75,000 building records with features + target label (`high_energy_usage`)\n* **X_test.csv** — 25,000 building records with features only (no target label)\n\nYou will train your model on the training data and generate predictions on X_test. The true test labels are held by the leaderboard — your score is computed when you submit.\n\n### Feature Guide\n\nThe dataset contains **45 features** across three categories:\n\n| Category | Features | Description |\n|---|---|---|\n| **Building characteristics** | `facility_type`, `floor_area`, `year_built`, `building_class` | Building use type (e.g., office, retail), gross floor area (sq ft), construction year, and building classification |\n| **Location** | `State_Factor`, `ELEVATION` | US state identifier and elevation in meters above sea level |\n| **Climate — monthly temps** | `january_min_temp` ... `december_max_temp` (36 features) | Min, average, and max temperature for each month (°F) |\n| **Climate — annual** | `avg_temp`, `heating_degree_days`, `cooling_degree_days` | Annual average temperature and degree-day measures of heating/cooling energy demand |\n\n**Target variable:** `high_energy_usage` — binary label where 1 = high energy user, 0 = not a high energy user.\n\nFor detailed feature descriptions including value distributions and ranges, see the full [Dataset Description](https://github.com/mparrott-at-wiris/aimodelshare/blob/master/datasets/qmss_climate/QMSS_COMPETITION_DATA_DESCRIPTION.md).",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "10015a2e34f9",
   "source": "import pandas as pd\n\n# 1. Load training data (features + target)\nDATA_BASE = \"https://raw.githubusercontent.com/mparrott-at-wiris/aimodelshare/master/datasets/qmss_climate\"\ntrain_df = pd.read_csv(f\"{DATA_BASE}/train.csv\")\n\n# 2. Load test data (features only — no target labels)\nX_test = pd.read_csv(f\"{DATA_BASE}/X_test.csv\")\n\n# 3. Separate training features and target\nNUMERIC_COLS = [\n    \"floor_area\", \"year_built\", \"ELEVATION\",\n    \"heating_degree_days\", \"cooling_degree_days\", \"avg_temp\",\n    \"january_min_temp\", \"january_avg_temp\", \"january_max_temp\",\n    \"february_min_temp\", \"february_avg_temp\", \"february_max_temp\",\n    \"march_min_temp\", \"march_avg_temp\", \"march_max_temp\",\n    \"april_min_temp\", \"april_avg_temp\", \"april_max_temp\",\n    \"may_min_temp\", \"may_avg_temp\", \"may_max_temp\",\n    \"june_min_temp\", \"june_avg_temp\", \"june_max_temp\",\n    \"july_min_temp\", \"july_avg_temp\", \"july_max_temp\",\n    \"august_min_temp\", \"august_avg_temp\", \"august_max_temp\",\n    \"september_min_temp\", \"september_avg_temp\", \"september_max_temp\",\n    \"october_min_temp\", \"october_avg_temp\", \"october_max_temp\",\n    \"november_min_temp\", \"november_avg_temp\", \"november_max_temp\",\n    \"december_min_temp\", \"december_avg_temp\", \"december_max_temp\",\n]\nCATEGORICAL_COLS = [\"facility_type\", \"building_class\", \"State_Factor\"]\nALL_FEATURES = NUMERIC_COLS + CATEGORICAL_COLS\n\nX_train = train_df[ALL_FEATURES]\ny_train = train_df[\"high_energy_usage\"]\n\nprint(f\"Training data: {X_train.shape[0]} samples, {X_train.shape[1]} features\")\nprint(f\"Test data: {X_test.shape[0]} samples (no labels — scored by leaderboard)\")\nprint(f\"\\nTraining target distribution:\\n{y_train.value_counts().to_dict()}\")\nprint(\"\\nFirst 5 rows of training data:\")\nX_train.head()",
   "metadata": {},
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "9eedc25b1412",
   "source": "---\n# **Step 3: Train Your Model**\n\nBelow is a **baseline example** using a Scikit-Learn Pipeline with Logistic Regression. This pipeline:\n1.  **Scales** numerical columns (floor area, temperatures, degree days, etc.).\n2.  **One-Hot Encodes** categorical columns (facility type, building class, state).\n3.  **Trains** a Logistic Regression classifier.\n\n**To improve your score**, consider:\n- Replacing the classifier with a more powerful model (RandomForest, GradientBoosting, XGBoost, Neural Network, etc.)\n- Engineering new features from existing ones (e.g., temperature ranges, climate-building interactions)\n- Tuning hyperparameters with cross-validation\n- Handling class imbalance if present in the training data",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "202fcaaf8509",
   "source": "from sklearn.pipeline import Pipeline\nfrom sklearn.compose import ColumnTransformer\nfrom sklearn.preprocessing import OneHotEncoder, StandardScaler\nfrom sklearn.linear_model import LogisticRegression\n\n# 1. Define Transformers\nnumeric_transformer = Pipeline(steps=[\n    ('scaler', StandardScaler())\n])\n\ncategorical_transformer = Pipeline(steps=[\n    ('encoder', OneHotEncoder(handle_unknown='ignore'))\n])\n\n# 2. Create Preprocessor\npreprocessor = ColumnTransformer(\n    transformers=[\n        ('num', numeric_transformer, NUMERIC_COLS),\n        ('cat', categorical_transformer, CATEGORICAL_COLS)\n    ])\n\n# 3. Create Pipeline (Preprocessor + Model)\n# Replace LogisticRegression with any model you want to try\npipeline = Pipeline(steps=[\n    ('preprocessor', preprocessor),\n    ('classifier', LogisticRegression(max_iter=1000))\n])\n\n# 4. Train the pipeline on training data\npipeline.fit(X_train, y_train)\n\n# 5. Generate predictions on the test set\npredictions = pipeline.predict(X_test)\n\nprint(f\"Model trained! Predictions generated for {len(predictions)} test samples.\")\nprint(f\"Training accuracy: {pipeline.score(X_train, y_train):.4f}\")\nprint(\"Submit your predictions in the next step to see your test score on the leaderboard.\")",
   "metadata": {},
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "1bbe3483ce72",
   "source": "---\n# **Step 4: Connect to the Leaderboard**\n\nThis step connects your notebook to the QMSS Climate Change ML Competition backend.\n\n*Note: You will be prompted to enter a username and password. If you don't have one, you will need to create one at [modelshare.ai](https://www.modelshare.ai)*",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "624e31556eed",
   "source": "from aimodelshare.aws import set_credentials\nfrom aimodelshare.playground import Competition\nimport os\n\nimport warnings\nwarnings.simplefilter(action='ignore', category=FutureWarning)\n\n# The specific Model Playground URL for the QMSS Climate Competition\nmy_playground_url = \"https://br3t6wp6yg.execute-api.us-east-1.amazonaws.com/prod/m\"\n\n# Set your credentials (pop-up will appear)\nset_credentials(apiurl=my_playground_url)\n\n# Generate your session access token\ntoken = os.getenv(\"AWS_TOKEN\")\n\n# Connect to the competition\nplayground = Competition(my_playground_url)",
   "metadata": {},
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "markdown",
   "id": "9ce60e75f46f",
   "source": "---\n# **Step 5: Submit & Check Results**\n\nSubmit your predictions to the leaderboard.\n\n- **Individual submission:** Leave the `\"Team\"` field as `\"\"` (your username is captured automatically).\n- **Team submission:** Set `\"Team\"` to your team name.",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "id": "d36304e4d7b8",
   "source": "# 1. Submit your predictions\nplayground.submit_model(\n    model=None,\n    preprocessor=None,\n    prediction_submission=predictions,\n    token=token,\n    input_dict={\n        \"Team\": \"\",  # Leave empty for individual submission, or set your team name (e.g., \"Climate Crusaders\")\n        \"description\": \"Logistic Regression with Sklearn Pipeline\",\n        \"tags\": \"sklearn, logistic_regression, climate, baseline\"\n    }\n)\n\nprint(\"Predictions submitted successfully!\")",
   "metadata": {},
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "id": "1a19eb5b5faf",
   "source": "# 2. Check the leaderboard\nprint(\"Loading leaderboard...\")\nleaderboard = playground.get_leaderboard()\nplayground.stylize_leaderboard(leaderboard)",
   "metadata": {},
   "outputs": [],
   "execution_count": null
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.0"
  },
  "colab": {
   "provenance": []
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}