{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "# agrobr — Demo Completo\n\nDados agrícolas brasileiros em uma linha de código.\n\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bruno-portfolio/agrobr/blob/main/examples/agrobr_demo.ipynb)\n\nEste notebook demonstra as principais funcionalidades do **agrobr v0.9.0**:\n- 13 fontes de dados com golden tests (CEPEA, CONAB, IBGE, NASA POWER, BCB/SICOR, ComexStat, ANDA, ABIOVE, USDA, IMEA, DERAL, INMET, Notícias Agrícolas)\n- Camada semântica com fallback automático\n- MetaInfo para proveniência e rastreabilidade (attempted_sources, selected_source)\n- Resiliência HTTP: retry centralizado, 429 handling, Retry-After\n- Cache DuckDB inteligente com histórico permanente\n- Modo síncrono e async\n\n> **Nota:** INMET requer token de autenticação (`AGROBR_INMET_TOKEN`). Sem token, dados retornam vazio (HTTP 204). Use **NASA POWER** como alternativa sem autenticação — seção 10 deste notebook."
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Instalação"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -q agrobr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## 1b. Configurar INMET Token (opcional)\n",
    "\n",
    "# A API de dados observacionais do INMET requer token de autenticacao.\n",
    "# Sem token, requisicoes retornam HTTP 204 (sem conteudo).\n",
    "# Para dados climaticos sem token, use NASA POWER (secao 10).\n",
    "\n",
    "import os\n",
    "\n",
    "os.environ[\"AGROBR_INMET_TOKEN\"] = \"seu-token-aqui\"  # Substitua pelo seu token real\n",
    "\n",
    "# Para obter o token: https://portal.inmet.gov.br\n",
    "# Sem token, listagem de estacoes funciona, mas dados observacionais nao."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Preço Diário — Camada Semântica\n",
    "\n",
    "O `datasets.preco_diario()` resolve a melhor fonte automaticamente (CEPEA → Notícias Agrícolas → cache)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from agrobr.sync import datasets\n",
    "\n",
    "df_preco = datasets.preco_diario(\"soja\", inicio=\"2025-01-01\")\n",
    "print(f\"Registros: {len(df_preco)}, Colunas: {df_preco.columns.tolist()}\")\n",
    "df_preco.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. MetaInfo — Proveniência\n",
    "\n",
    "Toda consulta pode retornar `MetaInfo` com rastreabilidade completa da origem dos dados."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_preco, meta = datasets.preco_diario(\"soja\", inicio=\"2025-01-01\", return_meta=True)\n",
    "\n",
    "print(\"=== MetaInfo — Proveniencia Auditavel ===\")\n",
    "print(f\"  source:            {meta.source}\")\n",
    "print(f\"  selected_source:   {meta.selected_source}\")\n",
    "print(f\"  attempted_sources: {meta.attempted_sources}\")\n",
    "print(f\"  dataset:           {meta.dataset}\")\n",
    "print(f\"  contract_version:  {meta.contract_version}\")\n",
    "print(f\"  schema_version:    {getattr(meta, 'schema_version', 'N/A')}\")\n",
    "print(f\"  fetch_timestamp:   {getattr(meta, 'fetch_timestamp', 'N/A')}\")\n",
    "print(f\"  records_count:     {meta.records_count}\")\n",
    "print(f\"  from_cache:        {meta.from_cache}\")\n",
    "print(f\"  fetch_duration_ms: {meta.fetch_duration_ms}\")\n",
    "print(f\"  agrobr_version:    {meta.agrobr_version}\")\n",
    "print()\n",
    "print(\"Quando fallback ocorre, attempted_sources mostra a cadeia percorrida.\")\n",
    "print(\"selected_source indica qual fonte efetivamente forneceu os dados.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Fallback Automático\n",
    "\n",
    "O dataset resolve sozinho quando a fonte primária falha.\n",
    "Observe `attempted_sources` — se CEPEA retornar 403, cai para Notícias Agrícolas ou cache."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# O fallback e transparente: voce pede o dado, o agrobr resolve a fonte\n",
    "df, meta = datasets.preco_diario(\"milho\", inicio=\"2025-01-01\", return_meta=True)\n",
    "\n",
    "print(\"Produto: milho\")\n",
    "print(f\"Fontes tentadas: {meta.attempted_sources}\")\n",
    "print(f\"Fonte selecionada: {meta.selected_source}\")\n",
    "print(f\"Registros: {meta.records_count}\")\n",
    "df.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Indicadores CEPEA\n",
    "\n",
    "Acesso direto aos indicadores CEPEA/ESALQ — 20 produtos disponíveis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from agrobr.sync import cepea\n",
    "\n",
    "# Listar todos os produtos disponiveis\n",
    "produtos = cepea.produtos()\n",
    "print(f\"Produtos CEPEA ({len(produtos)}): {produtos}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Serie historica de cafe\n",
    "df_cafe = cepea.indicador(\"cafe\", inicio=\"2025-01-01\")\n",
    "print(f\"Cafe: {len(df_cafe)} registros\")\n",
    "df_cafe.head(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. CONAB — Safras\n",
    "\n",
    "Estimativas de safra da CONAB. Requer acesso à página de boletins (pode falhar em ambientes sem Playwright)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from agrobr.sync import conab\n",
    "\n",
    "try:\n",
    "    df_safra = conab.safras(\"soja\", safra=\"2024/25\")\n",
    "    print(f\"Safra 2024/25: {len(df_safra)} UFs\")\n",
    "    display(df_safra[[\"uf\", \"area_plantada\", \"producao\", \"produtividade\"]].head(10))\n",
    "except Exception as e:\n",
    "    print(f\"CONAB safras indisponivel: {type(e).__name__}\")\n",
    "    print(\"Dica: em ambientes restritos, instale playwright (pip install agrobr[browser])\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 7. IBGE — PAM\n",
    "\n",
    "Produção Agrícola Municipal (anual) do IBGE/SIDRA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from agrobr.sync import ibge\n",
    "\n",
    "df_pam = ibge.pam(\"soja\", ano=2023, nivel=\"uf\")\n",
    "print(f\"PAM soja 2023: {len(df_pam)} registros, colunas: {df_pam.columns.tolist()}\")\n",
    "df_pam.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 8. Comparar Fontes — CONAB vs IBGE\n",
    "\n",
    "O agrobr permite cruzar dados de diferentes fontes para validação."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# IBGE PAM 2023 - producao por UF\n",
    "# Coluna pode ser \"producao\" (apos pivot) ou \"Quantidade produzida\" (nome SIDRA original)\n",
    "col_prod = \"producao\" if \"producao\" in df_pam.columns else \"Quantidade produzida\"\n",
    "col_local = \"localidade\" if \"localidade\" in df_pam.columns else df_pam.columns[0]\n",
    "\n",
    "if col_prod in df_pam.columns:\n",
    "    top_ibge = df_pam[[col_local, col_prod]].dropna(subset=[col_prod]).nlargest(5, col_prod)\n",
    "    print(\"=== Top 5 UFs - IBGE PAM 2023 (Toneladas) ===\")\n",
    "    print(top_ibge.to_string(index=False))\n",
    "else:\n",
    "    print(f\"Coluna de producao nao encontrada. Colunas disponiveis: {df_pam.columns.tolist()}\")\n",
    "    print(df_pam.head())\n",
    "\n",
    "# CONAB safra 2023/24\n",
    "try:\n",
    "    df_conab = conab.safras(\"soja\", safra=\"2023/24\")\n",
    "    top_conab = df_conab[[\"uf\", \"producao\"]].sort_values(\"producao\", ascending=False).head(5)\n",
    "    print(\"\\n=== Top 5 UFs - CONAB (safra 2023/24, mil ton) ===\")\n",
    "    print(top_conab.to_string(index=False))\n",
    "except Exception as e:\n",
    "    print(f\"\\nCONAB indisponivel: {type(e).__name__}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 9. Balanço Oferta & Demanda\n",
    "\n",
    "Balanço de suprimentos da CONAB (oferta, demanda, estoques)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    df_balanco = conab.balanco(\"soja\")\n",
    "    if len(df_balanco) > 0:\n",
    "        print(f\"Balanco soja: {len(df_balanco)} registros\")\n",
    "        display(df_balanco.head(10))\n",
    "    else:\n",
    "        print(\"Balanco retornou vazio (formato do boletim pode ter mudado)\")\n",
    "except Exception as e:\n",
    "    print(f\"Balanco indisponivel: {type(e).__name__}\")\n",
    "    print(\"Dica: balanco depende do XLSX da CONAB, que pode mudar de formato entre levantamentos\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 10. NASA POWER — Climatologia\n",
    "\n",
    "Dados climáticos via satélite NASA POWER. Resolução ~55 km (grid 0.5°), cobertura global, desde 1981.\n",
    "API REST pura, sem autenticação, 7 parâmetros agroclimáticos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from agrobr.sync import nasa_power\n",
    "\n",
    "# Clima mensal por UF (ponto central do estado)\n",
    "df_clima = nasa_power.clima_uf(\"MT\", ano=2025)\n",
    "print(f\"Clima MT 2025: {len(df_clima)} meses, colunas: {df_clima.columns.tolist()}\")\n",
    "df_clima"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Clima diario por coordenada especifica (Sorriso-MT)\n",
    "df_diario = nasa_power.clima_ponto(lat=-12.6, lon=-56.1, inicio=\"2025-01-01\", fim=\"2025-01-31\")\n",
    "print(f\"Diario jan/2025: {len(df_diario)} dias\")\n",
    "df_diario.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 11. Cache DuckDB\n",
    "\n",
    "O agrobr mantém cache local em DuckDB. Rode a célula abaixo 2x e compare o tempo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "t0 = time.time()\n",
    "df, meta = datasets.preco_diario(\"soja\", inicio=\"2025-01-01\", return_meta=True)\n",
    "elapsed = time.time() - t0\n",
    "\n",
    "print(f\"Tempo: {elapsed:.2f}s\")\n",
    "print(f\"Cache hit: {meta.from_cache}\")\n",
    "print(f\"Fonte: {meta.selected_source}\")\n",
    "print(f\"Registros: {meta.records_count}\")\n",
    "print(\"\\nDica: rode novamente para ver o cache hit!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 12. Pipeline Completo\n",
    "\n",
    "JOIN entre preço, clima e produção com gráficos e export."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "\n",
    "# --- Dados ---\n",
    "# Preco diario\n",
    "df_preco = datasets.preco_diario(\"soja\", inicio=\"2025-01-01\")\n",
    "\n",
    "# Clima mensal SP (principal praca de preco)\n",
    "df_clima_sp = nasa_power.clima_uf(\"SP\", ano=2025)\n",
    "\n",
    "# Producao por UF (CONAB com fallback inline)\n",
    "try:\n",
    "    df_prod = conab.safras(\"soja\", safra=\"2024/25\")\n",
    "    prod_source = \"CONAB\"\n",
    "except Exception:\n",
    "    df_prod = pd.DataFrame(\n",
    "        {\n",
    "            \"uf\": [\"MT\", \"PR\", \"RS\", \"GO\", \"MS\", \"BA\", \"MG\", \"SP\"],\n",
    "            \"producao\": [45200, 22100, 21800, 16200, 13400, 8500, 7200, 4300],\n",
    "        }\n",
    "    )\n",
    "    prod_source = \"CONAB (estimativa)\"\n",
    "\n",
    "print(f\"Preco: {len(df_preco)} registros\")\n",
    "print(f\"Clima SP: {len(df_clima_sp)} meses\")\n",
    "print(f\"Producao: {len(df_prod)} UFs (fonte: {prod_source})\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# --- Grafico 1: Serie temporal preco + precipitacao ---\n",
    "fig, ax1 = plt.subplots(figsize=(12, 5))\n",
    "\n",
    "ax1.plot(df_preco[\"data\"], df_preco[\"valor\"], color=\"tab:blue\", linewidth=2, label=\"Soja (R$/sc)\")\n",
    "ax1.set_xlabel(\"Data\")\n",
    "ax1.set_ylabel(\"R$/saca 60kg\", color=\"tab:blue\")\n",
    "ax1.tick_params(axis=\"y\", labelcolor=\"tab:blue\")\n",
    "\n",
    "ax2 = ax1.twinx()\n",
    "ax2.bar(\n",
    "    df_clima_sp[\"mes\"],\n",
    "    df_clima_sp[\"precip_acum_mm\"],\n",
    "    width=20,\n",
    "    alpha=0.3,\n",
    "    color=\"tab:cyan\",\n",
    "    label=\"Precipitacao SP (mm)\",\n",
    ")\n",
    "ax2.set_ylabel(\"Precipitacao (mm/mes)\", color=\"tab:cyan\")\n",
    "ax2.tick_params(axis=\"y\", labelcolor=\"tab:cyan\")\n",
    "\n",
    "fig.suptitle(\"Soja: Preco CEPEA vs Precipitacao SP (2025)\", fontsize=14)\n",
    "fig.legend(loc=\"upper left\", bbox_to_anchor=(0.1, 0.92))\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# --- Grafico 2: Producao por UF (barras horizontais) ---\n",
    "top_prod = (\n",
    "    df_prod[df_prod[\"uf\"].notna()][[\"uf\", \"producao\"]]\n",
    "    .sort_values(\"producao\", ascending=True)\n",
    "    .tail(10)\n",
    ")\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(10, 5))\n",
    "ax.barh(top_prod[\"uf\"], top_prod[\"producao\"].astype(float), color=\"tab:green\")\n",
    "ax.set_xlabel(\"Producao (mil toneladas)\")\n",
    "ax.set_title(f\"Top 10 UFs - Producao Soja ({prod_source})\")\n",
    "ax.grid(axis=\"x\", alpha=0.3)\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# --- Export Parquet e CSV ---\n",
    "import os\n",
    "import tempfile\n",
    "\n",
    "outdir = tempfile.mkdtemp(prefix=\"agrobr_\")\n",
    "path_parquet = os.path.join(outdir, \"soja_preco.parquet\")\n",
    "path_csv = os.path.join(outdir, \"clima_sp_2025.csv\")\n",
    "\n",
    "df_preco.to_parquet(path_parquet, index=False)\n",
    "df_clima_sp.to_csv(path_csv, index=False)\n",
    "\n",
    "print(\"Exportado:\")\n",
    "print(f\"  {path_parquet}\")\n",
    "print(f\"  {path_csv}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 13. Modo Async\n",
    "\n",
    "Para pipelines de alta performance, use a API async com `asyncio.gather`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import asyncio\n",
    "import time\n",
    "\n",
    "from agrobr import cepea as cepea_async\n",
    "\n",
    "\n",
    "async def fetch_multiplos():\n",
    "    produtos = [\"soja\", \"milho\", \"cafe\"]\n",
    "    t0 = time.time()\n",
    "    results = await asyncio.gather(\n",
    "        *[cepea_async.indicador(p, inicio=\"2025-01-01\") for p in produtos],\n",
    "        return_exceptions=True,\n",
    "    )\n",
    "    elapsed = time.time() - t0\n",
    "    print(f\"Tempo total (3 produtos em paralelo): {elapsed:.2f}s\\n\")\n",
    "    for prod, res in zip(produtos, results):\n",
    "        if isinstance(res, Exception):\n",
    "            print(f\"  {prod}: erro - {type(res).__name__}\")\n",
    "        else:\n",
    "            print(f\"  {prod}: {len(res)} registros\")\n",
    "\n",
    "\n",
    "# Compativel com Jupyter/Colab e script\n",
    "try:\n",
    "    loop = asyncio.get_running_loop()\n",
    "    import nest_asyncio\n",
    "\n",
    "    nest_asyncio.apply()\n",
    "    loop.run_until_complete(fetch_multiplos())\n",
    "except RuntimeError:\n",
    "    asyncio.run(fetch_multiplos())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": "## 14. Links\n\n| Recurso | Link |\n|---------|------|\n| PyPI | [pypi.org/project/agrobr](https://pypi.org/project/agrobr/) |\n| Documentação | [www.agrobr.dev/docs](https://www.agrobr.dev/docs/) |\n| GitHub | [github.com/bruno-portfolio/agrobr](https://github.com/bruno-portfolio/agrobr) |\n| Changelog | [CHANGELOG.md](https://github.com/bruno-portfolio/agrobr/blob/main/CHANGELOG.md) |\n| Colab | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/bruno-portfolio/agrobr/blob/main/examples/agrobr_demo.ipynb) |\n\n**agrobr v0.9.0** — 1529 testes, ~75% cobertura, 13/13 golden tests, resiliência HTTP completa."
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
