# 🤗 HuggingFace Hub Setup - Passo a Passo

Guia completo para fazer upload do modelo treinado para o HuggingFace Hub e configurar download automático.

---

## ✅ Pré-requisitos

- [x] Conta HuggingFace Pro (para modelos privados)
- [x] Modelo treinado em `models/domain-classifier/`
- [x] Python 3.10+ instalado

---

## 📋 Passo a Passo

### 1. Instalar HuggingFace CLI

```bash
# No seu ambiente de desenvolvimento
pip install huggingface_hub
```

### 2. Fazer Login no HuggingFace

```bash
# Fazer login (abrirá navegador ou pedirá token)
huggingface-cli login
```

**Ou criar token manualmente:**
1. Acesse: https://huggingface.co/settings/tokens
2. Clique em "New token"
3. Nome: `denes-router-upload`
4. Tipo: **Write** (para fazer upload)
5. Copie o token
6. Cole quando solicitado por `huggingface-cli login`

**Token será salvo em:** `~/.huggingface/token`

### 3. Criar Repositório Privado no HuggingFace

Opção A - Via CLI (recomendado):

```bash
huggingface-cli repo create denes-domain-classifier \
    --type model \
    --private
```

Opção B - Via Web:
1. Acesse: https://huggingface.co/new
2. Nome: `denes-domain-classifier`
3. Tipo: **Model**
4. Visibilidade: **Private** ✅ (você tem Pro!)
5. Clique em "Create model repository"

### 4. Upload do Modelo

Crie o script `upload_to_hf.py`:

```python
#!/usr/bin/env python3
"""Upload trained model to HuggingFace Hub."""

from pathlib import Path
from huggingface_hub import HfApi, create_repo

# Configuração
REPO_ID = "SEU_USERNAME/denes-domain-classifier"  # ⚠️ MUDAR USERNAME!
MODEL_DIR = Path(__file__).parent / "models" / "domain-classifier"

def upload_model():
    """Upload model to HuggingFace Hub."""

    # Verificar se modelo existe
    if not MODEL_DIR.exists():
        raise FileNotFoundError(f"Model not found at {MODEL_DIR}")

    if not (MODEL_DIR / "model.safetensors").exists():
        raise FileNotFoundError("model.safetensors not found in model directory")

    print(f"📦 Uploading model from: {MODEL_DIR}")
    print(f"📤 To repository: {REPO_ID}")
    print()

    # Criar API client
    api = HfApi()

    # Criar repositório (se não existir)
    try:
        create_repo(
            repo_id=REPO_ID,
            repo_type="model",
            private=True,  # Modelo privado (requer HF Pro)
            exist_ok=True
        )
        print("✓ Repository created/verified")
    except Exception as e:
        print(f"⚠️  Repository creation: {e}")

    # Upload de todos os arquivos
    print("\n📤 Uploading files...")

    api.upload_folder(
        folder_path=str(MODEL_DIR),
        repo_id=REPO_ID,
        repo_type="model",
        commit_message="Upload BERT domain classifier model (trained on 1577 examples)",
    )

    print("\n✅ Upload complete!")
    print(f"🔗 View at: https://huggingface.co/{REPO_ID}")
    print()
    print("📝 Next steps:")
    print(f"   1. Update model_loader.py with repo_id: {REPO_ID}")
    print("   2. Test download: python test_hf_download.py")
    print("   3. Deploy to production")


if __name__ == "__main__":
    upload_model()
```

**Editar e executar:**

```bash
# 1. Editar o script: mudar SEU_USERNAME para seu username do HuggingFace
nano upload_to_hf.py
# Mudar linha: REPO_ID = "seu-username-real/denes-domain-classifier"

# 2. Executar upload
python upload_to_hf.py
```

**Output esperado:**
```
📦 Uploading model from: models/domain-classifier
📤 To repository: seu-username/denes-domain-classifier

✓ Repository created/verified

📤 Uploading files...
Uploading: 100%|████████████| 638M/638M [01:23<00:00, 7.6MB/s]

✅ Upload complete!
🔗 View at: https://huggingface.co/seu-username/denes-domain-classifier
```

### 5. Verificar Upload

1. Acesse: `https://huggingface.co/SEU_USERNAME/denes-domain-classifier`
2. Verifique arquivos:
   - ✅ `config.json`
   - ✅ `model.safetensors` (~638MB)
   - ✅ `tokenizer_config.json`
   - ✅ `tokenizer.json`
   - ✅ `vocab.txt`
   - ✅ `special_tokens_map.json`

### 6. Atualizar `model_loader.py`

Editar `src/denes_router_classifier/model_loader.py`:

```python
"""Model loading with singleton pattern for efficient inference."""

from pathlib import Path
from typing import Optional
import logging
import os

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

logger = logging.getLogger(__name__)

# Singleton instances
_model: Optional[AutoModelForSequenceClassification] = None
_tokenizer: Optional[AutoTokenizer] = None
_loaded = False

# HuggingFace configuration
HF_REPO_ID = os.getenv("DENES_MODEL_REPO", "SEU_USERNAME/denes-domain-classifier")  # ⚠️ MUDAR!
USE_HF_HUB = os.getenv("DENES_USE_HF_HUB", "true").lower() == "true"


def get_model_path() -> Path:
    """Get the path to the trained model directory.

    Returns:
        Path to models/domain-classifier/ or downloads from HF Hub
    """
    local_path = Path(__file__).parent.parent.parent / "models" / "domain-classifier"

    # If local model exists, use it (development mode)
    if local_path.exists() and (local_path / "model.safetensors").exists():
        logger.info(f"Using local model at {local_path}")
        return local_path

    # Otherwise, return HF cache path (will be downloaded)
    if USE_HF_HUB:
        logger.info(f"Model will be downloaded from HuggingFace: {HF_REPO_ID}")
        return local_path  # Placeholder, actual loading uses from_pretrained

    raise FileNotFoundError(
        f"Model not found locally at {local_path} and HF Hub disabled. "
        "Set DENES_USE_HF_HUB=true to enable auto-download."
    )


def get_model() -> AutoModelForSequenceClassification:
    """Get loaded model (singleton pattern).

    Loads the model on first call and caches it for subsequent calls.
    Downloads from HuggingFace Hub if not available locally.

    Returns:
        Loaded BERT model for sequence classification

    Raises:
        FileNotFoundError: If model directory doesn't exist and HF Hub disabled
        RuntimeError: If model loading fails
    """
    global _model, _loaded

    if not _loaded:
        model_path = get_model_path()

        # Try local model first
        if model_path.exists() and (model_path / "model.safetensors").exists():
            logger.info(f"Loading model from {model_path}...")
            try:
                _model = AutoModelForSequenceClassification.from_pretrained(str(model_path))
                _model.eval()
                _model.to("cpu")
                _loaded = True
                logger.info("Model loaded successfully from local path")
                return _model
            except Exception as e:
                logger.warning(f"Failed to load local model: {e}")

        # Download from HuggingFace Hub
        if USE_HF_HUB:
            logger.info(f"Downloading model from HuggingFace Hub: {HF_REPO_ID}")
            try:
                _model = AutoModelForSequenceClassification.from_pretrained(
                    HF_REPO_ID,
                    use_auth_token=True,  # Usa token do huggingface-cli login
                    cache_dir=str(model_path.parent / ".hf_cache")
                )
                _model.eval()
                _model.to("cpu")
                _loaded = True
                logger.info("Model loaded successfully from HuggingFace Hub")
                return _model
            except Exception as e:
                raise RuntimeError(f"Failed to download/load model from HF Hub: {e}") from e

        raise RuntimeError("Failed to load model from local path and HF Hub disabled")

    return _model


def get_tokenizer() -> AutoTokenizer:
    """Get loaded tokenizer (singleton pattern).

    Loads the tokenizer on first call and caches it for subsequent calls.
    Downloads from HuggingFace Hub if not available locally.

    Returns:
        Loaded tokenizer for the BERT model

    Raises:
        FileNotFoundError: If model directory doesn't exist and HF Hub disabled
        RuntimeError: If tokenizer loading fails
    """
    global _tokenizer

    if _tokenizer is None:
        model_path = get_model_path()

        # Try local tokenizer first
        if model_path.exists() and (model_path / "tokenizer.json").exists():
            logger.info(f"Loading tokenizer from {model_path}...")
            try:
                _tokenizer = AutoTokenizer.from_pretrained(str(model_path), use_fast=True)
                logger.info("Tokenizer loaded successfully from local path")
                return _tokenizer
            except Exception as e:
                logger.warning(f"Failed to load local tokenizer: {e}")

        # Download from HuggingFace Hub
        if USE_HF_HUB:
            logger.info(f"Downloading tokenizer from HuggingFace Hub: {HF_REPO_ID}")
            try:
                _tokenizer = AutoTokenizer.from_pretrained(
                    HF_REPO_ID,
                    use_auth_token=True,
                    cache_dir=str(model_path.parent / ".hf_cache"),
                    use_fast=True
                )
                logger.info("Tokenizer loaded successfully from HuggingFace Hub")
                return _tokenizer
            except Exception as e:
                raise RuntimeError(f"Failed to download/load tokenizer from HF Hub: {e}") from e

        raise RuntimeError("Failed to load tokenizer from local path and HF Hub disabled")

    return _tokenizer


def is_model_loaded() -> bool:
    """Check if model and tokenizer are loaded.

    Returns:
        True if both model and tokenizer are loaded, False otherwise
    """
    return _loaded and _tokenizer is not None


def unload_model() -> None:
    """Unload model and tokenizer from memory.

    Useful for testing or freeing memory. Not typically needed in production.
    """
    global _model, _tokenizer, _loaded

    if _loaded or _tokenizer is not None:
        logger.info("Unloading model and tokenizer...")
        _model = None
        _tokenizer = None
        _loaded = False
        # Force garbage collection
        import gc

        gc.collect()
        logger.info("Model and tokenizer unloaded")
```

**⚠️ IMPORTANTE:** Mudar `HF_REPO_ID` para seu username real!

### 7. Testar Download

Criar `test_hf_download.py`:

```python
#!/usr/bin/env python3
"""Test HuggingFace model download."""

import os
import sys
from pathlib import Path

# Add src to path
sys.path.insert(0, str(Path(__file__).parent / "src"))

# Force HF Hub usage
os.environ["DENES_USE_HF_HUB"] = "true"
os.environ["DENES_MODEL_REPO"] = "SEU_USERNAME/denes-domain-classifier"  # ⚠️ MUDAR!

from denes_router_classifier import classify_domain

print("🧪 Testing HuggingFace Hub download...\n")

# This will trigger download if model not local
result = classify_domain("Busca el clima en Asunción")

print(f"\n✅ Success!")
print(f"   Domain: {result.primary}")
print(f"   Confidence: {result.confidence:.4f}")
print("\n🎉 Model downloaded and working correctly!\n")
```

```bash
# Renomear pasta local para forçar download (backup)
mv models/domain-classifier models/domain-classifier.backup

# Testar download
python test_hf_download.py

# Se funcionar, pode deletar backup
# rm -rf models/domain-classifier.backup
```

### 8. Configurar Produção

No servidor de produção:

```bash
# 1. Login no HuggingFace (uma vez)
pip install huggingface_hub
huggingface-cli login

# 2. Instalar package
cd denes-backend-python
uv add ../denes-router-classifier

# 3. Configurar variáveis de ambiente
echo "DENES_USE_HF_HUB=true" >> .env
echo "DENES_MODEL_REPO=seu-username/denes-domain-classifier" >> .env

# 4. Primeira chamada baixará o modelo automaticamente
uv run python -c "from denes_router_classifier import classify_domain; print(classify_domain('test'))"
```

---

## 🎯 Resumo dos Arquivos a Criar/Modificar

1. **✅ `upload_to_hf.py`** - Script de upload (mudar USERNAME)
2. **✅ `src/denes_router_classifier/model_loader.py`** - Adicionar download automático
3. **✅ `test_hf_download.py`** - Testar download

---

## 📊 Vantagens desta Solução

| Aspecto | Benefício |
|---------|-----------|
| **Custo** | 🆓 Grátis (você tem Pro - modelo privado) |
| **Versionamento** | ✅ Automático com commits |
| **Download** | ✅ Automático no primeiro uso |
| **Privacidade** | 🔒 Modelo privado (só você acessa) |
| **Cache** | ✅ HuggingFace faz cache local (~/.cache/huggingface/) |
| **Bandwidth** | ✅ Ilimitado |

---

## 🔒 Segurança

- ✅ Modelo privado (requer autenticação)
- ✅ Token salvo localmente em `~/.huggingface/token`
- ✅ No servidor, fazer login uma vez: `huggingface-cli login`
- ✅ Token **NUNCA** vai para o git (adicionar ao .gitignore se necessário)

---

## 🆘 Troubleshooting

### Erro: "Repository not found"
```bash
# Verificar se repositório existe
huggingface-cli repo-info seu-username/denes-domain-classifier

# Re-criar se necessário
huggingface-cli repo create denes-domain-classifier --type model --private
```

### Erro: "Authentication required"
```bash
# Re-fazer login
huggingface-cli logout
huggingface-cli login
```

### Download muito lento
```bash
# Cache ficará em ~/.cache/huggingface/
# Modelo será baixado apenas uma vez!
ls -lh ~/.cache/huggingface/hub/
```

---

## ✅ Próximos Passos

Após upload completo:

1. ✅ Verificar modelo em https://huggingface.co/SEU_USERNAME/denes-domain-classifier
2. ✅ Testar download com `test_hf_download.py`
3. ✅ Configurar produção com `huggingface-cli login`
4. ✅ Deploy! 🚀
