# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

TealFlowMCP is an MCP (Model Context Protocol) server that enables LLMs to discover, understand, and generate Teal R Shiny applications for clinical trial data analysis. The server provides tools to work with `teal.modules.clinical` and `teal.modules.general` packages programmatically.

**Tech Stack:**
- Python 3.10+ (server implementation)
- R (target application language)
- MCP protocol via FastMCP
- Teal framework for Shiny apps
- CDISC ADaM datasets (clinical data standard)

## Development Commands

### Environment Setup
```bash
# Install dependencies
uv sync

# Verify installation
uv run python tests/test_mcp_server.py
```

### Testing
```bash
# Run all tests
uv run python -m pytest tests/ -v

# Run specific test file
uv run python -m pytest tests/test_mcp_server.py -v

# Run single test
uv run python -m pytest tests/test_discovery.py::TestDatasetDiscovery::test_discover_rds_files -v

# Run with coverage
uv run python -m pytest tests/ --cov=tealflow_mcp --cov-report=term-missing -v
```

### Running the MCP Server
```bash
# Run server directly
uv --directory /absolute/path/to/TealFlowMCP/ run tealflow_mcp.py

# Test with MCP inspector
npx @modelcontextprotocol/inspector uv --directory /absolute/path/to/TealFlowMCP/ run tealflow_mcp.py
```

### Code Quality

```bash
# Check linting issues
uv run ruff check tealflow_mcp/ tests/

# Auto-fix linting issues
uv run ruff check tealflow_mcp/ tests/ --fix

# Format code
uv run ruff format tealflow_mcp/ tests/

# Type checking
uv run mypy tealflow_mcp/

# Run all checks (same as CI)
uv run ruff check tealflow_mcp/ tests/ && \
uv run ruff format tealflow_mcp/ tests/ --check && \
uv run mypy tealflow_mcp/ && \
uv run python -m pytest tests/ -v
```

See [LINTER_SETUP.md](LINTER_SETUP.md) for detailed linter configuration and [CI_SETUP.md](CI_SETUP.md) for GitHub Actions CI pipeline.

## Development Principles

### Test-Driven Development (TDD)

**CRITICAL RULE: Always write unit tests BEFORE implementing code changes.**

When making any code changes:

1. **Write tests first** - Create comprehensive unit tests that verify the desired behavior
2. **Run tests** - Ensure they fail initially (red phase)
3. **Implement code** - Write the minimal code to make tests pass (green phase)
4. **Refactor** - Clean up code while keeping tests passing

**This rule applies to:**
- Bug fixes - Write tests that expose the bug first
- New features - Write tests that define the feature behavior
- Refactoring - Write tests that preserve existing behavior
- Performance improvements - Write tests that verify correctness is maintained

**Never:**
- Write implementation code before tests
- Skip writing tests because "it's a small change"
- Propose code changes without corresponding test changes

**Example workflow:**
```bash
# 1. Write tests in tests/test_*.py
# 2. Run tests to verify they fail
uv run python -m pytest tests/test_module.py::TestClass::test_new_feature -v

# 3. Implement the feature
# 4. Run tests to verify they pass
uv run python -m pytest tests/test_module.py::TestClass::test_new_feature -v

# 5. Run full test suite
uv run python -m pytest tests/ -v
```

## Architecture

### Core Structure
The codebase follows a modular Python package structure:

```
TealFlowMCP/
├── tealflow_mcp.py              # MCP server entrypoint (FastMCP registration)
├── tealflow_mcp/                # Main Python package
│   ├── core/                    # Enums, constants, shared utilities
│   ├── data/                    # Data loaders for JSON knowledge base
│   ├── models/                  # Pydantic input models for MCP tools
│   ├── tools/                   # MCP tool implementations
│   │   ├── agent_guidance.py   # Returns guidance document for LLMs
│   │   ├── list_modules.py     # Lists available Teal modules
│   │   ├── module_details.py   # Module parameter details
│   │   ├── code_generation.py  # R code generation
│   │   ├── dataset_discovery.py # Scans directories for ADaM datasets
│   │   ├── check_shiny_startup.py # Validates Shiny app startup
│   │   ├── setup_renv.py       # R environment setup with renv
│   │   └── other_tools.py      # Dataset requirements, search, templates
│   └── utils/                   # Formatters and validators
├── knowledge_base/              # JSON metadata and R templates
│   ├── agent.md                 # Complete agent usage guide (CRITICAL)
│   ├── app.template.R           # Base Teal app template
│   ├── data.R                   # Data loading script
│   ├── *.json                   # Module metadata by analysis type
│   └── *.Rds                    # Sample ADaM datasets
├── sample_data/                 # Sample clinical trial datasets
└── tests/                       # Pytest test suite
```

### Key Architectural Patterns

**1. Tool Registration Pattern**
- `tealflow_mcp.py` is the entrypoint that registers all MCP tools using FastMCP
- Each tool is implemented in `tealflow_mcp/tools/` as an async function
- Tools accept Pydantic input models from `tealflow_mcp/models/`
- Tools return formatted strings (markdown or JSON) based on `response_format` parameter

**2. Knowledge Base as Data**
- All Teal module metadata is stored in JSON files in `knowledge_base/`
- Data loaders in `tealflow_mcp/data/loaders.py` read and parse JSON
- This enables updates to module information without code changes
- Structure: modules by package → by analysis type → module details

**3. R Code Generation**
- Templates in `knowledge_base/` (e.g., `app.template.R`) provide base structure
- Code generation tools assemble R code from module metadata + templates
- Generated code uses Flow's standard dataset configuration and variables

**4. Path Resolution Strategy**
- MCP servers run in a different working directory than the client
- All file operations **require absolute paths** (not relative paths)
- Discovery tools validate paths and provide clear error messages
- Critical for `tealflow_discover_datasets` and `tealflow_setup_renv_environment`

## Important Implementation Details

### Agent Guidance System
The `tealflow_agent_guidance` tool returns the contents of `knowledge_base/agent.md`, which contains comprehensive instructions for LLMs using this MCP server. This guidance document:
- Defines LLM role and responsibilities
- Provides step-by-step workflows for common tasks
- Explains Teal framework concepts
- Documents tool usage patterns and constraints
- **MUST be called first** in any Teal-related conversation

### Dataset Discovery
The `tealflow_discover_datasets` tool scans directories for ADaM dataset files:
- Supports `.Rds` and `.csv` files (case-insensitive extensions)
- Extracts ADaM dataset names from complex filenames (e.g., `project_ADSL_2024.Rds` → `ADSL`)
- Handles various naming conventions and normalizes to uppercase
- Returns metadata: file paths, formats, sizes, standard vs custom datasets

### R Environment Setup (renv)
The `tealflow_setup_renv_environment` tool manages R package installation:
- Respects existing `renv.lock` files (restores pinned versions)
- Only installs missing required packages: `shiny`, `teal`, `teal.modules.general`, `teal.modules.clinical`
- Creates/updates `global.R` with library calls for renv tracking
- Snapshots environment to ensure reproducibility

### Shiny App Validation
The `tealflow_check_shiny_startup` tool validates app startup without user interaction:
- Runs app with timeout (default 15s) to detect errors
- Classifies errors: `missing_package`, `syntax_error`, `object_not_found`, `timeout`, etc.
- Returns structured JSON with error type and relevant logs
- Used to verify apps before interactive launch

## Testing Strategy

Tests are organized by tool/feature:
- `test_mcp_server.py` - MCP server initialization and tool registration
- `test_discovery.py` - Dataset discovery functionality
- `test_extract_adam_name.py` - ADaM name extraction from filenames
- `test_check_shiny_startup.py` - Shiny app startup validation
- `test_setup_renv.py` - R environment setup

All tests use pytest and can be run individually or as a suite.

## Module Metadata Structure

Module information is stored across multiple JSON files:

**By Package:**
- `teal_modules_clinical_modules_requirements.json` - Clinical module parameters
- `teal_modules_general_modules_requirements.json` - General module parameters

**By Analysis Type:**
- `teal_modules_clinical_by_analysis_type.json` - Maps analysis types to clinical modules
- `teal_modules_general_by_analysis_type.json` - Maps analysis types to general modules

**Dataset Requirements:**
- `teal_modules_clinical_dataset_requirements.json` - Required ADaM datasets per module

This structure enables efficient searching by analysis type and validation of dataset compatibility.

## Critical Path Workflows

### Creating a Teal App
1. **Environment Setup**: Call `tealflow_setup_renv_environment` to install R packages
2. **Dataset Discovery**: Call `tealflow_discover_datasets` with absolute path to find available datasets
3. **Get Template**: Call `tealflow_get_app_template` for base app structure
4. **Find Modules**: Call `tealflow_search_modules_by_analysis` to find relevant modules
5. **Verify Compatibility**: Call `tealflow_check_dataset_requirements` for each module
6. **Generate Code**: Call `tealflow_generate_module_code` for each module
7. **Validate Startup**: Call `tealflow_check_shiny_startup` to ensure app runs

### Adding a Module
1. Call `tealflow_get_module_details` to understand parameters
2. Call `tealflow_check_dataset_requirements` to verify datasets
3. Call `tealflow_generate_module_code` to get R code
4. Insert code into appropriate section of app.R (modules() section)
5. Call `tealflow_check_shiny_startup` to validate

## R Code Conventions

Generated R code follows these patterns:
- Uses Flow's standard datasets: ADSL, ADTTE, ADRS, ADQS, ADAE
- Configuration variables: `arm_vars`, `strata_vars`, `facet_vars`, etc.
- Helper variables: `cs_arm_var`, `cs_strata_var` (choices_selected wrappers)
- Module registration in `modules()` list
- Data loading from `knowledge_base/data.R` (or project-specific path)

## Common Pitfalls

1. **Relative Paths**: Never use relative paths for file operations. Always request absolute paths from users.
2. **Missing Agent Guidance**: LLMs must call `tealflow_agent_guidance` first to understand workflows.
3. **Dataset Compatibility**: Always verify dataset requirements before suggesting modules to avoid runtime errors.
4. **renv Lockfiles**: Respect existing `renv.lock` files - don't force version changes.
5. **Shiny Validation Retries**: Limit startup validation retries to 2-3 attempts to avoid infinite loops.

## Debugging Environment-Specific Test Failures

When tests pass locally but fail in CI (or vice versa), **ALWAYS** investigate environment differences FIRST before proposing code changes:

### Systematic Debugging Checklist

1. **Check Version Differences** (FIRST PRIORITY):
   - Python version (local vs CI) - compare outputs
   - Package versions for ALL relevant dependencies
   - System dependencies (R packages via `Rscript -e "packageVersion('pkg')"`)
   - OS/platform differences

2. **Gather Facts Before Proposing Solutions**:
   - Check CI logs for version information
   - Run version check commands locally
   - Ask user to confirm versions if not visible in logs
   - **Do NOT assume or guess version numbers**

3. **Reproduce Locally**:
   - Match the CI environment versions
   - Verify the failure reproduces with matching versions
   - Only proceed to code investigation once reproduced

4. **Understand Root Cause**:
   - Why do different versions behave differently?
   - Is this a regression, deprecation, or intended change?
   - What's the proper fix vs workaround?

### Key Principle

**Do NOT propose code fixes for environment-specific failures until:**
1. Version differences are confirmed (not assumed)
2. The failure is reproduced locally with CI versions
3. The root cause is understood

Jumping to code fixes without understanding the environment difference wastes time and may create fragile workarounds instead of proper solutions.

## Standard ADaM Datasets

Flow's sample data includes:
- **ADSL**: Subject-Level Analysis Dataset (demographics, baseline characteristics)
- **ADTTE**: Time-to-Event Analysis Dataset (survival data)
- **ADRS**: Response Analysis Dataset (tumor response, efficacy endpoints)
- **ADQS**: Questionnaire Analysis Dataset (patient-reported outcomes)
- **ADAE**: Adverse Events Analysis Dataset (safety data)

These datasets follow CDISC standards and are used by `teal.modules.clinical`.
