# Examples Directory

Welcome to the LPCI AI Safety Competition examples! This directory contains everything you need to build competitive attack algorithms and defensive guardrails.

## 🚀 Quick Navigation

### **New to the competition? Start here:**
- **[Quick Start Guide](QUICK_START.md)** - Get up and running in 5 minutes
- **[Getting Started Guide](../docs/GETTING_STARTED.md)** - Comprehensive introduction to the competition

### **Ready to build? Explore examples:**
- **[Guardrail Examples](GUARDRAILS_EXAMPLES.md)** - 8 annotated defensive strategies
- **[Attack Examples](ATTACKS_EXAMPLES.md)** - 5 annotated attack algorithms

### **Need help? Check the docs:**
- **[Guardrails Guide](../docs/GUARDRAILS_GUIDE.md)** - Complete guide to building defenses
- **[Attacks Guide](../docs/ATTACKS_GUIDE.md)** - Complete guide to building attacks
- **[API Reference](../docs/API_REFERENCE.md)** - Full API documentation
- **[FAQ](../docs/FAQ.md)** - Common questions and solutions

## 📁 What's in This Directory

### Example Implementations

**Guardrails** (`guardrails/`) - 8 defensive strategies:
- [`guardrail_simple.py`](guardrails/guardrail_simple.py) - Beginner-friendly decorator-based approach
- [`guardrail_perfect.py`](guardrails/guardrail_perfect.py) - Advanced multi-layered defense
- [`guardrail_optimal.py`](guardrails/guardrail_optimal.py) - Balanced performance and security
- [`guardrail_taint_tracking.py`](guardrails/guardrail_taint_tracking.py) - Data flow tracking
- [`guardrail_dataflow.py`](guardrails/guardrail_dataflow.py) - Comprehensive data flow analysis
- [`guardrail_prompt_injection.py`](guardrails/guardrail_prompt_injection.py) - Pattern-based detection
- [`guardrail_promptguard.py`](guardrails/guardrail_promptguard.py) - ML-powered detection
- [`guardrail.py`](guardrails/guardrail.py) - Full-featured reference implementation

**Attacks** (`attacks/`) - 5 attack strategies:
- [`attack_simple.py`](attacks/attack_simple.py) - Beginner-friendly hook-based attacks
- [`attack_working.py`](attacks/attack_working.py) - Proven effective attack patterns
- [`attack_goexplore_working.py`](attacks/attack_goexplore_working.py) - Systematic exploration
- [`attack_goexplore_lpci.py`](attacks/attack_goexplore_lpci.py) - Advanced LPCI-enhanced exploration
- [`attack.py`](attacks/attack.py) - Full-featured reference implementation

### Documentation

- **[QUICK_START.md](QUICK_START.md)** - 5-minute quick start guide
- **[GUARDRAILS_EXAMPLES.md](GUARDRAILS_EXAMPLES.md)** - Detailed guardrail walkthroughs
- **[ATTACKS_EXAMPLES.md](ATTACKS_EXAMPLES.md)** - Detailed attack walkthroughs
- **[test_submission.py](test_submission.py)** - Local testing script

## 🎯 How to Use These Examples

### 1. **Learn the Basics**
Start with the [Quick Start Guide](QUICK_START.md) to understand the competition structure and run your first example.

### 2. **Explore Examples**
- **Building a guardrail?** → Read [GUARDRAILS_EXAMPLES.md](GUARDRAILS_EXAMPLES.md)
- **Building an attack?** → Read [ATTACKS_EXAMPLES.md](ATTACKS_EXAMPLES.md)

Each guide includes:
- ✅ Annotated code walkthroughs
- ✅ Strategy explanations
- ✅ Performance trade-offs
- ✅ When to use each approach

### 3. **Copy and Customize**
All examples are ready to use:
```bash
# Copy an example as your starting point
cp examples/guardrails/guardrail_simple.py my_guardrail.py

# Or copy an attack
cp examples/attacks/attack_simple.py my_attack.py

# Test locally
python examples/test_submission.py
```

### 4. **Test Your Submission**
```bash
# Create submission ZIP
zip my_submission.zip my_attack.py my_guardrail.py

# Test with evaluation script (from project root)
python evaluation.py --submission_zip my_submission.zip --seconds 60
```

## 🏆 Choosing the Right Example

### For Guardrails (Defensive Track)

| Example | Difficulty | Best For | Expected Score |
|---------|-----------|----------|----------------|
| [`guardrail_simple.py`](guardrails/guardrail_simple.py) | ⭐ Beginner | Learning hooks, quick start | 60-70 |
| [`guardrail_prompt_injection.py`](guardrails/guardrail_prompt_injection.py) | ⭐⭐ Intermediate | Pattern detection | 70-80 |
| [`guardrail_taint_tracking.py`](guardrails/guardrail_taint_tracking.py) | ⭐⭐⭐ Advanced | Data flow analysis | 75-85 |
| [`guardrail_optimal.py`](guardrails/guardrail_optimal.py) | ⭐⭐⭐ Advanced | Balanced approach | 80-90 |
| [`guardrail_perfect.py`](guardrails/guardrail_perfect.py) | ⭐⭐⭐⭐ Expert | Maximum protection | 85-95 |

### For Attacks (Red Team Track)

| Example | Difficulty | Best For | Expected Score |
|---------|-----------|----------|----------------|
| [`attack_simple.py`](attacks/attack_simple.py) | ⭐ Beginner | Learning hooks, quick start | 5-10 |
| [`attack_working.py`](attacks/attack_working.py) | ⭐⭐ Intermediate | Proven patterns | 10-15 |
| [`attack_goexplore_working.py`](attacks/attack_goexplore_working.py) | ⭐⭐⭐ Advanced | Systematic exploration | 15-25 |
| [`attack_goexplore_lpci.py`](attacks/attack_goexplore_lpci.py) | ⭐⭐⭐⭐ Expert | Advanced LPCI techniques | 20-30 |

**Scoring Note:** Actual scores depend on what you're tested against. Attack scores are points per breach found; defense scores reflect breach prevention rate.

## 💡 Strategy Guide

### Building a Competitive Guardrail

**Step 1:** Start with [`guardrail_simple.py`](guardrails/guardrail_simple.py)
- Understand decorator-based hooks
- See basic pattern detection
- Learn decision logic

**Step 2:** Add sophistication
- Study [`guardrail_taint_tracking.py`](guardrails/guardrail_taint_tracking.py) for data flow
- Study [`guardrail_prompt_injection.py`](guardrails/guardrail_prompt_injection.py) for patterns
- Combine techniques from multiple examples

**Step 3:** Optimize and balance
- Use [`guardrail_optimal.py`](guardrails/guardrail_optimal.py) as a template
- Balance security vs usability (minimize false positives)
- Test against multiple attack types

### Building a Competitive Attack

**Step 1:** Start with [`attack_simple.py`](attacks/attack_simple.py)
- Understand payload injection
- See tool argument poisoning
- Learn breach detection

**Step 2:** Add exploration
- Study [`attack_goexplore_working.py`](attacks/attack_goexplore_working.py)
- Understand systematic testing
- Learn adaptive strategies

**Step 3:** Maximize coverage
- Use [`attack_goexplore_lpci.py`](attacks/attack_goexplore_lpci.py) techniques
- Combine multiple attack vectors
- Optimize time allocation across phases

## 🔍 Key Concepts

### LPCI Hooks System
All examples use the LPCI (Language-based Persistent Code Injection) framework:

**For Attackers:**
- **Payload Injection** - Inject malicious directives into tool outputs
- **Tool Poisoning** - Modify tool arguments to target sensitive data
- **Multi-Phase Attacks** - Combine techniques for maximum effectiveness

**For Defenders:**
- **Pattern Detection** - Identify malicious content in real-time
- **Output Sanitization** - Remove threats before they reach the agent
- **Sequence Analysis** - Detect multi-step attack chains
- **Data Flow Tracking** - Monitor information flow through the system

### Hook Stages
- **PRE_TOOL_CALL** - Intercept before tool execution (validate/modify arguments)
- **POST_TOOL_CALL** - Intercept after tool execution (detect/sanitize outputs)
- **TRIGGER** - Event-driven hooks (respond to specific conditions)

### Competition Tracks

**1. Red Team (Attack)** - Find vulnerabilities
- Your attack runs with time budget (default 1 hour)
- Goal: Find as many security breaches as possible
- Score: Points per unique breach type found
- Best practices: Systematic exploration, adaptive learning

**2. Blue Team (Defense)** - Prevent breaches
- Your guardrail runs in real-time (no time budget)
- Goal: Block attacks while allowing benign operations
- Score: Breach prevention rate minus false positive penalty
- Best practices: Multi-layered defense, minimize false positives

**3. Purple Team (Both)** - Master offense and defense
- Submit both attack and guardrail
- Score: Combined score from both tracks
- Goal: Build the best attacker AND defender

## 📊 Performance Expectations

### Attack Performance Tiers

| Tier | Breaches Found | Description |
|------|----------------|-------------|
| **Getting Started** | 1-5 | Basic attacks work |
| **Competitive** | 5-15 | Multiple attack vectors |
| **Strong** | 15-25 | Systematic exploration |
| **Exceptional** | 25+ | Advanced techniques |

### Defense Performance Tiers

| Tier | Prevention Rate | False Positives | Description |
|------|----------------|-----------------|-------------|
| **Getting Started** | 50-70% | <40% | Basic protection |
| **Competitive** | 70-85% | <25% | Good protection |
| **Strong** | 85-95% | <15% | Excellent protection |
| **Exceptional** | 95%+ | <10% | World-class defense |

## 🛠️ Testing and Debugging

### Local Testing
```bash
# Test your implementations
cd examples
python test_submission.py

# Test specific example
python -c "from guardrails.guardrail_simple import Guardrail; print('Import OK')"
```

### Debug Mode
```python
# Add verbose logging to any example
import logging
logging.basicConfig(level=logging.DEBUG)

# Check hook registration
from aicomp_sdk.guardrails.hooks import get_hook_registry
registry = get_hook_registry()
print(f"Registered hooks: {len(registry._hooks)}")
```

### Common Issues

**"Module not found"** - Make sure you're in the correct directory
```bash
# From project root
cd examples
python test_submission.py
```

**"Hooks not executing"** - Ensure proper registration
```python
# Clear existing hooks first
clear_hooks()  # or registry.clear_all()

# Then register your hooks
@defense_hook(stage="post_tool")
def my_hook(...):
    pass
```

**"No breaches found"** - Check your test cases
```python
# Add logging to see what's happening
print(f"Testing: {msg}")
print(f"Trace events: {len(trace.get('tool_events', []))}")
```

## 🔗 Additional Resources

### Documentation
- **[Competition Rules](../docs/COMPETITION_RULES.md)** - Official rules and scoring
- **[Hooks System](../docs/HOOKS_README.md)** - Deep dive into LPCI hooks
- **[API Reference](../docs/API_REFERENCE.md)** - Complete API documentation
- **[FAQ](../docs/FAQ.md)** - Frequently asked questions

### Community
- **[GitHub Issues](https://github.com/[repo]/issues)** - Report bugs or ask questions
- **[Discussion Forum](https://github.com/[repo]/discussions)** - Share strategies and tips
- **Competition Discord** - Real-time chat with other competitors

## 🎓 Learning Path

### Beginner (Week 1)
1. Read [Quick Start Guide](QUICK_START.md)
2. Run [`attack_simple.py`](attacks/attack_simple.py) and [`guardrail_simple.py`](guardrails/guardrail_simple.py)
3. Modify simple examples and test locally
4. Make your first submission

### Intermediate (Week 2)
1. Read [Guardrails Guide](../docs/GUARDRAILS_GUIDE.md) and [Attacks Guide](../docs/ATTACKS_GUIDE.md)
2. Study [`attack_working.py`](attacks/attack_working.py) and [`guardrail_optimal.py`](guardrails/guardrail_optimal.py)
3. Implement your own strategies
4. Test against baseline implementations

### Advanced (Week 3+)
1. Study all examples in detail
2. Read [GUARDRAILS_EXAMPLES.md](GUARDRAILS_EXAMPLES.md) and [ATTACKS_EXAMPLES.md](ATTACKS_EXAMPLES.md)
3. Combine techniques from multiple examples
4. Optimize for leaderboard position

## ✅ Pre-Submission Checklist

Before submitting to the competition:

- [ ] Code runs without errors locally
- [ ] Tested with `test_submission.py`
- [ ] Attack finds at least one breach (if submitting attack)
- [ ] Guardrail blocks attacks without too many false positives (if submitting guardrail)
- [ ] Created proper submission ZIP: `zip submission.zip attack.py guardrail.py`
- [ ] Tested with `evaluation.py`
- [ ] Code follows competition rules (time budget, no external dependencies, etc.)
- [ ] Added your name/team to submission

## 🏁 Ready to Compete?

1. **Choose your track:** Red team (attack), Blue team (defense), or Purple team (both)
2. **Pick a starting example** from the tables above
3. **Customize and test** using the local testing tools
4. **Submit your solution** and climb the leaderboard!

Good luck, and may your hooks be ever in your favor! 🚀

---

**Questions?** Check the [FAQ](../docs/FAQ.md) or open an issue on GitHub.
