# Translation Troubleshooting Guide

## Overview

This guide covers common translation issues in AITF, their root causes, and step-by-step solutions. Use this when encountering translation failures, quality problems, or unexpected behavior.

## Quick Diagnostic Flowchart

```
Translation Issue
    ↓
Is it an error or quality issue?
    ├─ Error → Check API, configuration, file format
    └─ Quality → Check prompt, model, glossary, validation
    ↓
Identify component:
    ├─ API/Network → Section 1
    ├─ Configuration → Section 2
    ├─ Text Processing → Section 3
    ├─ Prompt/Model → Section 4
    ├─ Response/Validation → Section 5
    └─ Output/Format → Section 6
```

## Section 1: API and Network Issues

### Problem: API Connection Failures

**Symptoms**:
- "Connection refused" errors
- Timeout errors
- "API key invalid" messages

**Diagnostic Steps**:
1. Check API endpoint configuration:
   ```python
   # In config
   api_endpoint = "https://api.openai.com/v1"
   api_key = "sk-..."
   ```

2. Test connectivity:
   ```bash
   curl -H "Authorization: Bearer $API_KEY" \
        https://api.openai.com/v1/models
   ```

3. Check network/proxy settings:
   ```python
   # If using proxy
   config.proxy = "http://proxy:8080"
   ```

**Solutions**:
- Verify API key is valid and has credits
- Check API endpoint URL is correct
- Ensure network allows HTTPS connections
- Configure proxy if needed

### Problem: Rate Limiting

**Symptoms**:
- "Rate limit exceeded" errors
- Intermittent failures
- Slow progress

**Diagnostic Steps**:
1. Check API rate limits:
   ```python
   # OpenAI: typically 60 requests/min for gpt-3.5-turbo
   # Check your plan's limits
   ```

2. Review current settings:
   ```python
   config.max_concurrent_requests = 5  # Too high?
   config.request_delay = 1.0  # Too low?
   ```

3. Check logs for rate limit messages:
   ```bash
   grep "rate limit" logs/translation.log
   ```

**Solutions**:
- Reduce `max_concurrent_requests`
- Increase `request_delay`
- Implement exponential backoff
- Upgrade API plan
- Use multiple API keys with failover

### Problem: API Failover Issues

**Symptoms**:
- Not switching to backup API
- Switching too frequently

**Diagnostic Steps**:
1. Check failover configuration:
   ```python
   config.enable_api_failover = True
   config.api_failover_threshold = 3  # Errors before switch
   config.api_pipeline = [
       {"name": "primary", "endpoint": "...", "key": "..."},
       {"name": "backup", "endpoint": "...", "key": "..."}
   ]
   ```

2. Monitor error counts:
   ```python
   # In TaskExecutor
   logger.info(f"Consecutive errors: {self.consecutive_errors}")
   ```

**Solutions**:
- Adjust `api_failover_threshold`
- Verify backup API is functional
- Check API credentials for each endpoint
- Review failover logs

### Problem: Token Limit Exceeded

**Symptoms**:
- "Maximum context length exceeded" errors
- Truncated responses
- Incomplete translations

**Diagnostic Steps**:
1. Calculate token usage:
   ```python
   # Rough estimate: 1 token ≈ 4 characters for English
   # Japanese/Chinese: 1 token ≈ 1-2 characters
   batch_tokens = estimate_tokens(batch_text)
   prompt_tokens = estimate_tokens(system_prompt)
   total = batch_tokens + prompt_tokens
   ```

2. Check model limits:
   - GPT-3.5-turbo: 4096 tokens
   - GPT-4: 8192 tokens
   - GPT-4-32k: 32768 tokens

**Solutions**:
- Reduce batch size:
  ```python
  config.batch_size = 10  # Try smaller
  ```
- Simplify prompt (remove verbose instructions)
- Use model with larger context
- Split very long lines into multiple lines

## Section 2: Configuration Issues

### Problem: Language Detection Failures

**Symptoms**:
- Wrong source language detected
- "Unknown language" errors

**Diagnostic Steps**:
1. Check language configuration:
   ```python
   config.source_language = "Japanese"  # Explicit setting
   config.target_language = "Chinese"
   ```

2. Test auto-detection:
   ```python
   # In TranslatorUtil
   detected = get_source_language_for_file(file_path, content)
   print(f"Detected: {detected}")
   ```

**Solutions**:
- Set language explicitly in CLI:
  ```bash
  uv run aitf_cli.py translate file.txt -s Japanese -t Chinese
  ```
- Ensure source text has enough content for detection
- Check encoding (should be UTF-8)

### Problem: Profile Not Loading

**Symptoms**:
- Config settings not applied
- Using default settings instead

**Diagnostic Steps**:
1. Check profile name:
   ```bash
   uv run aitf_cli.py translate file.txt -p my_profile
   ```

2. Verify profile exists:
   ```bash
   ls Resource/profiles/my_profile/
   ```

3. Check profile structure:
   ```
   profiles/
     my_profile/
       config.json       # Profile config
       glossary.json     # Optional glossary
   ```

**Solutions**:
- Create missing profile:
  ```bash
  python aitf_cli.py profile create my_profile
  ```
- Check profile JSON syntax
- Verify config keys match expected schema

### Problem: Config Values Not Taking Effect

**Symptoms**:
- Changed config but behavior unchanged

**Diagnostic Steps**:
1. Check config hierarchy:
   - Default config (`default_config.py`)
   - Profile config (`profiles/.../config.json`)
   - CLI arguments
   - Priority: CLI > Profile > Default

2. Verify config loading:
   ```python
   # In TaskConfig
   logger.debug(f"Batch size: {self.batch_size}")
   ```

**Solutions**:
- Use CLI args for overrides
- Check profile JSON for typos
- Verify config keys in `ConfigRegistry.py`

### Problem: Cache Corruption

**Symptoms**:
- "Failed to load cache" errors
- Missing translations
- Inconsistent state

**Diagnostic Steps**:
1. Check cache files:
   ```bash
   ls -la Resource/cache/project_name/
   ```

2. Validate cache JSON:
   ```python
   import gzip, json
   with gzip.open('cache/file.json.gz', 'rt') as f:
       data = json.load(f)
       print(f"Valid: {bool(data)}")
   ```

3. Check cache statistics:
   ```python
   cache_manager.get_statistics()
   ```

**Solutions**:
- Clear cache for affected file:
  ```python
  cache_manager.clear_file_cache(file_path)
  ```
- Delete entire cache directory and retranslate
- Check disk space and permissions

## Section 3: Text Processing Issues

### Problem: Placeholder Corruption

**Symptoms**:
- `{player_name}` becomes `{玩家名字}` (translated)
- Placeholders missing in output
- Extra placeholders in output

**Diagnostic Steps**:
1. Check exclusion patterns:
   ```python
   # In config or regex.json
   exclusion_patterns = [
       r"\{[^}]+\}",  # Should match {placeholders}
   ]
   ```

2. Review preprocessing output:
   ```python
   # In TextProcessor.replace_all()
   print(f"Placeholders found: {placeholder_order}")
   ```

3. Check restoration:
   ```python
   # In TextProcessor.restore_all()
   print(f"Restored text: {final_text}")
   ```

**Solutions**:
- Add pattern to exclusion list
- Check regex syntax is correct
- Verify placeholder format is consistent
- Add to pre-translation rules if non-standard format

### Problem: Format Loss (Line Breaks, Spacing)

**Symptoms**:
- Multiple lines merged into one
- Lost indentation
- Wrong line endings (`\n` vs `\r\n`)

**Diagnostic Steps**:
1. Check line ending normalization:
   ```python
   # In TextProcessor._normalize_line_endings()
   normalized, line_endings = normalize(text)
   print(f"Line endings: {line_endings}")
   ```

2. Check whitespace handling:
   ```python
   # In TextProcessor.replace_all()
   whitespace_storage = extract_whitespace(text)
   ```

3. Verify restoration:
   ```python
   # In TextProcessor.restore_all()
   restored = restore_whitespace(text, whitespace_storage)
   ```

**Solutions**:
- Check `preserve_line_endings` config
- Verify whitespace extraction regex
- Check post-processing rules don't strip whitespace

### Problem: Special Characters Not Preserved

**Symptoms**:
- `<br>` tags removed
- HTML entities corrupted
- Escape sequences broken

**Diagnostic Steps**:
1. Identify character types:
   - HTML: `<br>`, `&nbsp;`
   - Escape: `\n`, `\t`, `\\`
   - Markup: `**bold**`, `*italic*`

2. Check exclusion patterns:
   ```python
   patterns = config.exclusion_list_data
   print(f"Patterns: {patterns}")
   ```

**Solutions**:
- Add pattern to exclusion list:
  ```json
  ["<[^>]+>", "&\\w+;", "\\\\[nrt]"]
  ```
- Use pre-translation rules to protect:
  ```python
  pre_translation_data = [
      {"pattern": "<br>", "replacement": "[[BR]]"}
  ]
  ```
- Restore in post-translation rules:
  ```python
  post_translation_data = [
      {"pattern": "\\[\\[BR\\]\\]", "replacement": "<br>"}
  ]
  ```

### Problem: Regex Rules Not Applying

**Symptoms**:
- Pre/post rules ignored
- Text not normalized as expected

**Diagnostic Steps**:
1. Check regex syntax:
   ```python
   import re
   pattern = r"\"(\w+)\""  # Test pattern
   test = "\"hello\""
   result = re.sub(pattern, '"\\1"', test)
   print(result)  # Should be "hello"
   ```

2. Verify rule loading:
   ```python
   # In TextProcessor.__init__()
   print(f"Pre-translation rules: {len(self.pre_translation_rules_compiled)}")
   ```

3. Check rule application:
   ```python
   # In TextProcessor
   text = apply_rules(text, self.pre_translation_rules_compiled)
   ```

**Solutions**:
- Fix regex syntax (escape special chars)
- Ensure rules are in correct config section
- Test regex independently
- Check for conflicting rules

## Section 4: Prompt and Model Issues

### Problem: Poor Translation Quality

**Symptoms**:
- Unnatural phrasing
- Wrong context interpretation
- Inconsistent style

**Diagnostic Steps**:
1. Review current prompt:
   ```python
   # In TranslatorTask.prepare()
   with open('prompt_debug.txt', 'w') as f:
       f.write(f"System:\n{self.system_prompt}\n\n")
       f.write(f"User:\n{self.messages}")
   ```

2. Check prompt template:
   ```bash
   cat Resource/Prompt/Translate/common_system_zh.txt
   ```

3. Test with different model:
   ```python
   config.model = "gpt-4"  # Try more capable model
   ```

**Solutions**:
- Add domain-specific instructions to prompt
- Include few-shot examples
- Use specialized prompt (Sakura for Japanese→Chinese)
- Add style guidelines
- Increase temperature for creativity

### Problem: Inconsistent Terminology

**Symptoms**:
- Same term translated differently
- Key terms not recognized

**Diagnostic Steps**:
1. Check glossary:
   ```python
   glossary = config.glossary_data
   print(f"Glossary entries: {len(glossary)}")
   ```

2. Verify glossary format:
   ```json
   {
     "魔法": "Magic",
     "剣": "Sword"
   }
   ```

3. Check prompt includes glossary:
   ```python
   # In PromptBuilder
   if self.glossary:
       prompt += f"\n\n术语表：\n{self.format_glossary()}"
   ```

**Solutions**:
- Add missing terms to glossary
- Enable glossary enforcement plugin
- Increase glossary visibility in prompt
- Use consistent terminology throughout project

### Problem: Model Hallucinations

**Symptoms**:
- Extra content not in source
- Made-up translations
- Invented details

**Diagnostic Steps**:
1. Check response:
   ```python
   # In ResponseExtractor
   if hallucination_detected:
       logger.warning(f"Hallucination: {translation}")
   ```

2. Review prompt instructions:
   - Are instructions clear about not adding content?
   - Is "忠实原文" (faithful to original) included?

**Solutions**:
- Lower temperature (0.3-0.5)
- Add explicit anti-hallucination instructions:
  ```
  严格忠实原文，不要添加、删除或修改任何内容。
  只翻译，不要解释或补充。
  ```
- Use more deterministic model
- Strengthen validation checks

### Problem: Wrong Prompt Template Used

**Symptoms**:
- Unexpected output format
- Missing instructions

**Diagnostic Steps**:
1. Check prompt builder selection:
   ```python
   # In TaskExecutor
   if config.prompt_builder == "sakura":
       builder = PromptBuilderSakura(config)
   elif config.prompt_builder == "local":
       builder = PromptBuilderLocal(config)
   ```

2. Verify template file:
   ```bash
   ls Resource/Prompt/Translate/
   # Should see: common_system_zh.txt, sakura_system_zh.txt, etc.
   ```

**Solutions**:
- Set correct prompt builder in config:
  ```python
  config.prompt_builder = "sakura"  # or "standard", "local"
  ```
- Create custom template if needed
- Check template file exists

## Section 5: Response and Validation Issues

### Problem: Response Extraction Failures

**Symptoms**:
- "Failed to extract translation" errors
- Empty translations
- Partial extractions

**Diagnostic Steps**:
1. Check raw response:
   ```python
   # In LLMRequester
   raw_response = response.text
   print(f"Raw API response:\n{raw_response}")
   ```

2. Check extraction logic:
   ```python
   # In ResponseExtractor
   # Try different formats:
   # Format 1: <textarea>...</textarea>
   # Format 2: 1. Translation
   # Format 3: 【1】Translation
   ```

3. Validate line count:
   ```python
   source_lines = len(source_dict)
   extracted_lines = len(extracted_dict)
   if source_lines != extracted_lines:
       logger.error("Line count mismatch")
   ```

**Solutions**:
- Check prompt specifies output format
- Update extraction patterns for new format
- Handle edge cases (empty lines, special chars)
- Add fallback extraction methods

### Problem: Validation Failures

**Symptoms**:
- "Translation too short/long" warnings
- "Placeholder mismatch" errors
- "Wrong language detected"

**Diagnostic Steps**:
1. Check validation thresholds:
   ```python
   # In ResponseChecker
   min_ratio = 0.5  # Adjust based on language pair
   max_ratio = 2.0
   ```

2. Review specific check:
   ```python
   # In AdvancedChecks
   def check_placeholders(source, translation):
       source_ph = extract_placeholders(source)
       trans_ph = extract_placeholders(translation)
       missing = set(source_ph) - set(trans_ph)
       if missing:
           return f"Missing placeholders: {missing}"
   ```

3. Disable problematic check:
   ```python
   config.disable_length_check = True  # Temporary
   ```

**Solutions**:
- Adjust validation thresholds
- Fix source of validation failure (e.g., placeholder extraction)
- Make checks more lenient for certain content types
- Disable check if false positive

### Problem: Empty or Null Responses

**Symptoms**:
- Empty strings in cache
- "Null response" errors

**Diagnostic Steps**:
1. Check API response status:
   ```python
   if response.status_code != 200:
       logger.error(f"API error: {response.status_code}")
   ```

2. Check response content:
   ```python
   content = response.json()
   if not content.get('choices'):
       logger.error("No choices in response")
   ```

3. Check for content filters:
   ```python
   if 'error' in content:
       if content['error']['code'] == 'content_filter':
           logger.warning("Content filtered by API")
   ```

**Solutions**:
- Handle API errors gracefully
- Retry with modified content if filtered
- Check API quotas and limits
- Log full response for debugging

## Section 6: Output and Format Issues

### Problem: File Format Not Supported

**Symptoms**:
- "Unknown file format" error
- File not read correctly
- Output format wrong

**Diagnostic Steps**:
1. Check file extension:
   ```python
   ext = os.path.splitext(file_path)[1]
   print(f"Extension: {ext}")
   ```

2. Check available readers/writers:
   ```bash
   ls ModuleFolders/Domain/FileReader/
   ls ModuleFolders/Domain/FileOutputer/
   ```

3. Test format detection:
   ```python
   # In FileReader
   reader = get_reader(file_path)
   print(f"Reader: {type(reader)}")
   ```

**Solutions**:
- Convert to supported format
- Add custom reader/writer
- Use generic format (TXT, JSON)
- Check file is not corrupted

### Problem: Output File Corruption

**Symptoms**:
- Output file invalid
- Encoding errors when opening
- Missing content

**Diagnostic Steps**:
1. Check file encoding:
   ```python
   with open(output_path, 'r', encoding='utf-8') as f:
       content = f.read()
   ```

2. Check writer logic:
   ```python
   # In specific writer (e.g., JsonWriter)
   json.dump(data, f, ensure_ascii=False, indent=2)
   ```

3. Verify cache state:
   ```python
   cache_items = cache_manager.get_file_items(file_path)
   print(f"Cached items: {len(cache_items)}")
   ```

**Solutions**:
- Use UTF-8 encoding consistently
- Check disk space and permissions
- Validate cache before writing
- Use atomic write operations

### Problem: Incorrect Output Structure

**Symptoms**:
- JSON structure wrong
- Missing keys/values
- Wrong nesting

**Diagnostic Steps**:
1. Compare input and output:
   ```python
   input_data = json.load(open(input_file))
   output_data = json.load(open(output_file))
   # Compare structure
   ```

2. Check writer implementation:
   ```python
   # In writer
   def write(self, items, output_path):
       # Should preserve structure
       pass
   ```

**Solutions**:
- Review writer for format specifics
- Add format-specific validation
- Test with minimal example
- Check FileReader/FileWriter alignment

## General Debugging Tips

### Enable Verbose Logging

```python
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
```

### Inspect Intermediate States

```python
# In TranslatorTask
def translate_batch(self, items):
    # Log each stage
    logger.debug(f"Source: {source_dict}")
    logger.debug(f"Processed: {processed_dict}")
    logger.debug(f"Prompt: {prompt}")
    logger.debug(f"Response: {response}")
    logger.debug(f"Extracted: {extracted}")
    logger.debug(f"Restored: {final}")
```

### Test in Isolation

```python
# Test single component
from ModuleFolders.Domain.TextProcessor.TextProcessor import TextProcessor

processor = TextProcessor(config)
text = "Test {placeholder} text"
processed, storage = processor.replace_all(text, ...)
restored = processor.restore_all(processed, storage)
assert text == restored
```

### Use CLI Debug Mode

```bash
uv run aitf_cli.py translate test.txt -o output --debug
```

## Getting Help

### Collect Diagnostic Information

```bash
# System info
python --version
uv --version

# Config dump
python -c "from ModuleFolders.Infrastructure.TaskConfig.TaskConfig import TaskConfig; import json; print(json.dumps(TaskConfig().__dict__, indent=2))"

# Test translation
uv run aitf_cli.py translate test.txt -o output --debug 2>&1 | tee debug.log
```

### Common Log Locations

- Translation logs: `logs/translation.log`
- API logs: `logs/api.log`
- Error logs: `logs/error.log`

### Report Template

When reporting issues, include:

1. **System Info**: Python version, OS
2. **Config**: Profile name, custom settings
3. **Input**: Sample source text
4. **Expected**: What you expected
5. **Actual**: What happened
6. **Logs**: Relevant log excerpts
7. **Repro Steps**: How to reproduce

## Summary

Troubleshooting translation issues requires:

1. **Systematic Diagnosis**: Identify the failing component
2. **Component Understanding**: Know how each stage works
3. **Logging**: Enable verbose output for details
4. **Testing**: Test components in isolation
5. **Documentation**: Reference workflow docs
6. **Patience**: Translation is complex, issues may be multi-layered

Use this guide as a reference for common issues, but don't hesitate to dive into the code for deeper understanding.
