# Translation Quality Optimization

## Overview

Translation quality in AiNiee-Next depends on multiple factors: prompt engineering, text preprocessing, model selection, glossary usage, and post-processing. This guide covers strategies for optimizing translation quality across these dimensions.

## Prompt Engineering

### Prompt Template Structure

**Location**: `Resource/Prompt/Translate/`

**Standard Template Components**:

1. **Role Definition**
   ```
   你是一名专业的翻译家，你的任务是把{source_language}文本翻译成{target_language}
   ```

2. **Format Instructions**
   ```
   逐行翻译，不要合并，原始保留文本中序号、标记符、占位符、换行符、转义符、代码调用过程等特殊内容
   ```

3. **Special Handling**
   ```
   ### 占位符说明
   文本中 `{...}` 花括号内的内容可能是占位符或变量，请尽量保持原样不翻译。
   ```

4. **Quality Guidelines**
   ```
   ### 翻译原则
   忠实准确地进行翻译。原作是伟大艺术作品，允许使用直白或粗俗的描述，不应随意删减或篡改
   ```

5. **Output Format**
   ```
   ###以textarea标签输出译文
   <textarea>
   1.{target_language}文本
   </textarea>
   ```

### Prompt Customization Strategies

#### 1. Domain-Specific Prompts

**Technical Documentation**:
```
你是一名技术文档翻译专家。确保术语准确性，保持技术概念的一致性。
代码块、命令、文件路径等技术内容保持原样。
```

**Literary Works**:
```
你是一名文学翻译家，注重文本的文学性、美感和情感表达。
保留原作的风格、语气和节奏，适当使用意译以传达原文意境。
```

**Game Scripts**:
```
你是一名游戏本地化专家。翻译要符合游戏语境，保持角色性格一致性。
对话要自然流畅，符合角色的语气特点。
```

#### 2. Few-Shot Examples

Add examples in prompt template:

```
### 翻译示例
原文：【1】こんにちは、今日は良い天気ですね。
译文：【1】你好，今天天气真好啊。

原文：【2】{player_name}は激しい戦いを繰り広げた。
译文：【2】{player_name}展开了激烈的战斗。
```

#### 3. Style Guidelines

**Formal Style**:
```
使用正式、书面化的语言，避免口语化表达。
```

**Casual Style**:
```
使用轻松、自然的口语化表达，贴近日常对话。
```

**Character Voice**:
```
根据角色特点调整：
- 年轻角色：活泼、现代的表达
- 老年角色：稳重、传统的表达
- 外国人角色：保留独特的语言习惯
```

#### 4. Context Enhancement

**Previous Context**:
```
### 上下文
前文内容：
{previous_lines}

请根据上下文保持翻译的连贯性和一致性。
```

**Scene Description**:
```
### 场景
当前场景：{scene_description}
根据场景氛围调整翻译风格。
```

### Prompt Variables

**Available Placeholders**:
- `{source_language}`: Source language name
- `{target_language}`: Target language name
- `{glossary}`: Formatted glossary entries
- `{previous_lines}`: Previous translation context

**Custom Placeholders**:
Can add custom placeholders via plugin hooks:
```python
def build_rag_context(config, rag_context_data):
    # Add custom context
    rag_context_data["rag_context"] = "Custom context here"
```

## Text Preprocessing Optimization

### Exclusion Patterns

**Location**: `Resource/Regex/regex.json` and config exclusion lists

**Common Patterns to Exclude**:

1. **Code Patterns**
   ```json
   [
     "<[^>]+>",           // HTML tags
     "\\{[^}]+\\}",       // Braced placeholders
     "\\[\\w+\\]",        // Square bracket tags
     "%[sdifr]",          // Printf format
     "\\$\\w+",           // Variable names
     "https?://\\S+"      // URLs
   ]
   ```

2. **Game Variables**
   ```json
   [
     "\\\\n\\[",          // RPG Maker codes
     "\\\\v\\[\\d+\\]",   // Variables
     "\\\\c\\[\\d+\\]",   // Color codes
   ]
   ```

3. **Special Markup**
   ```json
   [
     "<<.*?>>",           // RenPy labels
     "\\*\\*.*?\\*\\*",   // Markdown bold
     "\\_\\_.*?\\_\\_"    // Markdown italic
   ]
   ```

### Pre-Translation Rules

**Purpose**: Normalize text before translation

**Example Rules** (in config):
```python
pre_translation_data = [
    {
        "pattern": ""(\w+)"",        # Smart quotes
        "replacement": '"\\1"',
        "description": "Normalize quotes"
    },
    {
        "pattern": "…",              # Ellipsis
        "replacement": "...",
        "description": "Normalize ellipsis"
    }
]
```

### Post-Translation Rules

**Purpose**: Fix common translation artifacts

**Example Rules**:
```python
post_translation_data = [
    {
        "pattern": "\\s+([。，！？])",  # Space before punctuation
        "replacement": "\\1",
        "description": "Remove space before punctuation"
    },
    {
        "pattern": "([。，！？])\\s+",  # Space after punctuation
        "replacement": "\\1",
        "description": "Remove space after punctuation"
    }
]
```

## Model Selection

### Model Categories

#### 1. General Purpose Models
**Use Cases**: Standard translation, varied content
- GPT-4, GPT-3.5-turbo
- Claude-3, Claude-2
- High versatility, good quality

#### 2. Specialized Translation Models
**Use Cases**: Domain-specific content
- **Sakura**: Japanese→Chinese specialized
  - Location: `Resource/Prompt/Sakura/`
  - Best for: Japanese novels, games
- **Murasaki**: Japanese literary translation
  - Location: `Resource/Prompt/Translate/murasaki_*.txt`
  - Best for: Literary works with complex style

#### 3. Local/Offline Models
**Use Cases**: Privacy, cost, offline needs
- PromptBuilderLocal handles local models
- May need simpler prompts
- Quality varies by model size

### Model Configuration

**Temperature Settings**:
- **0.0-0.3**: Deterministic, consistent
  - Use for: Technical documentation, factual content
- **0.4-0.7**: Balanced creativity
  - Use for: General fiction, dialogue
- **0.8-1.0**: Creative, varied
  - Use for: Creative writing, poetry

**Top-P Settings**:
- **0.9-1.0**: Diverse vocabulary
- **0.7-0.9**: Balanced
- **0.5-0.7**: Conservative, focused

**Max Tokens**:
- Set based on batch size + context
- Typical: 2000-4000 tokens
- Monitor for truncation

## Glossary Management

### Glossary Structure

**Location**: Configured via profile settings

**Format** (JSON example):
```json
{
  "魔法": "Magic",
  "剣": "Sword",
  "勇者": "Hero",
  "魔王": "Demon King"
}
```

**Format** (TXT example):
```
魔法 = Magic
剣 = Sword
勇者 = Hero
```

### Glossary Integration

**Automatic Injection**:
- PromptBuilder adds glossary to system prompt
- Format: `术语表：魔法→Magic, 剣→Sword`

**Plugin Enforcement**:
```python
class GlossaryPlugin(Base):
    def on_preproces_text(self, config, source_dict, prompt_dict):
        glossary = self.load_glossary()
        # Highlight glossary terms
        for term, translation in glossary.items():
            for key, text in prompt_dict.items():
                if term in text:
                    # Mark term for LLM attention
                    prompt_dict[key] = text.replace(
                        term,
                        f"【{term}({translation})】"
                    )
```

### Glossary Best Practices

1. **Consistency**: Use same translation for same term
2. **Context**: Consider polysemy (multiple meanings)
3. **Updates**: Regularly update based on review feedback
4. **Import**: Import from existing translation memories (TMX, XLIFF)

### Term Extraction

**Automatic Extraction** (via GlossaryAnalyzer):
```bash
# Analyze source text for frequent terms
python -m ModuleFolders.Service.GlossaryAnalysis.GlossaryAnalyzer source.txt
```

## Response Validation

### Validation Criteria

**Location**: `ModuleFolders/Domain/ResponseChecker/`

#### Length Checks
```python
def check_length(source, translation):
    ratio = len(translation) / len(source)
    # Typical acceptable range: 0.5 - 2.0
    if ratio < 0.5:
        return "Too short"
    if ratio > 2.0:
        return "Too long"
    return "OK"
```

#### Line Count
```python
def check_line_count(source_dict, translation_dict):
    if len(source_dict) != len(translation_dict):
        return "Line count mismatch"
    return "OK"
```

#### Placeholder Preservation
```python
def check_placeholders(source, translation):
    source_placeholders = re.findall(r'\{[^}]+\}', source)
    trans_placeholders = re.findall(r'\{[^}]+\}', translation)

    if set(source_placeholders) != set(trans_placeholders):
        return "Placeholder mismatch"
    return "OK"
```

#### Language Detection
```python
def check_target_language(translation, expected_lang):
    detected_lang = detect_language(translation)
    if detected_lang != expected_lang:
        return "Wrong target language"
    return "OK"
```

### Validation Tuning

**Adjust Thresholds** (in config):
```python
response_check_config = {
    "min_length_ratio": 0.3,      # Lower for concise translations
    "max_length_ratio": 3.0,      # Higher for verbose languages
    "allow_placeholder_variation": False,
    "strict_line_count": True
}
```

**Disable Checks** (for specific cases):
```python
config.response_checks = {
    "length": False,              # Disable length check
    "placeholder": True,
    "language": True
}
```

## Quality Review Workflow

### Manual Review

1. **Spot Check**: Random sample of translations
2. **Critical Sections**: Key dialogues, important terms
3. **Problem Patterns**: Lines that failed validation previously

### Review Tools

**Cache Editor** (WebServer):
- View translation pairs
- Edit translations manually
- Mark for retranslation

**Comparison View**:
```
Source:     こんにちは、元気ですか？
Current:    你好，你好吗？
Previous:   你好，你好吗？（旧版）
```

### Iterative Improvement

1. **Identify Patterns**: Common issues across translations
2. **Update Prompts**: Add instructions for problematic cases
3. **Expand Glossary**: Add frequently mis-translated terms
4. **Tune Config**: Adjust preprocessing/validation rules
5. **Retranslate**: Re-run with improvements

## Advanced Techniques

### Context-Aware Translation

**Previous Lines Context**:
```python
config.context_lines = 3  # Include 3 previous lines
```

**RAG Context** (via RAGPlugin):
```python
def build_rag_context(config, rag_context_data):
    # Query translation memory for similar segments
    similar = query_translation_memory(source_text)
    rag_context_data["rag_context"] = format_context(similar)
```

### Multi-Pass Translation

**Pass 1**: Rough translation
**Pass 2**: Refinement with glossary
**Pass 3**: Polish for style

**Implementation**:
```python
# Configure multi-pass via Polishing mode
config.enable_polishing = True
config.polishing_passes = 3
```

### Terminology Extraction

**Automatic Term Mining**:
1. Extract frequent noun phrases
2. Check against existing glossary
3. Flag potential new terms
4. Manual confirmation
5. Add to glossary

### Consistency Checking

**Cross-File Consistency**:
```python
def check_consistency(cache_manager):
    # Find same source across files
    translations = cache_manager.find_by_source(source_text)

    if len(set(translations)) > 1:
        return "Inconsistent translations detected"
```

## Troubleshooting Quality Issues

### Common Problems

1. **Inconsistent Terminology**
   - **Cause**: Missing glossary entries
   - **Fix**: Add terms, enable glossary enforcement

2. **Wrong Context Interpretation**
   - **Cause**: Insufficient context
   - **Fix**: Increase `context_lines`, add scene descriptions

3. **Format Loss**
   - **Cause**: Exclusion pattern issues
   - **Fix**: Review regex patterns, add missing patterns

4. **Unnatural Phrasing**
   - **Cause**: Poor prompt instructions
   - **Fix**: Add style guidelines, few-shot examples

5. **Missing Translations**
   - **Cause**: Validation failures
   - **Fix**: Relax validation thresholds, check API responses

### Debugging Tools

**Prompt Inspection**:
```python
# In TranslatorTask.prepare()
print(f"System prompt: {self.system_prompt}")
print(f"User message: {self.messages}")
```

**Response Logging**:
```python
# In ResponseExtractor
logger.debug(f"Raw response: {raw_response}")
logger.debug(f"Extracted: {extracted_dict}")
```

**Cache Inspection**:
```python
# Check cache for patterns
cache_manager.analyze_failures()
```

## Performance vs Quality Tradeoffs

### Speed Optimization
- **Larger batches**: Fewer API calls, but less context per line
- **Lower temperature**: More deterministic, faster
- **Simpler prompts**: Faster processing, but less guidance

### Quality Optimization
- **Smaller batches**: More context, better quality
- **Higher temperature**: More varied, creative
- **Detailed prompts**: Better guidance, but slower

### Balanced Approach
```python
config.batch_size = 15                # Medium batch
config.temperature = 0.7              # Balanced creativity
config.context_lines = 2              # Some context
config.enable_glossary = True         # Consistency
config.response_validation = True     # Quality assurance
```

## Summary

Translation quality optimization requires:

1. **Prompt Engineering**: Clear instructions, examples, domain expertise
2. **Preprocessing**: Proper exclusion, normalization, and restoration
3. **Model Selection**: Right model for the content type
4. **Glossary**: Consistent terminology enforcement
5. **Validation**: Catch quality issues early
6. **Review**: Human oversight for critical content
7. **Iteration**: Continuous improvement based on feedback

Success comes from combining these techniques thoughtfully, testing changes incrementally, and maintaining a feedback loop between translation output and configuration adjustments.
