---
phase: 04-1-enhance-jq-yq-multiline
plan: 1
type: execute
wave: 1
depends_on: []
files_modified:
  - src/fs_mcp/server.py
autonomous: true

must_haves:
  truths:
    - "Agents can send multiline jq expressions with comments and they execute successfully"
    - "Agents can send multiline yq expressions with comments and they execute successfully"
    - "Complex queries with nested functions and special characters work without escaping issues"
    - "jq/yq syntax errors include helpful context and line numbers"
  artifacts:
    - path: "src/fs_mcp/server.py"
      provides: "query_json tool using temp file approach"
      contains: "jq -f"
    - path: "src/fs_mcp/server.py"
      provides: "query_yaml tool using temp file approach"
      contains: "yq -f"
  key_links:
    - from: "query_json function"
      to: "temporary file"
      via: "writes jq_expression to temp file"
      pattern: "tempfile.*query.*jq"
    - from: "query_yaml function"
      to: "temporary file"
      via: "writes yq_expression to temp file"
      pattern: "tempfile.*query.*yq"
    - from: "subprocess.run"
      to: "temp file path"
      via: "-f flag with temp file path"
      pattern: "\\[.*jq.*-f"
---

<objective>
Enhance query_json and query_yaml tools to handle complex multiline jq/yq expressions using a temp file approach, eliminating command-line escaping issues.

Purpose: Enable agents to execute complex queries with comments, nested functions, and special characters without breaking due to shell escaping limitations.

Output: Modified query_json and query_yaml functions that write query expressions to temporary files and use the -f flag to execute them.
</objective>

<execution_context>
@/Users/luutuankiet/.claude/get-shit-done/workflows/execute-plan.md
@/Users/luutuankiet/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@/Users/luutuankiet/dev/fs-mcp/.planning/PROJECT.md
@/Users/luutuankiet/dev/fs-mcp/.planning/ROADMAP.md
@/Users/luutuankiet/dev/fs-mcp/.planning/STATE.md
@/Users/luutuankiet/dev/fs-mcp/.planning/phases/04-1-enhance-jq-yq-multiline/04-1-CONTEXT.md
@/Users/luutuankiet/dev/fs-mcp/.planning/phases/04-add-jq-and-yq-for-querying-large-json-and-yaml-files/04-02-SUMMARY.md
@/Users/luutuankiet/dev/fs-mcp/src/fs_mcp/server.py
</context>

<tasks>

<task type="auto">
  <name>Task 1: Enhance query_json to use temp file approach</name>
  <files>src/fs_mcp/server.py</files>
  <action>
Modify the query_json function to eliminate command-line escaping issues:

1. **Create temporary file for query:**
   - Use tempfile.NamedTemporaryFile with mode='w', suffix='.jq', delete=False
   - Write jq_expression to the temp file
   - Close and flush the file before subprocess call

2. **Update subprocess command:**
   - Change from: `command = ['jq', '-c', jq_expression, str(validated_path)]`
   - To: `command = ['jq', '-c', '-f', temp_file_path, str(validated_path)]`
   - This passes the query via file instead of command-line argument

3. **Add temp file cleanup:**
   - Wrap subprocess execution in try/finally block
   - In finally clause, delete temp file using os.unlink(temp_file_path)
   - Ensure cleanup happens even if subprocess fails or times out

4. **Enhance error messages:**
   - For jq syntax errors (returncode != 0), parse stderr for line numbers
   - Format errors as: "jq syntax error: [original error]. Check your query for common issues (unclosed brackets, missing semicolons, undefined functions)."
   - Keep existing error handling for FileNotFoundError and TimeoutExpired

5. **Update docstring:**
   - Add multiline query example to docstring showing comments work:
     ```
     **Multiline Queries (with comments):**
     query_json("data.json", '''
     # Filter active items
     .items[] | select(.active == true)
     ''')
     ```
   - Keep existing examples and workflow guidance

**DO NOT:**
- Change timeout value (keep 30 seconds default)
- Change result limiting logic (keep 100 results max)
- Modify path validation or availability checks
- Change compact JSON output format (-c flag)
</action>
  <verify>
Run Python to test the modified function with a complex multiline query:

```bash
cd /Users/luutuankiet/dev/fs-mcp && python3 -c "
from src.fs_mcp.server import initialize, query_json
import json
import tempfile

# Initialize with current directory
initialize(['.'])

# Create test JSON file
test_data = {
    'items': [
        {'id': 1, 'name': 'Alice', 'active': True},
        {'id': 2, 'name': 'Bob', 'active': False},
        {'id': 3, 'name': 'Charlie', 'active': True}
    ]
}

with tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False) as f:
    json.dump(test_data, f)
    test_file = f.name

# Test multiline query with comments
query = '''
# Step 1: Filter active items
.items[] |
select(.active == true) |
# Step 2: Extract name
.name
'''

result = query_json(test_file, query)
print('Test 1 (multiline with comments):', result)

# Test complex nested query
query2 = '''
# Create summary
{
  total: .items | length,
  active: [.items[] | select(.active == true)] | length
}
'''

result2 = query_json(test_file, query2)
print('Test 2 (nested with comments):', result2)

import os
os.unlink(test_file)
"
```

Expected: Both queries execute successfully, returning results without escaping errors. Output should show "Alice" and "Charlie" for test 1, and counts for test 2.
</verify>
  <done>
query_json function uses temp file approach (-f flag), handles multiline queries with comments, cleans up temp files, and provides enhanced error messages. Verification tests pass with complex multiline queries.
</done>
</task>

<task type="auto">
  <name>Task 2: Enhance query_yaml to use temp file approach</name>
  <files>src/fs_mcp/server.py</files>
  <action>
Apply the same temp file pattern to query_yaml function for consistency:

1. **Create temporary file for query:**
   - Use tempfile.NamedTemporaryFile with mode='w', suffix='.yq', delete=False
   - Write yq_expression to the temp file
   - Close and flush the file before subprocess call

2. **Update subprocess command:**
   - Change from: `command = ['yq', '-o', 'json', '-c', yq_expression, str(validated_path)]`
   - To: `command = ['yq', '-o', 'json', '-c', '-f', temp_file_path, str(validated_path)]`
   - Note: Keep the -o json flag for consistent JSON output

3. **Add temp file cleanup:**
   - Wrap subprocess execution in try/finally block
   - In finally clause, delete temp file using os.unlink(temp_file_path)
   - Ensure cleanup happens even if subprocess fails or times out

4. **Enhance error messages:**
   - For yq syntax errors (returncode != 0), parse stderr for line numbers
   - Format errors as: "yq syntax error: [original error]. Check your query for common issues (unclosed brackets, missing semicolons, undefined functions)."
   - Keep existing error handling for FileNotFoundError and TimeoutExpired

5. **Update docstring:**
   - Add multiline query example to docstring matching query_json pattern:
     ```
     **Multiline Queries (with comments):**
     query_yaml("config.yaml", '''
     # Filter active services
     .services[] | select(.active == true)
     ''')
     ```
   - Keep existing examples and workflow guidance

**DO NOT:**
- Change timeout value (keep 30 seconds default)
- Change result limiting logic (keep 100 results max)
- Modify path validation or availability checks
- Remove -o json flag (needed for consistent output)
</action>
  <verify>
Run Python to test the modified function with a complex multiline query:

```bash
cd /Users/luutuankiet/dev/fs-mcp && python3 -c "
from src.fs_mcp.server import initialize, query_yaml
import tempfile

# Initialize with current directory
initialize(['.'])

# Create test YAML file
test_yaml = '''
services:
  - name: web
    active: true
    port: 8080
  - name: cache
    active: false
    port: 6379
  - name: db
    active: true
    port: 5432
'''

with tempfile.NamedTemporaryFile(mode='w', suffix='.yaml', delete=False) as f:
    f.write(test_yaml)
    test_file = f.name

# Test multiline query with comments
query = '''
# Step 1: Filter active services
.services[] |
select(.active == true) |
# Step 2: Extract name and port
{name, port}
'''

result = query_yaml(test_file, query)
print('Test 1 (multiline with comments):', result)

# Test complex nested query
query2 = '''
# Create summary
{
  total: .services | length,
  active: [.services[] | select(.active == true)] | length
}
'''

result2 = query_yaml(test_file, query2)
print('Test 2 (nested with comments):', result2)

import os
os.unlink(test_file)
"
```

Expected: Both queries execute successfully, returning JSON results without escaping errors. Output should show web and db services for test 1, and counts for test 2.
</verify>
  <done>
query_yaml function uses temp file approach (-f flag), handles multiline queries with comments, cleans up temp files, and provides enhanced error messages. Verification tests pass with complex multiline queries.
</done>
</task>

</tasks>

<verification>
1. Both query_json and query_yaml use temp file approach with -f flag
2. Complex multiline queries with comments execute successfully
3. Temp files are cleaned up after execution (check no orphaned files in /tmp)
4. Error messages include helpful context for syntax errors
5. Existing functionality preserved (timeouts, result limiting, path validation)
</verification>

<success_criteria>
1. Agents can send multiline jq/yq expressions with comments without escaping issues
2. Complex queries from CONTEXT.md example (dbt lineage traversal) would execute successfully
3. Temp files are properly created, used, and cleaned up
4. Error messages provide actionable guidance with line numbers when possible
5. Both tools maintain consistency with existing patterns (timeout, result limits, output format)
6. All verification tests pass showing multiline queries work correctly
</success_criteria>

<output>
After completion, create `.planning/phases/04-1-enhance-jq-yq-multiline/04-1-01-SUMMARY.md`
</output>
