# Phase 4: User Acceptance Testing

## Summary

Phase 4 adds TypeScript and JavaScript language adapters using tree-sitter, and builds the security analysis layer (secret detection via `detect-secrets`, dependency CVE scanning via `pip-audit` and `npm audit`).

### What Was Built

1. **Tree-sitter Grammar Dependencies** (`pyproject.toml`)
   - Added `tree-sitter-javascript>=0.23`
   - Added `tree-sitter-typescript>=0.23`

2. **Shared Tree-sitter Utilities** (`analysis/adapters/_treesitter_utils.py`)
   - `traverse()` — Depth-first tree traversal using TreeCursor
   - `extract_es_imports()` — ES module import extraction
   - `extract_require_imports()` — CommonJS require() extraction
   - `extract_definitions()` — Function/class/method extraction
   - `extract_entry_points()` — Entry point detection (Express, Fastify, Koa, etc.)
   - `calculate_complexity()` — Cyclomatic complexity calculation

3. **TypeScript Adapter** (`analysis/adapters/typescript_adapter.py`)
   - Handles `.ts` and `.tsx` files
   - Uses separate languages for TS and TSX
   - Import resolution with extension trial

4. **JavaScript Adapter** (`analysis/adapters/javascript_adapter.py`)
   - Handles `.js` and `.jsx` files
   - Supports both ES modules and CommonJS
   - Detects `module.exports` and `exports.*` patterns

5. **Security Data Structures** (`analysis/models.py`)
   - `SecretFinding` — Detected potential secrets
   - `VulnerabilityFinding` — Known CVEs in dependencies
   - `SecurityResult` — Aggregate security analysis
   - Added `security` field to `AnalysisResult`

6. **Secret Detection** (`analysis/security/secrets.py`)
   - `scan_for_secrets()` — Scans files using detect-secrets
   - Skips binary files and files >1MB
   - Returns hashed values only (actual secrets never stored)

7. **Vulnerability Scanning** (`analysis/security/vulnerabilities.py`)
   - `scan_vulnerabilities()` — Scans Python and JS manifests
   - Python: `requirements.txt` or `pyproject.toml` via pip-audit
   - JavaScript: `package.json` with lockfile via npm audit
   - Graceful degradation when tools not installed

8. **Orchestrator Integration** (`analysis/orchestrator.py`)
   - Added Stage 9: Security scanning
   - Security results attached to AnalysisResult

## Automated Testing

Run lint and type checks:

```bash
ruff check src/unfamiliar/
mypy src/unfamiliar/
```

Verify imports:

```bash
python -c "from unfamiliar.analysis.adapters import TypeScriptAdapter, JavaScriptAdapter"
python -c "from unfamiliar.analysis.models import SecurityResult, SecretFinding, VulnerabilityFinding"
python -c "from unfamiliar.analysis.security import scan_for_secrets, scan_vulnerabilities"
python -c "from unfamiliar.analysis import analyze_repository"
```

Verify adapter registration:

```bash
python -c "from unfamiliar.analysis.adapters import create_default_registry; r = create_default_registry(); print(r.supported_languages())"
```

Expected output:
```
{'PythonAdapter': frozenset({'.py'}), 'TypeScriptAdapter': frozenset({'.tsx', '.ts'}), 'JavaScriptAdapter': frozenset({'.jsx', '.js'})}
```

## Manual Testing Steps

### 1. Test Analysis on This Repository

```bash
cd /home/erinmeaker/Documents/source/unfamiliar
python -c "
from pathlib import Path
from unfamiliar.analysis import analyze_repository
result = analyze_repository(Path('.'))
print(f'Files: {len(result.files)}')
print(f'Parsed: {len(result.parsed_files)}')
print(f'Languages: {result.languages}')
print(f'Security: {len(result.security.secrets)} secrets, {len(result.security.vulnerabilities)} vulns')
"
```

Expected:
- Files discovered: ~61
- Parsed files: ~43 (Python files only in this repo)
- Languages: Python and unknown
- Security: pyproject.toml scanned

### 2. Test TypeScript Adapter

Create a test TypeScript file and verify parsing:

```bash
cat > /tmp/test.ts << 'EOF'
import { useState } from 'react';
import type { User } from './types';

export function App(): JSX.Element {
    const [count, setCount] = useState(0);
    if (count > 0) {
        console.log(count);
    }
    return <div>{count}</div>;
}

class UserService {
    async getUser(id: string): Promise<User> {
        return { id, name: 'Test' };
    }
}
EOF

python -c "
from pathlib import Path
from unfamiliar.analysis.adapters import TypeScriptAdapter

adapter = TypeScriptAdapter()
with open('/tmp/test.ts', 'rb') as f:
    content = f.read()

result = adapter.parse_file(Path('test.ts'), content)
print(f'Language: {result.language}')
print(f'Imports: {len(result.imports)}')
for imp in result.imports:
    print(f'  - {imp.target} (relative: {imp.is_relative})')
print(f'Definitions: {len(result.definitions)}')
for d in result.definitions:
    print(f'  - {d.name} ({d.kind}, async={d.is_async})')
print(f'Entry points: {len(result.entry_points)}')
print(f'Complexity: {result.complexity}')
"
```

Expected:
- Language: typescript
- Imports: 2 (react, ./types)
- Definitions: App (function), UserService (class), UserService.getUser (method)
- Complexity: 2 (if statement, && could add more)

### 3. Test JavaScript Adapter with CommonJS

```bash
cat > /tmp/test.js << 'EOF'
const express = require('express');
const { foo } = require('./utils');

const app = express();

app.get('/', (req, res) => {
    res.send('Hello');
});

module.exports = app;

if (require.main === module) {
    app.listen(3000);
}
EOF

python -c "
from pathlib import Path
from unfamiliar.analysis.adapters import JavaScriptAdapter

adapter = JavaScriptAdapter()
with open('/tmp/test.js', 'rb') as f:
    content = f.read()

result = adapter.parse_file(Path('test.js'), content)
print(f'Language: {result.language}')
print(f'Imports: {len(result.imports)}')
for imp in result.imports:
    print(f'  - {imp.target} (relative: {imp.is_relative})')
print(f'Entry points: {len(result.entry_points)}')
for ep in result.entry_points:
    print(f'  - {ep.description} ({ep.entry_type.value})')
"
```

Expected:
- Language: javascript
- Imports: 2 (express, ./utils)
- Entry points: Express app creation, module.exports, require.main === module

### 4. Test Import Resolution

```bash
python -c "
from unfamiliar.analysis.adapters import TypeScriptAdapter

adapter = TypeScriptAdapter()
repo_files = {'src/utils.ts', 'src/utils/index.ts', 'src/components/Button.tsx'}

# Test relative import resolution
tests = [
    ('src/app.ts', './utils', 'src/utils.ts'),
    ('src/app.ts', './utils/', 'src/utils/index.ts'),
    ('src/views/Home.tsx', '../components/Button', 'src/components/Button.tsx'),
]

for importing, target, expected in tests:
    result = adapter.resolve_import(importing, target, repo_files)
    status = '✓' if result == expected else '✗'
    print(f'{status} {importing} -> {target}: {result} (expected: {expected})')
"
```

Expected: All tests pass with ✓

### 5. Test Secret Detection

Create a file with a fake secret:

```bash
cat > /tmp/secrets_test.py << 'EOF'
# Fake AWS key for testing
AWS_KEY = "AKIAIOSFODNN7EXAMPLE"
EOF

python -c "
from pathlib import Path
from unfamiliar.analysis.security import scan_for_secrets

secrets = scan_for_secrets(['/tmp/secrets_test.py'], Path('/tmp').parent)
print(f'Secrets found: {len(secrets)}')
for s in secrets:
    print(f'  - {s.secret_type} at line {s.line_number}')
"
```

Expected: At least one finding (AWS Access Key detection)

### 6. Test Vulnerability Scanning

```bash
python -c "
from pathlib import Path
from unfamiliar.analysis.security import scan_vulnerabilities

findings, manifests, errors = scan_vulnerabilities(Path('.'))
print(f'Manifests scanned: {manifests}')
print(f'Vulnerabilities: {len(findings)}')
print(f'Errors: {errors}')
"
```

Expected: pyproject.toml scanned, vulnerabilities list (may be empty if no known CVEs)

## Expected Behaviors

### Language Detection
- `.ts` files → TypeScriptAdapter with language_typescript()
- `.tsx` files → TypeScriptAdapter with language_tsx()
- `.js` files → JavaScriptAdapter
- `.jsx` files → JavaScriptAdapter

### Import Resolution
- Relative imports (starting with `.`) are resolved to repo files
- Bare imports return None (external packages)
- Extension trial order for TS: .ts, .tsx, .js, .jsx
- Extension trial order for JS: .js, .jsx, .ts, .tsx

### Security Scanning
- Missing tools (pip-audit, npm, detect-secrets) produce warnings, not crashes
- Large files (>1MB) are skipped for secret detection
- Binary files are skipped for secret detection
- Network unavailability is handled gracefully

### Entry Point Detection
- Express/Fastify app creation → API_ROUTE
- `app.listen()` calls → API_ROUTE
- `module.exports` assignment → MAIN
- `if (require.main === module)` → MAIN
- Default export in index files → MAIN

## Edge Cases to Check

1. **Empty files**: Should parse without errors
2. **Syntax errors**: Should return partial results with warning logged
3. **Mixed ES modules and CommonJS**: Both import types should be captured
4. **Path aliases** (`@/components`): Should return None (known limitation)
5. **Network offline**: Vulnerability scanning should fail gracefully
6. **No lockfile for JS**: npm audit should not be attempted

## Phase Completion Checklist

- [ ] All lint checks pass (ruff check)
- [ ] All type checks pass (mypy)
- [ ] All imports work without circular dependencies
- [ ] TypeScript adapter parses .ts and .tsx files
- [ ] JavaScript adapter parses .js and .jsx files
- [ ] CommonJS patterns detected (require, module.exports)
- [ ] ES module patterns detected (import/export)
- [ ] Import resolution works for relative imports
- [ ] Security scanning integrated into analyze_repository()
- [ ] Secret detection finds test secrets
- [ ] Vulnerability scanning handles missing tools gracefully
- [ ] No regressions in Phase 1/2/3 functionality
