Audit Logging & Compliance — Harness Tutorials

# Audit Logging & Compliance

Generate tamper-proof, SHA-256 hash-chained audit trails of every agent action — with built-in PII scanning for regulated environments.

Intermediate 20 min read

## 1Why Audit Logging?

Regulatory frameworks don't just require controls — they require proof that controls were enforced. When an AI agent modifies production code, you need a tamper-proof record of every action it took.

Harness audit logging addresses three major compliance frameworks:

- SOC 2 — requires complete audit trails for all system access and changes
- HIPAA — requires access logs for any system that touches patient-adjacent data
- PCI-DSS — requires monitoring and logging of all access to cardholder data environments

⚡ Unique to Harness

No other coding agent provides audit logging. Claude Code has no audit trail. Cursor has no compliance features. Harness gives you SHA-256 hash-chained logs that auditors can verify.

Feature | Harness | Claude Code | Cursor | Copilot
--- | --- | --- | --- | ---
Audit trail | ✓ | ✗ | ✗ | ✗
Hash chain verification | ✓ | ✗ | ✗ | ✗
PII scanning | ✓ | ✗ | ✗ | ✗
Tamper detection | ✓ | ✗ | ✗ | ✗
Retention policies | ✓ | ✗ | ✗ | ✗

## 2Enable Audit Logging

Audit logging is disabled by default. Enable it in your `.harness/config.toml`:

TOML .harness/config.toml

    [audit]
    enabled = true
    scan_pii = true
    retention_days = 90
    retention_max_size_mb = 500
    log_tool_args = true

Once enabled, audit logging is fully automatic — every session creates a new JSONL file:

Bash

    harness "Fix the login bug"
    # Audit logging is automatic when enabled in config
    # Log written to ~/.harness/audit/audit-<id>.jsonl

ℹ Log Location
Audit logs are written to `~/.harness/audit/` by default. Each session gets its own `audit-<id>.jsonl` file. You can override the directory with the `audit_dir` constructor parameter when using the SDK directly.

## 3Run a Session & Inspect the Trail

Run a task that involves reading and editing a file:

Bash

    harness "Add input validation to user_form.py"

The resulting audit log is a JSONL file — one JSON object per line, each linked to the previous via SHA-256:

JSON ~/.harness/audit/audit-abc123.jsonl

    {"event_id": "evt_001", "timestamp": "2025-01-15T10:30:00Z", "event_type": "session_start", "session_id": "abc123", "data": {"provider": "anthropic", "model": "claude-sonnet-4-20250514"}, "prev_hash": "0000000000000000000000000000000000000000000000000000000000000000", "hash": "a1b2c3d4e5f6..."}
    {"event_id": "evt_002", "timestamp": "2025-01-15T10:30:01Z", "event_type": "tool_call", "session_id": "abc123", "data": {"tool": "Read", "args": {"file_path": "user_form.py"}}, "prev_hash": "a1b2c3d4e5f6...", "hash": "e5f6g7h8..."}
    {"event_id": "evt_003", "timestamp": "2025-01-15T10:30:02Z", "event_type": "tool_result", "session_id": "abc123", "data": {"tool": "Read", "is_error": false, "content_length": 1234}, "prev_hash": "e5f6g7h8...", "hash": "i9j0k1l2..."}
    {"event_id": "evt_004", "timestamp": "2025-01-15T10:30:03Z", "event_type": "permission_decision", "session_id": "abc123", "data": {"tool": "Edit", "decision": "allow", "mode": "accept_edits"}, "prev_hash": "i9j0k1l2...", "hash": "m3n4o5p6..."}
    {"event_id": "evt_005", "timestamp": "2025-01-15T10:30:04Z", "event_type": "tool_call", "session_id": "abc123", "data": {"tool": "Edit", "args": {"file_path": "user_form.py", "old_string": "...", "new_string": "..."}}, "prev_hash": "m3n4o5p6...", "hash": "q7r8s9t0..."}
    {"event_id": "evt_006", "timestamp": "2025-01-15T10:30:10Z", "event_type": "provider_call", "session_id": "abc123", "data": {"provider": "anthropic", "model": "claude-sonnet-4-20250514", "input_tokens": 2500, "output_tokens": 800, "cost": 0.0195}, "prev_hash": "q7r8s9t0...", "hash": "u1v2w3x4..."}
    {"event_id": "evt_007", "timestamp": "2025-01-15T10:30:11Z", "event_type": "session_end", "session_id": "abc123", "data": {"turns": 3, "total_tokens": 5200, "total_cost": 0.042}, "prev_hash": "u1v2w3x4...", "hash": "y5z6a7b8..."}

ℹ Hash Chain Structure
Each event contains a SHA-256 hash of the previous event, creating a tamper-proof chain. If any line is modified, the chain breaks. The first event has `"prev_hash"` set to a 64-character string of zeros as the chain anchor.

### Event Types

Event Type | Triggered When
--- | ---
`session_start` | Agent session begins — records provider, model
`tool_call` | Agent invokes a tool (Read, Edit, Bash, etc.)
`tool_result` | Tool execution completes — records success/error, content length
`permission_decision` | Permission system allows or denies a tool call
`provider_call` | LLM API call — records tokens used and cost
`pii_detected` | PII scanner finds a sensitive pattern in tool output
`session_end` | Session completes — records total turns, tokens, cost

## 4Hash Chain Integrity

Use `AuditLogger.verify_chain()` to verify the integrity of any audit log. This is a static method — you don't need a running session:

Python

    from pathlib import Path
    from harness.audit.logger import AuditLogger

    log_path = Path("~/.harness/audit/audit-abc123.jsonl").expanduser()
    valid, errors = AuditLogger.verify_chain(log_path)

    if valid:
        print("✓ Audit chain is intact — no tampering detected")
    else:
        print("✗ Chain broken!")
        for err in errors:
            print(f"  - {err}")

✓ What verify_chain checks

- Every event's `hash` matches the SHA-256 of its own content
- Every event's `prev_hash` matches the `hash` of the preceding event
- The first event has `prev_hash` equal to a 64-character string of zeros
- Returns `(True, [])` if intact, `(False, [error_messages])` if broken

## 5Tamper Detection Demo

The following script demonstrates how tamper detection works. It verifies the chain, modifies an event, then verifies again to show the failure:

- 1
  Run a session to generate an audit log

  Any `harness` command with audit enabled creates a JSONL file.

- 2
  Verify chain — passes

  `verify_chain()` returns `(True, [])`.

- 3
  Edit one line in the JSONL — change "Read" to "Bash"

  Simulates a malicious actor trying to hide that a file was read.

- 4
  Verify chain again — fails, shows which event broke

  The hash of the modified event no longer matches what the next event recorded as `prev_hash`.

Python tamper_demo.py

    # tamper_demo.py
    import json
    from pathlib import Path
    from harness.audit.logger import AuditLogger

    log_path = Path("~/.harness/audit/audit-abc123.jsonl").expanduser()

    # Step 1: Verify before tampering
    valid, errors = AuditLogger.verify_chain(log_path)
    print(f"Before: valid={valid}, errors={len(errors)}")  # valid=True, errors=0

    # Step 2: Tamper with line 2 (change tool name)
    lines = log_path.read_text().strip().split("\n")
    event = json.loads(lines[1])
    event["tool"] = "Bash"  # Change Read to Bash
    lines[1] = json.dumps(event)
    log_path.write_text("\n".join(lines) + "\n")

    # Step 3: Verify after tampering
    valid, errors = AuditLogger.verify_chain(log_path)
    print(f"After: valid={valid}, errors={len(errors)}")  # valid=False, errors=1
    print(f"Broken at: {errors[0]}")  # "Hash mismatch at event evt_003"

⚠ Demo Only
This demo intentionally tampers with an audit log to show detection. In production, audit logs should be on append-only storage (e.g., AWS S3 with Object Lock, or a WORM-compliant log management system) to prevent modification entirely.

## 6PII Scanning

When `scan_pii = true`, Harness inspects all tool outputs for sensitive data patterns before logging. Detections are recorded as `pii_detected` events — the actual value is never stored, only the pattern name and context.

Harness ships with 12 built-in patterns:

Pattern | Example Match
--- | ---
email | user@example.com
phone_us | +1-555-123-4567
ssn | 123-45-6789
credit_card | 4111-1111-1111-1111
aws_access_key | AKIA...
aws_secret_key | wJalrXUtnFEMI/K7MDENG/...
github_token | ghp_xxxxxxxxxxxxxxxxxxxx
jwt | eyJhbGciOi...
slack_token | xoxb-...
private_key_header | -----BEGIN RSA PRIVATE KEY-----
generic_api_key | api_key = "..."
ip_address | 192.168.1.100

Enable it in config:

TOML .harness/config.toml

    [audit]
    enabled = true
    scan_pii = true  # Enable PII scanning on all tool outputs

When PII is detected, it's logged as a separate event:

JSON

    {"event_type": "pii_detected", "session_id": "abc123", "data": {"pattern_name": "email", "context": "Found in tool output for Read(user_data.csv)"}}

ℹ Privacy-Preserving Design
The actual PII value is never stored in the audit log. Only the pattern name (`"email"`, `"ssn"`, etc.) and the context (which tool produced it) are recorded. This satisfies audit requirements without creating a secondary data exposure risk.

## 7Retention Policies

Harness automatically enforces retention policies at session end. Configure time-based or size-based limits, or both:

TOML .harness/config.toml

    [audit]
    retention_days = 90          # Delete logs older than 90 days
    retention_max_size_mb = 500  # Cap at 500MB total

SOC 2 typically requires 12 months of audit log retention:

TOML

    [audit]
    enabled = true
    retention_days = 365          # 12 months
    retention_max_size_mb = 2048  # 2GB cap

HIPAA requires 6 years of access log retention:

TOML

    [audit]
    enabled = true
    scan_pii = true
    retention_days = 2190         # 6 years
    retention_max_size_mb = 0     # No size limit (0 = unlimited)

PCI-DSS requires 12 months with 3 months immediately available:

TOML

    [audit]
    enabled = true
    scan_pii = true
    retention_days = 365
    retention_max_size_mb = 1024

## 8Export for Compliance

Audit logs are JSONL — easy to parse and transform. The script below collects all sessions and produces a single CSV that auditors can review in a spreadsheet:

Python compliance_export.py

    # compliance_export.py
    import json
    import csv
    from pathlib import Path

    audit_dir = Path("~/.harness/audit").expanduser()
    rows = []

    for log_file in sorted(audit_dir.glob("*.jsonl")):
        for line in log_file.read_text().strip().split("\n"):
            event = json.loads(line)
            rows.append({
                "timestamp": event.get("timestamp", ""),
                "session": log_file.stem,
                "event_type": event.get("event_type", ""),
                "tool": event.get("data", {}).get("tool", ""),
                "decision": event.get("data", {}).get("decision", ""),
                "cost": event.get("data", {}).get("cost", 0),
                "tokens": event.get("data", {}).get("input_tokens", 0) + event.get("data", {}).get("output_tokens", 0),
            })

    if not rows:
        print("No audit events found.")
    else:
        with open("audit_report.csv", "w", newline="") as f:
            writer = csv.DictWriter(f, fieldnames=rows[0].keys())
            writer.writeheader()
            writer.writerows(rows)

        print(f"Exported {len(rows)} events to audit_report.csv")

▶ Try It
Run this script with `uv run compliance_export.py` after completing a few Harness sessions. Open `audit_report.csv` in a spreadsheet to browse your agent's full action history, including token costs per session.

## 9Build a Compliance Check Script

Automate compliance verification in CI/CD. This script checks all audit logs for chain integrity and PII detections, then exits non-zero if any issues are found:

Python compliance_check.py

    # compliance_check.py
    import json
    from pathlib import Path
    from harness.audit.logger import AuditLogger

    def run_compliance_check():
        audit_dir = Path("~/.harness/audit").expanduser()
        results = {"passed": 0, "failed": 0, "warnings": 0}

        for log_file in sorted(audit_dir.glob("*.jsonl")):
            # Check 1: Hash chain integrity
            valid, errors = AuditLogger.verify_chain(log_file)
            if valid:
                results["passed"] += 1
                print(f"  ✓ {log_file.name}: chain intact")
            else:
                results["failed"] += 1
                print(f"  ✗ {log_file.name}: chain BROKEN ({len(errors)} errors)")

            # Check 2: PII detections
            pii_events = []
            for line in log_file.read_text().strip().split("\n"):
                event = json.loads(line)
                if event.get("event_type") == "pii_detected":
                    pii_events.append(event)
            if pii_events:
                results["warnings"] += 1
                print(f"  ⚠ {log_file.name}: {len(pii_events)} PII detections")

        print(f"\nResults: {results['passed']} passed, {results['failed']} failed, {results['warnings']} warnings")
        return results["failed"] == 0

    if __name__ == "__main__":
        import sys
        sys.exit(0 if run_compliance_check() else 1)

The full `AuditLogger` API — use this when integrating audit logging into custom pipelines:

Python

    from harness.audit.logger import AuditLogger, AuditEventType
    from pathlib import Path

    # Use as a context manager (auto-closes on exit)
    with AuditLogger(
        session_id="my_session_001",
        enabled=True,
        log_tool_args=True,
        audit_dir=Path("/var/log/harness/audit"),
    ) as logger:
        # Record session start
        logger.log_session_start(provider="anthropic", model="claude-sonnet-4-20250514")

        # Record a tool call
        logger.log_tool_call("Read", args={"file_path": "app.py"})
        logger.log_tool_result("Read", is_error=False, content_length=4200)

        # Record permission decision
        logger.log_permission_decision("Edit", decision="allow", mode="accept_edits")

        # Record provider call (LLM API usage)
        logger.log_provider_call(
            "anthropic", "claude-sonnet-4-20250514",
            input_tokens=1800, output_tokens=450, cost=0.0125
        )

        # Record session end
        logger.log_session_end(turns=2, total_tokens=4500, total_cost=0.031)

        print(f"Log path: {logger.log_path}")
        print(f"Events logged: {logger.event_count}")

    # Verify after the fact
    valid, errors = AuditLogger.verify_chain(logger.log_path)
    print(f"Chain valid: {valid}")

The `AuditConfig` dataclass mirrors the TOML config: `enabled`, `scan_pii`, `retention_days`, `retention_max_size_mb`, `log_tool_args`. Note: `enabled` defaults to `False` — audit logging is off unless explicitly enabled.

## 10Next Steps

You now have a complete audit logging setup. Here is what to explore next:

- Add the compliance check script to your CI pipeline — exit code 1 blocks deploys on tampered logs
- Configure append-only storage (S3 Object Lock, Wasabi WORM) for production audit logs
- Combine audit logging with Policy-as-Code (Tutorial 05) for a complete compliance posture
- Use `retention_max_size_mb = 0` to disable size limits for HIPAA 6-year retention