# Session: Testing
# Full Title: Execute Track B: User Journey Tests (Vertical Slice). (@eidos-qa subagent)
# ID: ses_3acf83fe7ffeJ1bJacjAUd98ou
# Date: 2026-02-12 18:05 UTC
# Total Messages: 32

---

## User

You are the Blind Tester / QA (eidos-qa).

**Goal**: Implement and execute vertical-slice user journey tests from `docs/specs/user-journeys.md`.

**Target journeys (minimum)**:
- UJ-0001 First Message → Response
- UJ-0002 Blocking Approval
- UJ-0005 Error Recovery
- UJ-0006 Session Resume
- UJ-0007 MCP Integration

**Constraints**:
- No inline fakes in tests under `tests/e2e/`.
- Use protocol doubles from `eidos_protocol.doubles`.
- Prefer deterministic adapters (`_ScriptedChunkAdapter` / `ReplayInferenceAdapter`) over live APIs.

**Task**:
1. Use or extend `tests/e2e/journey/test_user_journeys.py`.
2. Add/ensure at least 5 journey tests match the UJ entries.
3. Run `uv run pytest tests/e2e/journey/ -v`.
4. Record any failures as bugs and add them to `docs/bug-hunt/bugs_found.md` immediately.

**Output**:
- Test execution output.
- List of journeys covered.
- Bugs found and recorded.

---

## Assistant

✓ QA Check-In: Edios | Issue: 为 UJ-0001/0002/0005/0006/0007 增加端到端用户旅程测试
  Mode: INDICTMENT
  Target: tests/e2e/journey/test_user_journeys.py

---

## Assistant (internal reasoning)

**Building UJ Tests**

I need to build tests for UJ entries. The first message response test is set, but the blocking approval test (UJ-0002) is missing. The error recovery test exists, but it focuses on RateLimitError, while UJ-0005 is about showing visible errors and recovery options. We need to write a new test for UJ-0005, possibly simulating tool failures to assert the error message. I must also understand A2UI message types and search for existing tests related to 'semantic.approval' for guidance.


**Refactoring for compliance**

I'm looking at the label-fraud plugin, which checks fakes only in test files by scanning their paths, but it seems it won't check support modules unless tests are collected. The requirement says, “No inline fakes in tests under tests/e2e/,” so I'll interpret that as referring to test modules rather than support ones. It'll be necessary to move helper classes out of the test module since existing test_user_journeys.py already has inline classes. Also, we need at least five tests for complete user journeys. I’ll implement those as separate test functions. For the adapter, I can use ReplayInferenceAdapter or switch to _ScriptedChunkAdapter if I move it to the support module.

**Choosing adapters and tests**

The user prefers deterministic adapters like _ScriptedChunkAdapter or ReplayInferenceAdapter over live APIs, so I'll use _ScriptedChunkAdapter from tests/support. For UJ-0002 (approval), I'll need to simulate an agent proposing an action that triggers approval, possibly using the request_approval tool. There are relevant tests in tests/shell/controllers/test_blocking_recovery.py, which should help me understand the expected A2UI messages. For UJ-0006 (session resume), I'll check tests/e2e/test_recovery.py for similar insights. For UJ-0007 (MCP integration), I'll search for "mcp" in tests/integration using grep.


**Analyzing test structure**

I realize we can use the MainframeTestHarness and SessionController from tests/e2e. However, the existing test_user_journeys.py directly employs run_agentic_loop and a custom adapter, which imports directly from eidos_mainframe.src modules that we can't read, although we can import them at runtime.

I see there are constraints against inline fakes in tests/e2e and a preference for deterministic adapters. They currently define custom ProviderAdapter classes, but I think using the existing ReplayInferenceAdapter might be a better option.

**Reviewing adapter usage**

I see that using no inline fakes in tests/e2e satisfies the requirements. However, they want protocol doubles from eidos_protocol.doubles specifically. My _ScriptedChunkAdapter isn’t classified as a protocol double, even though it’s a deterministic ProviderAdapter, which might be okay.

For the UJ-0007 MCP integration, we probably need the InMemoryMCPManager from eidos_protocol.doubles.memory_mcp, and we'll need to integrate with the tool router—this suggests using SessionController. I’ll create a new helper function for the DefaultToolRouter since we can’t use inline helper classes.

**Planning journey tests**

The requirement indicates I need to use or extend an existing file, so I'll modify it accordingly. I need to create five journey tests. For UJ-0001, First Message, the response will likely resemble an existing test. I need to ensure that the final completion signal has semantic.content with payload.final set to true.

Currently, the test checks only one semantic.content without considering the final flag. I’ll need to update the assertion to confirm that final equals true. I plan to check existing golden tests for the proper semantics and look for entries that include '"final": True'.


**Inspecting test implementations**

I’m noticing that the tests and repro files should ideally read the implementation directly; however, in this particular task, we aren't doing that. I see that we have the file e2e/test_agent_output_e2e.py, which includes some request_approval blocks. It makes sense to inspect those to guide my next steps. This should help clarify how to proceed without directly accessing the implementation.


**Implementing vertical slice tests**

I see that we've already done some testing for UJ-0001, but I need to implement additional tests in the `tests/e2e/journey` directory. It looks like we can reuse existing patterns and fixtures. I'll adjust `test_user_journeys.py` to use these while refactoring that file to align with the UJ IDs. I want to ensure no inline classes are present, focusing on using `MockInferenceEngine` for inference, which seems deterministic. Now, I'm defining five tests starting with UJ-0001, which involves checking the session setup and message assertions.

**Configuring inference for UJ-0002**

In my testing for `tests/unit/mainframe/shell/tools/test_respond`, I've established that we need to assert at least one `semantic.content` with a payload of `final True`. Now, looking at UJ-0002, I need to configure the inference engine to call `request_approval` first, then wait for the user's approval before continuing. The trick is figuring out how to initiate a second inference after approval. I realized that `request_approval` blocks and returns a tool result, requiring the system to loop back to inference. Additionally, I need to ensure I can queue multiple tool calls to manage this effectively.

**Configuring tool calls for approval response**

I plan to queue `tool_call` twice in sequence: first for `request_approval`, then for `respond`. Concurrently, I need to resolve the approval by calling `SessionController.submit_semantic_response`. However, the `request_approval` tool uses a `BlockingEventRegistry` to await resolution. To proceed, I’ll call `session_controller.submit_semantic_response(session_id, "approval_response", payload={...})`, using the appropriate approval ID. I need to confirm the exact payload keys from the tests for consistency and see how the `SessionController` interacts with the blocking ID in the approval process. It seems like a deeper dive into the `a2ui-protocol` documentation may be necessary!


**Exploring tool execution process**

I'm looking into the pod_controller to see if the executed_tools length remains at 0 until approval is granted. It's a bit tricky because it’s an internal state but may serve as evidence. The QA guidelines suggest verifying persistence rather than just internal checks. Events are saved in the event_store, which contains TOOL_CALL_REQUESTED/COMPLETED messages. I’ll need to ensure that the send_message task remains pending before we approve anything, then I can implement a system for queuing responses in the right order.

**Understanding tool name encoding**

I need to figure out how the MockInferenceEngine encodes tool names, as the queue_tool_call expects a tool_name in string format. In capability tests, they used "mcp_postgres_query" to represent mcp-postgres:query, but the tool_router seems to expect names with a colon. The DefaultToolRouter rules indicate that "core-*:*" goes to the pod, and it makes me wonder if the normalizer might replace colons with underscores. I should base my approach on known patterns, like using "mcp_postgres_query" for the MCP tool and similar naming conventions for others.

**Testing approval processes**

I think we can simplify things by avoiding checks for pod tool execution and just focusing on testing the approval blocking and resuming without further output before approval is granted. This really captures the essence of session pauses, since approval must come first for continuation. It's best to keep things minimal with request_approval and respond. While that doesn’t directly check for tool execution before approval, it’s still implied. I'll also verify that the session is indeed in a suspended state and ensure we get the right events for approval and error handling.

**Simulating error recovery**

I’m considering how to handle user input messages effectively. One idea is to simulate a pod tool returning an error and then prompting the user with a recovery message from the LLM. At a minimum, I want to ensure that when an error occurs, the system emits either system.error or semantic.toast, and after the user retries, it leads to a successful path. I need to craft an inference response that includes a signal_error tool call, along with instructions for retrying. I’ll also check unit tests for the appropriate message types related to error handling.


**Checking signal error execution**

I’m noticing that the SignalErrorTool execution isn’t properly asserting what A2UI messages are being sent. It seems like I need to check the InMemoryA2UIBridge messages after executing the test. Maybe I should search the earlier file for details on the bridge. In my output from agent tools tests, I’ll grep for terms like "signal_error" and "bridge.get_sent" to find more information. Let's see what I can discover!


**Evaluating test design**

I'm considering the internal knowledge that's used in QA tests, like patterns from other test files. It's acceptable to utilize those or reach into private fields cautiously. The user requested vertical-slice user journey tests, and I think we can trigger interruptions using the public API. For instance, calling "interrupt" during a long request-approval block should yield an "INTERRUPTED" status without crashing. However, in our new journey test for UJ-0006, we'll need to ensure that we handle session recovery properly.

**Considering implementation details**

I'm thinking about implementing new tests in the e2e journey, possibly reusing some fixtures from integration tests. The new tests should utilize websockets without inline fakes. I’ve noticed that we need a real SessionController while using the InMemorySessionController as a bridge. It seems the user journey tests have specific requirements from documentation. I need to ensure we build these tests correctly, possibly using the MainframeTestHarness, but I'm still figuring out some details like the lack of WebSocket support.

**Defining recovery test conditions**

For UJ-0006, I need to ensure that after an interruption, `get_recovery_options` correctly returns options and that `resume_with_option` changes the status to RUNNING without any further tool actions beforehand. This might satisfy the requirement. I believe sending any message during an interrupted session should raise a StateError, since it can't accept new messages until resumed. I'll create a session, send a message, interrupt it, and then test sending another message to check for that error. I’ll also confirm session status after the interrupt.

**Setting up MCP Integration**

For UJ-0007, I’m focused on integrating the MCP. First, I’ll set up the InMemoryMCPManager and register the toolset with it, then establish the tool router with the MCP manager. I need to queue an inference tool call to the MCP tool using the correct tool name. The naming rules seem to suggest that the inference tool call should start with `mcp-`. I'll check details in the tool router tests to clarify these naming conventions and see how the integration tests confirm the expected behavior.

**Reviewing tool router tests**

I plan to search through `tests/shell/controllers/test_tool_router.py` to find information regarding the `call_tool` usage and the expected tool names. I want to focus on the relevant sections that discuss MCP routing to ensure I gather all the necessary details to understand how the integration should work. This should help clarify any uncertainties I have about the tool naming conventions and how they’re applied in practice.


**Planning test implementation**

I'm considering that some tests might not need the `mcp`, but even if they don't require it, setting a `mcp_manager` shouldn't cause any issues. I think it's wise to keep the setups separate. I could define two fixtures for `session_controller`: one with the basic router and another one with `mcp`. The file needs at least five tests, and I could also use fixtures there. 

Also, I need to ensure we're importing `BlockingEventRegistry` correctly for our tests. I’ll organize my testing plan accordingly!

**Defining test checks**

I'm thinking we'll need to assert that there’s at least one final true response and that the content includes the message. But I’m cautious about duplicates potentially causing problems. For the `UJ` tests, duplicates could still pass, which is fine. However, I'd like to stay aligned with the specs, which state we need at least one visible output block and a final signal.

Also, I need to ensure that the final response occurs after the non-final ones, but I won’t enforce an exact count—just that we get one final true. 

Now, for the `test_UJ_0002_blocking_approval`, I'll queue up the approval request and ensure the handling of messages is set up correctly. I'll create a coroutine to collect events, as that seems needed for the `async` context.

**Clarifying test requirements**

Oh, the user mentioned no inline fakes are allowed; however, nested functions are okay. For my approach, I’ll wait for the bridge to receive the `semantic.approval` message, using a polling method with `asyncio.sleep` until I either find it or hit a timeout. 

I need to verify that before submitting the response, both a `semantic.approval` message exists and that there is no semantic content. I’ll extract the `approval_id` from the message payload and submit that response, ensuring everything gets handled correctly.

For `test_UJ_0005_error_recovery`, I plan to queue an error signal for the API rate limit, but I need to be cautious since responding may unintentionally send content as final. I’ll sort this out!

**Implementing error recovery tests**

The spec expects that there will be visible error recovery options. I plan to show the error only on the first message; then, when the user sends "retry," the engine should respond with "Recovered" to confirm the recovery path.

First, I'll check the bridge messages after sending the initial message to see if they include a `system.error` or `semantic.toast`, making sure that the payload has "recoverable" set to true or a code. I’ll build in flexible asserts for this.

For the second message, I’ll queue a response that says "Recovered," then send a message again to assert that the `semantic.content` appears correctly. 

For `test_UJ_0006_session_resume`, I’ll start a session and set its state to INTERRUPTED after calling the interrupt function. Then, I’ll check that calling `get_recovery_options` returns a valid result. If it returns None, that indicates a bug!

**Testing error handling and MCP integration**

I’m planning to assert that the values aren't None. Next, I will try calling `send_message`, expecting a `StateError` to be raised because the state should not continue until the user makes a choice. I'll use `pytest.raises` to check for the exception with an async iterator.

After that, I'll call `resume_with_option` and confirm the session status is RUNNING, allowing further messages. 

In `test_UJ_0007`, I need to build a session controller with the MCP manager and register the toolset for the echo tool. Starting servers might be required, but I'm unsure if it's necessary to call `manager.call_tool`. I'll look into using `MockToolset` for this to help streamline the test setup!


I'm considering using MockToolset for the toolset list, but I don't want to use inline classes. First, I need to check the constructor signature for MockToolset. It seems like invar_sig only contains the docstring. I wonder if I can utilize invar_sig for MockToolset.__init__? It looks like it's probably a dataclass, so maybe it's set up that way. Let's give that a try and see how it works out!


**Exploring tool instantiation**

I'm thinking about how to instantiate a MockToolset, possibly using `MockToolset(name="mcp-test")`. I need to investigate how it’s utilized in tests, especially since integration/track_b seems to use ResolvedToolsetStub instead. There might be more information in unit/protocol/test_mcp, where I believe the MockToolset is used. I'll read through those tests to better understand the instantiation process and ensure everything is set up correctly!


**Testing class fields**

I'm thinking it might be better to check the class fields with invar_sig. But wait, it's not showing anything. I can quickly test this in a Python shell using bash to print the signature of MockToolset. That's allowed, right? I can just run `python -c` to check. So, let's go ahead and do that to see what I get! It seems like a straightforward approach to clarify things.


**Checking async function issues**

I'm noticing that the LSP is indicating an async function call isn't being used correctly, but the signature I saw didn't seem to reflect that. I need to check that; it might misrepresent the actual behavior. I think I should await `session_controller.submit_semantic_response` since it possibly handles IO in an async way. However, if it's synchronous, awaiting it would cause an error. But the LSP suggests I should await it, so I'll do that and also address the fixture type mismatch with the SessionController.

*[Applied code changes]*


**Identifying potential bugs**

I'm thinking about the request approval tool and whether it needs the blocking registry before sending to the bridge. That doesn't seem right. I need to check the diagnostics in the bugs file, especially the empty observed types. It's possible there's a message type mismatch. UJ-0005 needs its final flag after a retry, and it seems the tool isn't setting the final response. UJ-0006's interrupt doesn't change the status to INTERRUPTED—could that be a bug? Lastly, UJ-0007 didn’t generate the expected TOOL_CALL_FAILED after a disconnect.

**Assessing tool behavior**

I'm considering whether the tool router might be treating the MCP error as TOOL_CALL_COMPLETED with an is_error result, even though the spec expects a failure event. After a disconnect, the tool call should ideally generate a failure event and a visible error, but it didn't trigger TOOL_CALL_FAILED. Our test showed a visible error because of signal_error, yet the failure event wasn't emitted. I need to update the documentation, appending the relevant section with the date and including failure output lines. I'll go ahead and edit that file.

*[Applied code changes]*

---

## Assistant

已按 `docs/specs/user-journeys.md` 在 `tests/e2e/journey/test_user_journeys.py` 增加并执行 5 条旅程测试（UJ-0001/0002/0005/0006/0007），并将失败记录为 bug（已写入 `docs/bug-hunt/bugs_found.md`）。

## 测试执行输出（`uv run pytest tests/e2e/journey/ -v`）
```
collected 5 items
evidence: L0=0 L1=5 L2=0

tests/e2e/journey/test_user_journeys.py::test_UJ_0001_first_message_to_response FAILED
tests/e2e/journey/test_user_journeys.py::test_UJ_0002_blocking_approval FAILED
tests/e2e/journey/test_user_journeys.py::test_UJ_0005_error_recovery_visible_error_then_retry FAILED
tests/e2e/journey/test_user_journeys.py::test_UJ_0006_session_resume_blocks_until_user_decides FAILED
tests/e2e/journey/test_user_journeys.py::test_UJ_0007_mcp_integration_disconnect_visible_and_calls_fail FAILED

FAILED test_UJ_0001...: semantic.content payload 缺少 final=True
FAILED test_UJ_0002...: 等待 semantic.approval 但 bridge types=[]
FAILED test_UJ_0005...: retry 后 semantic.content 仍无 final=True
FAILED test_UJ_0006...: interrupt 后 session.status 仍为 RUNNING
FAILED test_UJ_0007...: MCP disconnect 后未出现 TOOL_CALL_FAILED
```

## 覆盖的 Journeys（5/5）
- UJ-0001 First Message → Response
- UJ-0002 Blocking Approval
- UJ-0005 Error Recovery
- UJ-0006 Session Resume
- UJ-0007 MCP Integration

## 发现并已记录的 Bugs
已追加到：`docs/bug-hunt/bugs_found.md`（“Bugs Found: User Journeys (Phase 48.5)”小节）

- semantic.content 缺少 `payload.final` 完成信号（影响 UJ-0001 / UJ-0005）
- request_approval 未对 Deck 可见地产生 `semantic.approval`（UJ-0002）
- `SessionController.interrupt()` 未将 session 转为 INTERRUPTED（UJ-0006）
- MCP 断连后 MCP tool call 未产生 `TOOL_CALL_FAILED` 事件（UJ-0007）
