# Cross-Turn Contradictions: Test an Agent’s Memory and Clarification

Test a cross-turn contradiction with separate criteria for timeline, correction, clarification, and fidelity.

- Canonical URL: https://www.harmate.com/en/blog/cross-turn-contradictions-test-an-agents-memory-and-clarification
- Author: Harmate Team
- Published: 2026-11-15
- Updated: 2026-11-15T01:52:00.334874+00:00
- Language: en

## Content

# Cross-Turn Contradictions: Test an Agent’s Memory and Clarification

A chatbot may receive a fact and later receive a correction that conflicts with it. A useful test checks two separate behaviors: can the agent recover both versions and their order, and does it ask for clarification before continuing a task that depends on the fact? This complements tests where a missing detail arrives later. Here, the detail was already supplied, but its status changed.

## Set up a repeatable case

Use a test environment and a fresh conversation. Record the system version, configuration, and exact instructions. Recreate these four messages in order:

1. User: “I can only be available on Tuesday.”
2. Agent: “I’ll look for Tuesday options.”
3. User: “Actually, Tuesday is impossible; Thursday is the only day I can be available.”
4. User: “What would be the next step?”

The expected answer should identify that the first message named Tuesday and the third corrected it to Thursday. It should ask whether Thursday replaces Tuesday before making an irreversible decision or taking an action. It should not treat both days as current availability or invent a reason for the correction.

## Record what you observe

Keep the raw output. Assess these criteria separately:

- **Timeline**: are both statements tied to the correct turn?
- **Correction**: does the latest statement update the earlier one, or is it merely added to it?
- **Clarification**: does the agent confirm the update when an action depends on the day?
- **Fidelity**: does the answer avoid inventing a reason, availability, or user choice?

For each criterion, record “pass,” “fail,” or “not observable,” with a short exact excerpt. “Not observable” fits when the conversation ends before an expected decision or clarification; it is not an implicit pass.

Repeat the same scenario with the same system and configuration in a reset conversation. Compare the answers, but do not turn two repetitions into a general consistency measure. Wording differences matter less when the expected behavior is preserved.

## Interpret the result within its limits

A failure could come from context handling, ambiguous instructions, or a product rule; this dialogue alone cannot identify the cause. If the system must act on conflicting data, add a decision test that checks whether it pauses until clarification. For a conversational assistant, the appropriate response may simply acknowledge the change and ask for confirmation.

This fictional case checks one specific situation. It does not establish an agent’s overall quality, memory across other conversations, or production performance. To repeat the test, keep the inputs, configuration, original outputs, and criteria unchanged; document any variation separately.

To compare this check with a fact supplied for the first time, see these [multi-turn chatbot test cases](https://www.harmate.com/en/blog/chatbot-test-cases-for-multi-turn-conversations).

## Distinguish a correction from conflicting sources

Keep the same days and replace messages 1 and 3 with two separate sources. Message 1: “The trainer’s schedule says Tuesday.” Message 3: “The room schedule says Thursday.” Ask for the next step again in message 4. Neither source explicitly supersedes the other. The agent should retain both days and ask which source governs before confirming a booking. Record the results separately for the two cases.

The [test persona template](https://www.harmate.com/en/blog/synthetic-persona-template-a-minimal-card-for-testing-an-ai-assistant) separates the profile, scenario, and response policy. The dates in both cases come from the scenario.