# How to Interpret Pre- and Post-Training Results When Respondents Differ Compare pre- and post-training results without confusing unmatched groups, matched pairs, attrition, or causal effects. - Canonical URL: https://www.harmate.com/en/blog/interpret-pre-post-training-results-when-respondents-differ - Author: Harmate Team - Published: 2026-10-28 - Updated: 2026-10-28T17:03:40.176242+00:00 - Language: en ## Content # How to Interpret Pre- and Post-Training Results When Respondents Differ You collected a measure before training and a follow-up afterward, but the files do not contain exactly the same people. You can describe each group’s results if the questions and conditions are comparable. You cannot tell who improved individually unless you can link the responses from the same people. That distinction changes the result’s label, its denominators, and what you can conclude. It applies to a knowledge check, a task scored against shared criteria, or a self-report measure; these sources do not show the same thing. ## Define the measure before comparing results Write the learning objective as a task. For example: “prepare a response to a customer request and justify the next step.” Decide what evidence would help examine that objective: an exercise, a scenario, or a question about an experience. A satisfaction rating measures reaction; it cannot replace this evidence. Check that both collections concern the same skill and use stable wording, scale, or scoring rubric. If the question, options, or task difficulty changed, explain the difference and avoid presenting the values as directly comparable. A confidence question before and after is self-assessment; using a numeric scale does not turn it into a skill test. The [CDC distinguishes before-and-after assessment, retrospective assessment, and a post-only measure](https://www.cdc.gov/training-development/php/about/evaluate-training-measuring-effectiveness.html). A test administered before and after lets you observe a difference in results. A post-only assessment describes the level at that point but does not show the starting level. A retrospective question—“before, I would rate myself…; now, I would rate myself…”—collects a perception after the fact, not a baseline assessment made before training. ## When respondents are unmatched: compare groups Imagine one cohort of 10 eligible participants invited to respond before and after the course. All 10 respond before, but only 8 afterward. Without a stable, authorized identifier linking responses, you cannot pair each person’s responses across waves: you can compare aggregate results for each wave, but not individual change. Compare their distributions or their rate of meeting a defined task criterion, and name each denominator. **Fictional example.** Before training on customer interviews, all 10 responses from the cohort are complete and scorable on the same exercise, using three stated criteria: identify the customer’s goal, check the context, and propose a next step. Four of 10 meet all three criteria (40%). After training, 8 people respond; all 8 responses are complete and scorable using the same rubric, and 6 meet all three criteria (75%). The difference between the observed proportions is 35 percentage points. That difference describes the two respondent groups. It does not show that four people acquired the criteria, because the files do not identify who answered at both times. The two missing post-course responses may matter too: people who did not respond may differ from those who did. Do not classify them as failures, successes, or people who made no progress. Show side by side how many people were eligible or invited, how many responses arrived, how many were usable for each question, and the result. Describe differences in experience, role, recruitment, or testing conditions when relevant. A percentage alone hides group size and nonresponse. ## If you have matched pairs, keep them separate Individual comparison requires a key that reliably links both responses from the same person. Use a pseudonymous identifier limited to this purpose; do not reconstruct identity from names, job titles, or similar answers. Check duplicates, missing keys, and ambiguous matches before calculating anything. **Fictional matched example, from a separate cohort of 10 eligible participants.** Eight people have valid responses at both times using the same task and rubric; the other two have no usable pair. Among these eight pairs, four move from “criteria not met” to “criteria met,” three remain in the same category, and one moves from met to not met. You can describe these transitions for the eight matched people. You cannot extend that result to the two people without pairs or to people outside this cohort. Record how many people were eligible, how many answered at both times, and how many were excluded from the matched analysis, including the reason when known. Matched respondents may form a different subgroup. A paired analysis describes that subgroup; it does not remove possible bias from nonresponse. ## Check explanations before attributing the difference to training Even with the same people and measure, an observed increase does not prove that training caused it. Participants may practise between tests, receive other support, encounter a different task, or respond under better conditions. With different groups, participant turnover and nonresponse add further possibilities. Ask what changed between collections: instructions, scoring, time allowed, delivery, group composition, prior experience, or opportunity to practise. Keep the limitation beside the result: “The share of respondents meeting all three criteria was higher after the session; respondents are not all individually linked, and we cannot attribute the difference to the course alone.” If results include open answers, use them to understand situations and exceptions—for example, why someone completed the task or what prevented them from finishing. Do not count a few comments as an estimate of how common a barrier is in the whole group. For post-session experience and opportunities to apply learning at work, see our [post-training evaluation guide](/en/blog/post-training-evaluation-questions-and-method-for-assessing-workplace-application). It covers workplace transfer, a different question from change in a before-and-after measure. ## A short record for reporting results | What to specify | Example wording | |---|---| | Measure | Exercise scored against three stated criteria | | Cohort | 10 eligible people invited at both times | | Before | 10 responses received and scorable; 4/10 meet all three criteria (40%) | | After | 8 responses received and scorable; 6/8 meet all three criteria (75%) | | Matching | Unmatched groups; no individual change measured | | Nonresponse | 2 cohort members did not respond after training; their identities and results are unknown | | Interpretation | Observed results were higher after the session; other explanations remain possible | | Next step | Examine missing responses and repeat the same task if a decision warrants it | If the same people are linked, replace “unmatched groups” with the number of pairs analyzed and report the observed transitions. Do not combine matched and unmatched results into one percentage without explaining the rule. Before sharing the report, check: are the objective and task explicit? Were instructions and criteria stable? Are denominators and absences visible? Are the groups truly the same people? Does the text separate observation from causal explanation? Our [training evaluation report example](/en/blog/training-evaluation-report-a-complete-example-for-a-client) keeps sources, limits, and proposed decisions in distinct sections. To prepare responses you have already collected, you can manually import a CSV, JSON, or XLSX file into [Harmate](/en/product/response-import) and review the columns. An account is required. Import does not match people or calculate a paired pre/post analysis; you must preserve your identification method and interpret results in light of the collection design.