AI Scenarios
Workers practise a hard conversation out loud with an AI persona, get scored against their manager’s criteria, and retry until they pass. The first step towards an AI tutor.
- Problem
- Frontline workers need to be ready for any situation, but testing how they’d handle one meant someone had to run it with them.
- My part
- Led product design and managed the project after our Product Manager left at the start: designed AI Scenarios end to end, set the scenarios-first roadmap and named it with product marketing.
- Outcome
- Shipped inside Paths in November 2026, as committed. An enterprise customer called it “a huge deal”.
01Why
Frontline workers had nowhere safe to rehearse high-stakes moments.
Frontline workers need to be ready for anything. A drive-thru worker gets a customer whose order is wrong for the second time. A bartender has to refuse service to someone who’s had too much. Most of this is taught with modules and PDFs, and the only way to test how someone really handles it was for another person to play it out with them. That rarely happened, so the first real practice was often in front of a real customer.
02What customers told us
Practice only gets used if it sits inside the training people already do.
We shaped the MVP with a small group of customers, and what we heard was consistent. Practice only gets used if it sits inside the training people already do. Workers would rather talk than type. Managers want a real score, with a pass mark around 70 to 80%, and the final word on it, with the AI doing the first pass. Workers want to know what to do better, not just whether they passed. And because the AI adapts to every reply, there’s no script to copy and pass around.
03Key decisions
Practice before a chatbot.
That shaped three decisions: what to build the practice from, where it should live, and how to earn the trust of the people using it.
Built from the customer's content
Scenarios come from the customer's own SOPs and standards. A retail customer called this the money-maker.
ImpactPractice customers trust, and the content model the tutor needs.
In the workflow, not locked to it
Scenarios launched as a Paths block, next to the SOP they test, but stand alone so they can later open on demand or from the AI assistant.
ImpactValue now, no rebuild later.
Designed for trust
Workers see what they’ll be scored on up front, managers have the final say on scores, and admins can switch AI off. I also named it AI Scenarios, in customers’ own words.
ImpactReady for regulated and enterprise customers.
04What we built
The admin writes the situation and the marking guide; the AI plays the other person.
For the admin, setting up a scenario should feel like describing the situation to a colleague. They answer a few questions in one panel, select Generate scenario, and the preview beside it becomes the worker’s intro card, ready to try.
05How it works for workers
Read what good looks like, talk it through, get scored, retry.
For the drive-thru worker, the scenario is simply the next step in their path. Before they start, they see the situation, the AI’s personality, how they can answer, and exactly what they’ll be scored on. The AI opens the conversation in character, and they reply by voice or by typing, switching freely in the same screen. The AI scores the attempt against the criteria. If it falls short, the step shows Not yet passed with a Try again button, and the worker can’t move on until they clear the pass mark, 75% by default.
06Where this is going
Scenarios first, tutor later.
Scenarios are the first stage of a longer plan towards an AI tutor. Each stage is useful on its own and builds something the tutor will need. On purpose, the MVP leaves out a standalone coach, engagement games and anything beyond training.
| Stage | What the worker gets | What it needs underneath |
|---|---|---|
| Now | Scenario practice: rehearse a real situation with AI, grounded in the site's SOP, inside a path | Customer documents as context, criteria, scoring |
| Next | More context per scenario, and practice outside a path, on demand | Multiple documents per scenario, a standalone entry point |
| Then | Ask questions about completed training and get answers from the company's own material | Semantic search across course content |
| Vision | A tutor that guides each worker through what they need, at their own pace, at the moment of need | Skills data from the matrix, so the tutor knows each worker's gaps |
07Outcome
Shipped as committed, and customers asked for more.
After an alpha with 10 customers, the MVP shipped in November 2026 as committed, as a block inside Paths. Customer reaction was strong. A retail customer called scenario practice the one capability they'd use tomorrow, and a state energy regulator picked out scenarios as the way to assess the skills they define. The next requests are about depth: more context per scenario and on-demand access, both steps towards the tutor. The bet is that practice with scoring and feedback raises readiness for high-stakes moments. Measured results aren’t in yet; these are the targets we set.
This will change the game for us. Absolutely. This is a huge deal.
| Target | Goal in the first 30 days |
|---|---|
| Completion | 70% of started sessions reach the feedback screen |
| First-attempt pass rate | 50% or more |
| Retries | 80% of failed attempts are retried, not abandoned |
| Voice adoption | 60% of sessions use voice |
| Manager review | 90% of queued reviews actioned within 48 hours (over 60 days) |
08What I learned
Slicing a big vision into small steps let us ship sooner, get value to customers early and keep improving on what they told us.





