← Back to projects

AI Scenarios

Workers practise a hard conversation out loud with an AI persona, get scored against their manager’s criteria, and retry until they pass. The first step towards an AI tutor.

Role
Lead Product Designer
Team
1 designer, 2 engineers
Timeline
3 months, 2026
Problem
Frontline workers need to be ready for any situation, but testing how they’d handle one meant someone had to run it with them.
My part
Led product design and managed the project after our Product Manager left at the start: designed AI Scenarios end to end, set the scenarios-first roadmap and named it with product marketing.
Outcome
Shipped inside Paths in November 2026, as committed. An enterprise customer called it “a huge deal”.

01Why

Frontline workers had nowhere safe to rehearse high-stakes moments.

Frontline workers need to be ready for anything. A drive-thru worker gets a customer whose order is wrong for the second time. A bartender has to refuse service to someone who’s had too much. Most of this is taught with modules and PDFs, and the only way to test how someone really handles it was for another person to play it out with them. That rarely happened, so the first real practice was often in front of a real customer.

02What customers told us

Practice only gets used if it sits inside the training people already do.

We shaped the MVP with a small group of customers, and what we heard was consistent. Practice only gets used if it sits inside the training people already do. Workers would rather talk than type. Managers want a real score, with a pass mark around 70 to 80%, and the final word on it, with the AI doing the first pass. Workers want to know what to do better, not just whether they passed. And because the AI adapts to every reply, there’s no script to copy and pass around.

03Key decisions

Practice before a chatbot.

That shaped three decisions: what to build the practice from, where it should live, and how to earn the trust of the people using it.

  1. Sketch: the customer’s SOP with their logo turns into a scenario with the same logo and their criteria to be scored on.

    Built from the customer's content

    Scenarios come from the customer's own SOPs and standards. A retail customer called this the money-maker.

    ImpactPractice customers trust, and the content model the tutor needs.

  2. Sketch: a path of steps with the scenario as one of them, and dashed arrows to later entry points, on demand and the AI assistant.

    In the workflow, not locked to it

    Scenarios launched as a Paths block, next to the SOP they test, but stand alone so they can later open on demand or from the AI assistant.

    ImpactValue now, no rebuild later.

  3. Sketch: criteria shown up front, a manager editing a score, and a switch to turn AI off.

    Designed for trust

    Workers see what they’ll be scored on up front, managers have the final say on scores, and admins can switch AI off. I also named it AI Scenarios, in customers’ own words.

    ImpactReady for regulated and enterprise customers.

04What we built

The admin writes the situation and the marking guide; the AI plays the other person.

For the admin, setting up a scenario should feel like describing the situation to a colleague. They answer a few questions in one panel, select Generate scenario, and the preview beside it becomes the worker’s intro card, ready to try.

Close-up of the Scenario description field: Gardener cuts down expensive orchid by mistake.

05How it works for workers

Read what good looks like, talk it through, get scored, retry.

For the drive-thru worker, the scenario is simply the next step in their path. Before they start, they see the situation, the AI’s personality, how they can answer, and exactly what they’ll be scored on. The AI opens the conversation in character, and they reply by voice or by typing, switching freely in the same screen. The AI scores the attempt against the criteria. If it falls short, the step shows Not yet passed with a Try again button, and the worker can’t move on until they clear the pass mark, 75% by default.

A path called Staying safe and compliant in the kitchen, with a scenario as its first step: the scenario card shows the AI personality, response mode, a Start scenario button and five things the worker will be scored on.

06Where this is going

Scenarios first, tutor later.

Scenarios are the first stage of a longer plan towards an AI tutor. Each stage is useful on its own and builds something the tutor will need. On purpose, the MVP leaves out a standalone coach, engagement games and anything beyond training.

StageWhat the worker getsWhat it needs underneath
NowScenario practice: rehearse a real situation with AI, grounded in the site's SOP, inside a pathCustomer documents as context, criteria, scoring
NextMore context per scenario, and practice outside a path, on demandMultiple documents per scenario, a standalone entry point
ThenAsk questions about completed training and get answers from the company's own materialSemantic search across course content
VisionA tutor that guides each worker through what they need, at their own pace, at the moment of needSkills data from the matrix, so the tutor knows each worker's gaps

07Outcome

Shipped as committed, and customers asked for more.

After an alpha with 10 customers, the MVP shipped in November 2026 as committed, as a block inside Paths. Customer reaction was strong. A retail customer called scenario practice the one capability they'd use tomorrow, and a state energy regulator picked out scenarios as the way to assess the skills they define. The next requests are about depth: more context per scenario and on-demand access, both steps towards the tutor. The bet is that practice with scoring and feedback raises readiness for high-stakes moments. Measured results aren’t in yet; these are the targets we set.

This will change the game for us. Absolutely. This is a huge deal.
Enterprise food and agriculture customer, live demo, September 2026
TargetGoal in the first 30 days
Completion70% of started sessions reach the feedback screen
First-attempt pass rate50% or more
Retries80% of failed attempts are retried, not abandoned
Voice adoption60% of sessions use voice
Manager review90% of queued reviews actioned within 48 hours (over 60 days)

08What I learned

Slicing a big vision into small steps let us ship sooner, get value to customers early and keep improving on what they told us.