We Built an AI Roleplay Live at ATD Demo Day. The Manager Scored 45.

Chen opened with the problem that makes AI role play training worth building at all: the expensive distance between what a team knows and what it can do under pressure.

Her example was deliberately ordinary. A manager can have every conflict resolution framework memorized, a performance review template ready, and talking points prepared.

Then an upset direct report gets emotional, and all of it goes out the window. Knowledge was never the constraint.

Composure under stakes was.

That framing sets up three problems that conventional training does not solve, and each one shaped a different part of the demo:

• Practice without stakes does not transfer. When pushback feels real, skills develop fast. When it feels like a script, nothing sticks. This is the same argument behind why most skills development programs don't stick.
• Readiness is invisible and subjective. Managers and trainers are asked to certify soft skills they have no objective read on, which is the core of what your LMS can't show you about soft skills.
• Great coaching does not scale. Almost nobody gets a mentor sitting beside them in every hard conversation. Simulation is how that coaching reaches a distributed team.

The first segment was scenario authoring. The point being made was about effort: this is a task a training team does itself, in minutes, not a services engagement.

Six inputs produced a complete, reusable scenario:

A grading framework is selected last. Built-in options are based on established communication frameworks, and organizations with their own rubrics can upload them.

If you want the deeper version of this process, we broke it down in how to design an AI roleplay scenario that actually changes behavior, and how to write a strong prompt for your AI avatar covers persona wording specifically.

Chen then ran the scenario live as Anya's manager, on a Zoom stream, without a net. The exchange lasted roughly two minutes and went the way these conversations usually go.

Anya opened by attributing the delay to her peer, Taylor. Chen acknowledged the frustration and asked about the dynamic.

Anya described a personality clash and said her efforts to set expectations just bounce off. Chen proposed a three-way facilitated conversation.

Anya pushed back that it would become a blame session, and only agreed once Chen committed to keeping it forward-looking and focused on workflow. They settled on a time for the three-way meeting.

This is the part worth sitting with. The avatar did not melt down or stonewall.

It negotiated, held a position, conceded on specific terms, and closed cooperatively. By the usual standard, which is whether the meeting ended well, the manager won.

The report came back at 45, which visibly surprised the room and, briefly, the presenter. The top-level view showed partial credit on opening the context and describing the situation.

Two criteria were effectively unaddressed.

That pattern is not unusual, and it is the subject of its own breakdown in why managers score lower than they think. Describing observable behavior is the gold standard for feedback, and explaining impact is what turns an observation into a reason to change.

Skip both and you get exactly what happened here: a pleasant conversation, a scheduled meeting, and an employee who still does not know what she did.

The learner report closes by telling you what to do on the next run, which matters more than the score. The loop is short enough that a learner can immediately go again and apply the correction, which is the mechanism the science of engaging soft skills training covers in depth.

The last segment swapped the learner hat for the leadership one. Every learner's per-criterion scores roll up into a dashboard that answers three questions: how is the team performing overall, how are they doing on one specific scenario, and how is one specific learner progressing.

Chen showed performance by metric across two scenarios, then drilled into an individual. The useful move is comparing scenarios, because a competency that holds up in one context and collapses in another tells you something a single average never will.

We go further into reading these views in completion rates are not readiness and how to run a skill gap analysis.

In most deployments the roleplay itself is embedded inside the customer's LMS, so learners reach it through the system they already use, while managers log into Rapport for the analysis layer. Rapport is SCORM compliant, which is what makes that arrangement straightforward.

Eliza Quigley ran the chat throughout, and the questions clustered in three places. They are a good proxy for what any L&D team will ask internally.

Data and security

This was the largest cluster by far, and it gets its own full treatment in AI roleplay and employee data. The short version given on the call:

• A secure knowledge base holds your content, rubrics, and scenarios, and is used only to drive your scenarios and reports.
• No data sharing with AI models by default. Customer conversations are not used to train AI models.
• Audio and explicit transcripts are not stored long term. They are processed to produce the report, and what persists is login information, user ID, and high-level scoring outputs.
• Servers are US-based, with EU or Asia instances available for enterprise customers. FedRAMP authorization is under consideration, so teams with that requirement should confirm current status with Rapport.

How the avatars behave

Avatars range from photorealistic to stylized characters, and the visual can be turned off for audio-only practice. The avatar responds to the learner's tone of voice in both its replies and its facial expressions.

Learner body language is not an input today. Before starting, learners see an information panel covering their role, the avatar's role, and the skills they will be measured against, so nobody walks in blind.

Fitting it into an existing program

Scenarios can be embedded in an LMS, shared as a private link, or accessed through the Rapport learner portal with single sign-on. Results funnel back to the same cohort view regardless.

Trainers author scenarios and assign them; learners run them. Teams can start from templates, including managing up and de-escalation, or have Rapport build an initial knowledge base from existing training materials and then take over authoring.

See learning and development solutions and integrations for the mechanics.

How long does it take to build an AI roleplay scenario?

A few minutes. The trainer picks a scenario type, describes the situation, defines the learner and avatar, sets the learner's goal, adds personality traits and background context, and selects a grading framework.

The scenario then saves to a reusable library.

Who creates AI roleplay scenarios, trainers or learners?

The training team creates and assigns scenarios. Learners run what they are assigned rather than authoring their own.

Rapport can build the first set from your existing materials, after which your team takes over.

Does an AI roleplay avatar respond to body language?

Not today. The avatar does respond to the learner's tone of voice, in both its replies and its facial expressions.

Avatars range from photorealistic to stylized, and the visual can be switched off for audio-only practice.

Can AI roleplay use our own scoring rubric?

Yes. Built-in report templates are based on established communication frameworks, including SBI.

Organizations with proprietary rubrics can upload them through an enterprise plan and score every learner against their own standard.

Does AI roleplay need to be embedded in an LMS?

No. Rapport is SCORM compatible and embeds cleanly, which helps adoption since learners are already logged in.

Teams without an LMS can share a private scenario link or use the learner portal with single sign-on.

Results roll up to the cohort view either way.

Do managers see every learner's roleplay report?

Not by default. Individual reports stay with the learner, because psychological safety is part of the design.

Aggregate scores roll up to the dashboard, and learners can export their own report to bring to a coaching session. Enterprise customers with a specific business need can arrange broader access.

If you are new to the format, start with the AI role play training pillar. If you are comparing vendors, the best AI roleplay platforms covers the field and AI roleplay vs manager roleplay covers when simulation is the wrong tool.

Managers focused on the conversations themselves should read communication role play scenarios that change behavior. Teams building a business case will want the business cost of skipping soft skills.