TL;DR: Using an AI model to rehearse an interview, a negotiation or a conversation you are dreading works, but the default version of it is close to useless, and there is now a measured reason why. Kamoi and colleagues introduced CoCoEval in March 2026, an evaluation framework that detects ten types of inconsistent and uncollaborative behaviour turn by turn, and used it to compare conversations simulated by GPT-4.1, GPT-5.1 and Claude Opus 4 against real human conversations from academic, business and governmental meetings and from debates. Three findings matter for anyone practising. First, under vanilla prompting the simulated conversations exhibited far fewer inconsistent and uncollaborative behaviours than the human ones. Second, prompt engineering did not give reliable control: different prompts produced under-production or over-production of those behaviours. Third, supervised fine-tuning on human conversation pushed models to overproduce a narrow set of behaviours such as repetition. In plain terms: your AI sparring partner is systematically easier than the person you are actually going to face, you cannot fix that with one clever instruction, and the fix has to be structural. Below: the five-phase protocol, a difficulty specification table with the tell for each scenario type, paste-ready prompts that keep the opponent and the coach separate, and the limits of what rehearsal buys.
Almost everyone who uses these models has done some version of this. A hard meeting is coming. You open a chat window, describe the situation, ask the model to play the other person, and run through it a few times.
It feels productive. You come away calmer. Then the real conversation happens and the other person does something your rehearsal never did: they misunderstood your second point, they interrupted before you got to the reason, they simply did not accept a premise you thought was settled.
That gap is not bad luck, and it is not a limitation of your prompt. It has been measured.
The finding that should change how you rehearse
Kamoi, Godbole, Yang, Zhang, Wan and Zhou published “Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction” in March 2026. Their framework, CoCoEval, detects ten types of inconsistent and uncollaborative behaviour at the level of individual turns, using a model as judge, and they applied it to conversations generated by GPT-4.1, GPT-5.1 and Claude Opus 4, comparing the frequency of those behaviours against real human conversations drawn from academic, business and governmental meetings as well as debates.
Their stated conclusions are three, and each one has a direct consequence for how you should practise.
| Finding | What the authors report | What it means for rehearsal |
|---|---|---|
| Underproduction under default prompting | “LLM-simulated conversations exhibit far fewer inconsistent and uncollaborative behaviors than human conversations” | Your default sparring partner is easier than reality. Practice against it builds confidence that the real conversation will not honour |
| Prompt engineering is unreliable | “prompt engineering does not provide reliable control over these behaviors, as our results show that different prompts lead to their under- or overproduction” | You cannot solve this with one well-written system prompt. A single prompt may undershoot or overshoot, and you will not know which |
| Fine-tuning distorts | “supervised fine-tuning on human conversations can lead LLMs to overproduce a narrow set of behaviors, such as repetition” | Even the heavier fix does not produce balanced difficulty. It trades one distortion for another |
This table reports the findings of the cited study; the rehearsal implications in the third column are ours.
Note carefully what the second finding rules out. The standard advice for this problem is “just tell it to be tougher.” The study’s own result is that instruction-level control is unreliable in both directions. So the method cannot rest on getting the instruction right. It has to rest on something you can verify from the outside, which means running the scenario more than once and checking whether the difficulty actually appeared.
That single structural change is most of the value in this piece.
Why practise with a model at all
It is worth being clear about what problem this solves, because the honest case for AI rehearsal is narrower and better than the usual one.
Yang, Ziems, Held, Shaikh, Bernstein and Mitchell of Stanford made the argument in “Social Skill Training with Large Language Models” in April 2024. Their opening observation is the whole point: “People rely on social skills like conflict resolution to communicate effectively and to thrive in both work and personal life. However, practice environments for social skills are typically out of reach for most people.”
That is the real constraint. Not that models are good counterparts. That almost nobody has access to a willing, patient, repeatable counterpart who will run the same difficult conversation with you eleven times at eleven in the evening.
Their proposed structure is the second thing worth taking from that paper. They describe an AI Partner, AI Mentor framework that “merges experiential learning with realistic practice and tailored feedback.” The partner plays the counterpart. The mentor gives the feedback. They are separate roles.
Most people collapse those two roles into one chat, and it is the single most common reason the practice fails. A model that is simultaneously your opponent and your coach cannot be a convincing opponent, because a good opponent does not stop mid-sentence to tell you that your framing was strong. Keeping them separate is not stylistic. It is what makes the opponent hold character.
What deliberate practice actually requires
The reason rehearsal works at all, when it works, is well documented, and so is the reason it usually does not.
Ericsson, Krampe and Tesch-Römer set out the framework in “The Role of Deliberate Practice in the Acquisition of Expert Performance” in Psychological Review in 1993. Their account of expert performance is that it results from prolonged, effortful activities specifically designed to improve performance, not from accumulated experience. The operative word is designed. Doing the thing repeatedly is not practice. Doing the specific part you are bad at, with feedback, is.
They also found something useful about duration: reviewing studies across domains, they reported essentially no benefit from practice durations beyond four hours a day, reduced benefits beyond two, and suggested the effective duration of deliberate practice may be closer to one hour a day. For rehearsal purposes that argues strongly for short, dense sessions rather than long ones.
There is a second reason short and dense wins, from a different literature. Roediger and Karpicke, in “Test-enhanced learning: taking memory tests improves long-term retention” in Psychological Science in 2006, had students study prose passages and then either restudy or take recall tests, with a final test after five minutes, two days or one week. At five minutes, restudying won. At the delayed intervals, prior testing produced substantially greater retention than studying.
The parallel to rehearsal is exact. Reading your notes about what you will say is restudying. Saying it out loud, unprompted, against resistance, is retrieval. The first feels better immediately. The second is what survives to the day of the meeting.
The same generation-before-reception principle drives the Socratic prompting method for learning material, and this piece is its behavioural counterpart.
The sparring protocol
Five phases. The phases exist to solve specific documented problems, so the third column names which.
| Phase | What you do | What the model does | Problem it solves |
|---|---|---|---|
| 1. Brief | Specify the counterpart’s identity, interest, constraint and mood, plus the exact difficult behaviours required. Give the model information you do not have | Reads and holds the role, including the private information | Vague roles produce agreeable roles. A counterpart with a stated interest has something to defend |
| 2. Spar | Speak or type your actual words in real time. No stage directions, no meta-commentary, no restarts | Stays in character. Never coaches, never breaks frame, never summarises | Retrieval under resistance, not restudy. Breaking frame turns practice back into reading |
| 3. Freeze | End the scene at the hardest moment, not at a resolution | Stops | The resolution is the least useful part. The moment you did not know what to say is the material |
| 4. Debrief | Switch to a separate conversation with the mentor role. Paste the transcript | Answers three specific questions, not general praise | Separating partner from mentor keeps the opponent credible and the feedback usable |
| 5. Re-run | Run the same scenario again with one variable changed and a different difficulty specification | Plays a materially different counterpart | The documented unreliability of prompt control. One run tells you nothing about difficulty. Three runs tell you the range |
This table is a CEOtudent editorial framework. It combines the partner and mentor separation described by Yang and colleagues with the deliberate-practice and retrieval-practice principles cited above, and adds the multiple-run requirement in direct response to the CoCoEval finding on unreliable prompt control.
Phase five is the one people skip and the one that matters most. If prompt-level difficulty control is unreliable in both directions, a single run is a single unverified sample. Running three and taking the hardest one as your reference is a crude fix, but it is a fix that does not depend on the instruction having worked.
Specifying difficulty, and how to tell when it failed
Since you cannot trust an instruction to produce difficulty, you have to name the specific behaviours you want and then check for them afterwards. Generic requests like “be a tough interviewer” are exactly the kind of instruction the study found unreliable.
Here is the specification by scenario, with the observable tell that your counterpart has gone soft.
| Scenario | Difficult behaviours to require explicitly | The tell that it went soft |
|---|---|---|
| Job interview | Interrupting a long answer at the 30-second mark; asking the same question again because the first answer did not land; visible loss of interest; declining to acknowledge a strength you claim | Every answer is met with “That’s a great example.” Follow-ups build on your answer instead of challenging its premise |
| Salary negotiation | Stating a firm constraint and not moving from it; going quiet after your number; reframing your value downward; ending the exchange without agreement | The counterpart concedes within three turns, or explains its own reasoning to you unprompted |
| Client pushback | Misremembering what was agreed; raising an objection you already answered; bringing in an absent decision-maker’s opinion; questioning the scope you thought was settled | It accepts your version of the history. Real clients rarely do |
| Difficult feedback to a report | Becoming defensive; disputing the specific example you cite; making it about someone else; going silent | It thanks you for the feedback in the first two turns and agrees to change |
| Saying no to a manager | Not accepting the reason; restating the request as though you had not declined; applying time pressure; implying consequence | It says “that’s fair, I’ll find someone else” |
| Investor or buyer question round | Asking the second and third question in a chain rather than moving on; challenging an assumption in your numbers; expressing scepticism without explaining it | It asks one question per topic and moves through your deck in order |
This table is a CEOtudent editorial framework. The behaviour categories are informed by the classes of inconsistent and uncollaborative behaviour that CoCoEval was built to detect; the scenario mapping and the failure tells are ours.
Use the third column as a live check during the session. If you see the tell, the run does not count. Restart with the specific behaviour named again and more sharply, and treat the soft run as evidence that this model, today, needs a heavier specification.
The two prompts
Two separate conversations. Do not merge them.
The partner prompt. Open a new conversation and paste this, filled in.
You are playing a single role in a rehearsal. You are [role, seniority, relationship to me]. Your interest in this conversation is [what they want]. Your constraint is [what limits them]. Your mood entering the room is [mood]. You know something I do not: [private information the counterpart holds].
Required behaviours during this scene, which you must actually perform and not merely describe: [pick three from the difficulty table]. Perform each at least once.
Rules: Stay in character for the entire scene. Never coach me. Never comment on my performance. Never summarise. Never break frame for any reason, including if I ask you to. Speak only as the character, one turn at a time, in the length that character would actually speak. Do not resolve the conversation. If I say the word STOP, end the scene immediately and say nothing further.
Begin. You speak first.
The private information line is doing more work than it looks. A counterpart who knows something you do not has a reason to behave in ways you cannot predict, which is the closest thing to genuine difficulty you can install by instruction.
The mentor prompt. New conversation, paste the transcript, then this.
Below is a transcript of a rehearsal in which I was [me] and the other party was [role]. Analyse my turns only.
Answer exactly three questions and nothing else.
1. What was I trying to achieve, judging only from what I actually said and not from what I probably intended?
2. Where specifically did I lose ground, quoting the exact line, and what was the mechanism of the loss?
3. What is the single change to my next attempt that would most alter the outcome, stated as a sentence I could actually say?Do not praise anything. Do not summarise the transcript. Do not offer general communication advice. If my turns were adequate throughout, say so in one sentence and stop.
Those three questions are not arbitrary. Hattie and Timperley, in “The Power of Feedback” in the Review of Educational Research in 2007, argued that effective feedback must answer three questions: where am I going, how am I going, and where to next. Their broader conclusion is worth keeping in mind while reading any model’s assessment of you: feedback is among the most powerful influences on learning and achievement, but the impact can be positive or negative, and the type of feedback and the way it is given are differentially effective. Undirected praise is the failure mode, and it is precisely what a model produces by default.
The honest limits
This is a rehearsal method. It is not a performance guarantee, and the gap between those two is larger than most advice on this topic admits.
| Claim | Status | Basis |
|---|---|---|
| A model can simulate a conversational counterpart | Supported | Documented in the LLM roleplay literature and the basis of the CoCoEval comparison |
| The default simulation is easier than a real counterpart | Supported | The direct finding of CoCoEval across three frontier models against human meeting and debate data |
| Telling the model to be difficult fixes this | Unsupported | The same study found prompt engineering did not give reliable control, producing under- or overproduction depending on the prompt |
| Rehearsing an interview will improve your interview performance | Not established | No study cited here measured transfer from AI rehearsal to real-world outcomes. The mechanism is plausible; the effect is unmeasured |
| Enough practice will make you good at this | Partial at best | Macnamara, Hambrick and Oswald’s meta-analysis found deliberate practice explained 26% of performance variance in games, 21% in music, 18% in sports, 4% in education and less than 1% in professions |
This table is a CEOtudent editorial framework. Each claim is graded against the strongest published test available to us, with Supported meaning a study directly tested it, Unsupported meaning the strongest test contradicts it, and Not established meaning no cited study measured it either way.
That last row deserves more than a line. Macnamara, Hambrick and Oswald published “Deliberate Practice and Performance in Music, Games, Sports, Education, and Professions: A Meta-Analysis” in Psychological Science in 2014, and found deliberate practice explained 26% of the variance in performance for games, 21% for music, 18% for sports, 4% for education and less than 1% for professions. Their conclusion was that deliberate practice is important but not as important as had been argued.
Professional performance is the row you are in. Under 1% of the variance.
That is not an argument against rehearsing. It is an argument against expecting rehearsal to be the thing that decides the outcome. What rehearsal reliably does is remove the specific failure of not having said the words before, which is a real failure and worth removing. What it does not do is substitute for the substance of your case, your actual leverage in the negotiation, or the preparation that has nothing to do with delivery.
Practise the delivery. Do not confuse it with the argument.
A realistic session
Twenty minutes, three runs, one scenario.
Minutes 0 to 3: write the brief. Role, interest, constraint, mood, private information, three required behaviours from the table.
Minutes 3 to 8: run one. Speak your actual words. Freeze at the hardest moment, not at the resolution. Check the tell column. If the counterpart went soft, this run does not count.
Minutes 8 to 12: run two, same scenario, one variable changed and a sharper difficulty specification. This is where the unreliability of prompt control gets absorbed rather than argued with.
Minutes 12 to 16: run three, hardest specification you can write.
Minutes 16 to 20: paste all three transcripts into the mentor conversation and ask the three questions across the set rather than per run. The pattern across three attempts is more informative than any single critique, because it shows you what you do every time under pressure rather than what you did once.
Then stop. Ericsson and colleagues found the effective duration of deliberate practice may be closer to an hour a day than the four or more that intuition suggests, and twenty dense minutes against real resistance is worth more than an evening of comfortable roleplay.
For the surrounding layers: AI tutors in 2026 compares which products are built to teach rather than to answer, what the evidence says about learning ranks the study techniques themselves by research strength, the 20-hour AI tutor protocol is the compressed sprint version for knowledge rather than behaviour, and context engineering covers the wider skill of specifying what a model needs to hold in order to behave the way you need.
Frequently asked questions
Can ChatGPT or Claude really simulate a job interviewer?
It can hold the role convincingly. What it does not do by default is behave like a difficult one. The CoCoEval evaluation found that conversations simulated by GPT-4.1, GPT-5.1 and Claude Opus 4 contained far fewer inconsistent and uncollaborative behaviours than real human conversations under ordinary prompting. So expect a competent, cooperative, unusually attentive interviewer unless you specify otherwise and verify that the specification took.
How do I make the AI a tougher practice partner?
Name specific behaviours rather than asking for difficulty in general: interrupting, misremembering what was agreed, raising an objection you already answered, going silent after your number. Then verify from the outside by running the scenario more than once and checking for the observable tells that it went soft. The study that documented the underproduction problem also found prompt engineering did not give reliable control in either direction, which is why verification across runs matters more than the wording of any single instruction.
Should the AI give me feedback during the roleplay?
No. Use two separate conversations, a partner and a mentor, following the framework Yang and colleagues described. A model that coaches mid-scene cannot hold a credible adversarial character, and breaking frame converts the exercise from retrieval practice back into reading, which the testing-effect research suggests is the weaker of the two at any delay.
Does practising with AI actually improve real performance?
Not established by any study cited here. The underlying mechanisms, retrieval practice and deliberate practice, are well evidenced in other domains, but no research examined here measured transfer from AI rehearsal to real interview or negotiation outcomes. Treat the method as removing a specific deficiency, never having said the words aloud under resistance, rather than as a demonstrated performance improvement.
How long should a practice session be?
Short and dense. Ericsson and colleagues reported essentially no benefit from practice beyond four hours a day, reduced benefits beyond two, and suggested the effective duration may be closer to one hour. For conversational rehearsal, twenty minutes covering three runs of one scenario plus a debrief is a reasonable unit, and it beats an hour of agreeable roleplay.
Is this worth doing for every difficult conversation?
No. Reserve it for conversations where your delivery under pressure is genuinely the binding constraint, and where you will not get a second attempt. Where the constraint is the strength of your case, your leverage, or the facts you have assembled, rehearsal will make you feel more prepared without making you more likely to succeed, and that particular illusion is worth avoiding.
Sources
- Kamoi, Godbole, Yang, Zhang, Wan and Zhou, Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction, arXiv preprint, March 2026, not peer-reviewed
- Yang, Ziems, Held, Shaikh, Bernstein and Mitchell, Social Skill Training with Large Language Models, arXiv preprint, April 2024, not peer-reviewed
- Ericsson, Krampe and Tesch-Römer, The Role of Deliberate Practice in the Acquisition of Expert Performance, Psychological Review, volume 100, issue 3, 1993
- Macnamara, Hambrick and Oswald, Deliberate Practice and Performance in Music, Games, Sports, Education, and Professions: A Meta-Analysis, Psychological Science, volume 25, 2014, pages 1608 to 1618
- Roediger and Karpicke, Test-enhanced learning: taking memory tests improves long-term retention, Psychological Science, volume 17, issue 3, 2006, pages 249 to 255
- Hattie and Timperley, The Power of Feedback, Review of Educational Research, volume 77, issue 1, 2007, pages 81 to 112
This content was compiled with the support of AI following in-depth research, then written and prepared for publication by the CEOtudent editorial team.
This post is also available in:
