Key Takeaways
AI works like an overly confident intern: fast and capable, but it rarely asks questions and sounds just as sure when it’s wrong. When a practicum student uses AI, that’s a practicum student supervising an intern, a responsibility we would never give them.
Medical educators have named the risk that matters most in training: never-skilling, when a learner never builds a foundational skill because AI did the effortful work that the skill grows from.
A polished product is no longer good evidence of competence. Supervisors need to sample the reasoning, not just the report.
AI access can be granted the same way we already grant independence: gradually, task by task, as trainees show they can do and check the work themselves.
The simplest supervision move is also the most revealing: "I like this. Tell me more about it."
Would you let a practicum student supervise an intern?
I encourage people to think of AI as an overly confident intern. Every supervisor has worked with one. They’re quick, eager, and capable of producing polished work, and they’re also the most challenging interns to supervise, because they don’t ask enough questions. They sound just as sure when they’re wrong as when they’re right, so you can’t use their confidence to tell you where to look. AI behaves the same way. It has limited context about the student in front of you, it rarely says “I’m not sure,” and its output looks finished whether or not it’s accurate.
Now consider who is supervising that intern. When a practicum student uses AI to draft an interpretation or a set of recommendations, the practicum student is the supervisor. We would never let a practicum student supervise an intern. We generally wouldn’t even let one intern supervise another. A supervisor needs the competence to catch what the supervisee gets wrong, and trainees are still building that competence.
That's the tension behind the question I hear most often from school psychology supervisors: How do I give my trainees the exposure to AI they need, but in a responsible way? Trainees will practice in a world where these tools are routine, so sending them out without AI skills is a disservice. But supervisors also notice that when a supervisee hands them a report that reads better than anything that student could produce on their own, they aren’t sure what it tells them.
AI isn't the problem. The problem is that supervision has always relied on certain signals of competence, and AI has quietly broken some of them.
Deskilling, never-skilling, and why the difference matters
Medicine has been working through this longer than we have, and it has given us useful language. A recent paper in Nature Medicine (Ke et al., 2026) separates three risks:
Deskilling: experienced professionals lose skills they once had because they've stopped using them.
Mis-skilling: learners absorb AI errors or biases as if they were correct, and those patterns become their own reasoning.
Never-skilling: learners never build foundational skills in the first place, because AI did the effortful work that skill grows from.
For supervisors, never-skilling is the biggest one to watch, because you can't lose a skill you never built. An experienced school psychologist who uses AI to draft a report is a seasoned supervisor checking an intern's work. A first-year practicum student using the same tool is in a worse position: a practicum student supervising an overly confident intern, while possibly skipping the very struggle that builds interpretive judgment.
Ke and colleagues present never-skilling as a risk model, not an established finding in clinical trainees. No one has yet followed a cohort of AI-trained school psychology interns and measured the outcome. But the signals from nearby fields point in a consistent direction:
In a randomized study, high school students with unrestricted access to a GPT-based tutor did better during practice but showed a 17% relative drop on a later exam taken without it (Bastani et al., 2025).
In a randomized trial with 52 mostly junior software engineers learning a new programming library, those who used AI averaged 50% on a follow-up quiz, compared with 67% for those who worked by hand (Shen & Tamkin, 2026).
In that same study, how people used AI mattered. Participants who used it to ask conceptual questions did far better than those who simply handed off the work.
That last finding is the hopeful part, and I'll come back to it.
The product is no longer the proof
For most of our careers, the product was a reasonable stand-in for the skill. A strong report usually meant the student understood the assessment. A well-reasoned intervention plan usually meant they could conceptualize the case.
That inference no longer holds. In a research project I led, 249 licensed psychologists rated psychological reports without knowing who wrote them (Lockwood et al., 2025). Some differences between psychologist-written and GPT-4-generated reports were statistically significant, but most effect sizes were small, meaning most differences had little practical meaning. On some dimensions, such as readability and jargon, there was no significant difference at all. The notable exception was recommendations, where raters favored the AI over the psychologists, a moderate effect. The study also used a model that is now outdated: GPT-4 in 2023, working from a single prompt. The technology has improved substantially since then.
If experienced psychologists can't tell the difference on the page, supervisors can't rely on the page either. The intern's work looks finished whether or not anyone understood it. Ke and colleagues call the result false proficiency: work that looks competent while independent skill lags behind. It tends to show up as:
Strong written products paired with weak verbal explanations
Confidence that collapses when the AI isn't available
Trouble walking you through the reasoning behind a conclusion
The harder problem isn't hallucination
Much of the public conversation about AI focuses on hallucinations: invented citations and wrong facts. Those matter, but they're the easier problem, because you can catch them by checking a source.
The harder problem is content that is true in general but not supported by this student's data. Think of a real, evidence-based intervention recommended with no basis in the evaluation. Nothing on the page looks wrong, and a citation checker won't flag it. Only someone who knows the case, and knows the domain, will catch it.
This is where trainees are most vulnerable to automation bias (i.e., the tendency to over-trust output from an automated system, even when other evidence should raise doubts; e.g., accepting a computer-generated score interpretation without checking it against the protocol). Research consistently finds this bias is strongest when verification is difficult (Goddard et al., 2012; Lyell & Coiera, 2017). A trainee without independent reasoning can't really verify an AI's interpretation. They can only accept it. And polished output can make a supervisee confident enough to work outside their actual competence.
Checking doesn’t fully solve this, either. A trainee can verify that every fact in a draft is accurate and still be caught, because the problem was never the facts. Fact-checking confirms that a statement is true. It doesn’t confirm that the statement belongs in this student’s report, and that second judgment is the one that requires competence.
Practical implications for supervisors
None of this argues for banning AI. Prohibition usually pushes use underground, where you can't supervise it, and it teaches nothing about privacy, verification, or disclosure. Unrestricted use has the opposite problem: you get a better product and a weaker learner. What works is structured use, the same way you'd structure a trainee’s growing responsibilities.
Practices that build on established supervision principles:
Have trainees explain first, then compare. Before they open an AI tool, ask them to tell you their interpretation or hypothesis. A written attempt is easy to generate with AI; a spoken one isn’t. Once they’ve seen the AI’s answer, it tends to anchor their thinking, so the independent attempt is what keeps the learning in.
Grant AI access the way you grant independence. Early on, trainees attempt the work first and use AI only afterward. As they show they can do and check the work, AI can come in earlier. This happens task by task: a trainee might be ready for AI help with a parent letter long before they're ready for it on a complex interpretation.
Sample the reasoning, not just the report. Ask, "I like this. Tell me more about it." Then follow up: Where did this conclusion come from? What else could explain these data? How would you explain this to a parent? It's the same preparation you'd give a supervisee before a team meeting.
Approaches that are promising but less tested:
Use AI to make trainees think harder. Have AI play a skeptical parent, rehearse the hard questions a team might ask, or argue against the trainee's interpretation. This is the kind of use the coding study associated with better learning.
Seed errors on purpose. Give a supervisee a polished AI interpretation that contains a subtle mistake, and ask them to find it, explain why it matters, and fix it.
Ask about AI use routinely. Normalize a few questions in case review: What did you ask AI to do? What did you do before you used it? What did you verify, and how? These parallel structured approaches now being proposed in medical education (Abdulnour et al., 2025).
Supervisors already bring the most important ingredient: the professional expertise to judge whether an interpretation fits the student. What many need to add is some hands-on familiarity with AI tools. You don’t need to become an AI expert, but a little direct experience is important.
Conclusion
Trainees need real exposure to AI, and they need it in a way that doesn't quietly replace the skills that make them school psychologists. The goal isn't to keep the AI intern out of the building. It's to make sure the person supervising that intern has the competence to do it, and that's the competence supervision exists to build.
A useful question to carry into your next supervision session: Is my supervisee getting better at the work, or getting better at producing the work? Those used to be the same thing. AI makes it possible for them to come apart, and supervision is one of the few places positioned to notice.
References
Abdulnour, R.-E. E., Gin, B., & Boscardin, C. K. (2025). Educational strategies for clinical supervision of artificial intelligence use. New England Journal of Medicine, 393, 786–797. https://doi.org/10.1056/NEJMra2503232
Bastani, H., et al. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122, e2422633122. https://doi.org/10.1073/pnas.2422633122
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012). Automation bias: A systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association, 19(1), 121–127. https://doi.org/10.1136/amiajnl-2011-000089
Ke, Y., Jin, L., Ong, J. C. L., et al. (2026). AI-induced never-skilling in medical education. Nature Medicine. Advance online publication. https://doi.org/10.1038/s41591-026-04438-y
Lockwood, A. B., Farmer, R. L., Shergill, G., Benson, N. F., & Gilbert, K. (2025). Human vs. machine: Comparing AI-generated and human-written psychological reports. Journal of Psychoeducational Assessment, 43(6), 559–573. https://doi.org/10.1177/07342829251346623
Lyell, D., & Coiera, E. (2017). Automation bias and verification complexity: A systematic review. Journal of the American Medical Informatics Association, 24(2), 423–431. https://doi.org/10.1093/jamia/ocw105
Shen, J. H., & Tamkin, A. (2026). How AI impacts skill formation [Preprint]. arXiv. https://arxiv.org/abs/2601.20245
AI Disclosure: Generative AI was used to assist with drafting and editing this post and to create the accompanying image. I reviewed and revised the content and take responsibility for its accuracy and final form.