How to Survive (and Even Thrive) in the Age of AI: Rethinking Assessment in School Psychology

A recent paper published in npg Mental Health Research explores the potential impact of large language models (LLMs) on the future of behavioral healthcare. LLMs, like the technology behind ChatGPT, are advanced AI systems capable of understanding and generating human-like text. This blog will focus on two key areas highlighted in the paper: assessment integration and changes to workforce dynamics and compensation models. Given that school psychology is deeply rooted in assessment, these areas have significant implications for the field.

Assessment: A New Era?

The paper points to a future where LLMs could revolutionize assessment. Imagine an LLM analyzing a student's written work, speech patterns, or even social media interactions to detect subtle signs of learning disabilities, emotional distress, or behavioral issues. LLMs could eventually conduct formal diagnostic interviews, using conversational chatbots or voice interfaces, to gather comprehensive information for assessment. This technology could:

• Increase Access: LLM-based assessments could be administered remotely, reaching students in rural or underserved areas who may lack access to school psychologists.

• Reduce Clinician Burden: LLMs could automate administration, scoring and interpretation of many standardized tests, freeing school psychologists to focus on more complex evaluations and interventions.

• Improve Accessibility: These tools could be particularly helpful for students with disabilities or those who find spoken communication more comfortable.

• Enhance Accuracy: LLMs, trained on vast datasets, could identify patterns and risk factors that humans might miss, leading to earlier and more accurate diagnoses.

However, it is crucial to approach this integration cautiously. The paper emphasizes the need for rigorous evaluation to ensure LLMs are reliable, valid, and free from bias. School psychologists must play an active role in this process, working alongside AI developers and historically marginalized groups to guide the ethical and effective implementation of these tools.

Workforce Dynamics: A Paradigm Shift?

The integration of LLMs could also significantly change the school psychology workforce. The paper suggests a potential shift towards a model where:

• Psychologists Focus on Complex Cases: School psychologists could operate at the top of their license, focusing on complex cases, supervision, and consultation.

• Paraprofessionals Handle Routine Assessments: Supported by LLM tools, paraprofessionals and technicians could take on routine assessments, expanding the reach of services.

This model could address the shortage of school psychologists and increase access to services. However, it necessitates careful consideration of:

• Training and Supervision: New training programs and certification standards will be needed to prepare paraprofessionals to work effectively with LLMs under school psychologists' supervision.

• Increased Supervision Load: School psychologists may find themselves supervising larger teams of non-professionals or even overseeing semi-autonomous LLM systems.

• Increased burnout: While is might seem counterintuitive, AI assistance might lead to more burnout if reduces "clinicians’ direct patient contact and perhaps increase their exposure to challenging or complicated cases not suitable for the LLM, which may lead to burnout and make clinical jobs less attractive" (p. 9).

• Compensation Implications: These shifts raise questions about compensation (not noted in the paper but they seem clear to me):

  • Will demand for school psychologists decrease if LLMs effectively handle routine tasks?

  • Will reliance on non-professionals put downward pressure on salaries for school psychologists?

The authors acknowledge that changes in workforce dynamics could lead to unintended consequences and emphasize the need for research to determine appropriate caseloads and guidelines for safe and effective implementation.

This is an exciting, yet complex moment for our field. The potential benefits of LLMs are vast, from reducing workload to expanding service reach, but it’s also crucial that we proceed thoughtfully to ensure these technologies truly benefit students. To stay relevant in this evolving landscape, I believe that school psychologists need:

• Leadership Skills: To lead teams of paraprofessionals and guide the integration of new technologies.

• Technical Skills: Understanding how LLMs and related technologies work will be essential to effectively supervise their use.

• Data Analysis Skills: The ability to interpret data generated by LLMs and use it to inform assessments and interventions.

• Interpersonal Skills: Strong communication and relationship-building skills will remain crucial, especially in supervising teams and maintaining the human element of care.

• Advocacy Skills: School psychologists will need to advocate for ethical and fair use of LLMs, ensuring that technology serves all students equitably.

• Flexibility and Adaptability: The ability to adapt to new roles, technologies, and changing workflows will be essential as the landscape of school psychology evolves.

• Collaboration Skills: Working effectively with interdisciplinary teams, including AI developers, educators, and paraprofessionals, will be key to successful integration of LLMs.

By developing these skills, school psychologists can continue to be leaders in assessment, ensuring that new technologies are used in ways that truly benefit students and maintain high standards of care. I'd love to hear your thoughts—how do you see LLMs fitting into school psychology? Could they be a helpful tool, or are there risks we need to guard against?

For on-the-go school psychologists, a podcast version of this material is available [here]. If you enjoyed this post, consider signing up for my weekly newsletter to stay updated on the latest in school psychology and AI.