System Maintenance Notice

Please note that maintenance work is scheduled between 4 AM and 5 AM - CET. During this time, you may experience reduced performance. Thanks for your patience.

Publications

Journal Article Epublish 4 citations

PubMed: 42493532 DOI: 10.1038/s41746-026-02928-4

Evaluating large language models for assessment of psychosis risk.

NPJ Digit Med 9(1)

Authors: Taiyu Zhu, Alexander Tashevski, Maxime Taquet, Matilda Azis, Tia Jani, Matthew R Broome, Thomas Kabir, Amedeo Minichino, Graham K Murray, Matthew M Nour, Ilina Singh, Paolo Fusar-Poli, Alejo Nevado-Holgado, Philip McGuire, Dominic Oliver

Affiliations

  • Department of Biostatistics & Health Informatics, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK.
  • Department of Psychiatry, University of Oxford, Oxford, UK.
  • NIHR Maudsley Biomedical Research Centre, London, UK.
  • Department of Psychiatry, University of Oxford, Oxford, UK.
  • NIHR Oxford Health Biomedical Research Centre, Oxford, UK.
  • Oxford Health NHS Foundation Trust, Oxford, UK.
  • Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK.
  • Institute for Mental Health, University of Birmingham, Birmingham, UK.
  • Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK.
  • Birmingham and Solihull Mental Health NHS Foundation Trust, Birmingham, UK.
  • OPEN Early Detection Service, Oxford Health NHS Foundation Trust, Oxford, UK.
  • Department of Psychiatry, University of Cambridge, Cambridge, UK.
  • Cambridgeshire and Peterborough NHS Foundation Trust, Cambridge, UK.
  • Max Planck UCL Centre for Computational Psychiatry and Ageing, University College London, London, UK.
  • NIHR Maudsley Biomedical Research Centre, London, UK. Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK. Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy.
  • Department of Psychiatry, University of Oxford, Oxford, UK.
  • NIHR Oxford Health Biomedical Research Centre, Oxford, UK.
  • Early Psychosis: Interventions and Clinical-Detection (EPIC) Lab, Department of Psychosis Studies, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK.
  • OPEN Early Detection Service, Oxford Health NHS Foundation Trust, Oxford, UK.

Abstract

Psychosis prevention relies on early detection of individuals at clinical high risk for psychosis (CHR-P). The effectiveness of the CHR-P state is constrained, in part, due to clinical assessments requiring specialist interpretation of narrative interviews, limiting scalability. Here, we evaluate whether large language models (LLMs; deep learning models trained on large text corpora to process and generate language) can extract clinically meaningful information from such interviews to support psychosis risk assessment. We assessed 11 open-weight LLMs on 678 partial PSYCHS interview transcripts from 373 participants (77.7% CHR-P). Models inferred CHR-P status and estimated severity and frequency across 15 symptom domains, benchmarked against researcher-rated scores. Larger models achieved the strongest classification performance (Llama-3.3-70B: accuracy = 0.80, sensitivity = 0.93, specificity = 0.58). LLM-generated symptom scores showed good correlations with researcher-rated scores (ICC sev = 0.74, ICC freq = 0.75). Performance disparities were minimal across most demographic groups but varied across sites. Generated summaries were largely faithful to source transcripts, with low rates of clinically relevant confabulation (3%). Errors primarily reflected over-pathologisation of non-clinical experiences. While accuracy scaled with model size, smaller models achieved competitive performance with substantially lower computational cost. These findings demonstrate that open-weight LLMs have the potential to assess psychosis risk from psychometric interview transcripts, supporting scalable, human-in-the-loop approaches to early detection.

MeSH

Language, Psychotic Disorders