Development and temporal validation of a clinical prediction model of transition to psychosis in individuals at ultra-high risk in the UHR 1000+ cohort

Simon Hartmann; Dominic Dwyer; Blake Cavve; Enda M Byrne; Isabelle Scott; Caroline Gao; Cassandra Wannan; Hok Pan Yuen; Jessica Hartmann; Ashleigh Lin; Stephen J Wood; Johanna T W Wigman; Christel M Middeldorp; Andrew Thompson; Paul Amminger; Monika Schlögelhofer; Anita Riecher-Rössler; Eric Y H Chen; Ian B Hickie; Lisa J Phillips; Miriam R Schäfer; Nilufar Mossaheb; Stefan Smesny; Gregor Berger; Lieuwe de Haan; Merete Nordentoft; Swapna Verma; Dorien H Nieman; Patrick D McGorry; Alison R Yung; Scott R Clark; Barnaby Nelson

doi:10.1002/wps.21240

Development and temporal validation of a clinical prediction model of transition to psychosis in individuals at ultra-high risk in the UHR 1000+ cohort

World Psychiatry. 2024 Oct;23(3):400-410. doi: 10.1002/wps.21240.

Authors

Simon Hartmann^{1

2

3}, Dominic Dwyer^{2

3}, Blake Cavve⁴, Enda M Byrne⁵, Isabelle Scott^{2

3}, Caroline Gao^{2

3}, Cassandra Wannan^{2

3}, Hok Pan Yuen^{2

3}, Jessica Hartmann^{2

3

6}, Ashleigh Lin⁵, Stephen J Wood^{2

3

7}, Johanna T W Wigman⁸, Christel M Middeldorp^{9

10

11

12

13}, Andrew Thompson^{2

3}, Paul Amminger^{2

3}, Monika Schlögelhofer¹⁴, Anita Riecher-Rössler¹⁵, Eric Y H Chen^{16

17

18}, Ian B Hickie¹⁹, Lisa J Phillips²⁰, Miriam R Schäfer^{2

3}, Nilufar Mossaheb²¹, Stefan Smesny²², Gregor Berger²³, Lieuwe de Haan²⁴, Merete Nordentoft^{25

26}, Swapna Verma^{18

27}, Dorien H Nieman²³, Patrick D McGorry^{2

3}, Alison R Yung²⁸, Scott R Clark¹, Barnaby Nelson^{2

3}

Affiliations

¹ Discipline of Psychiatry, Adelaide Medical School, University of Adelaide, Adelaide, SA, Australia.
² Orygen, Melbourne, VIC, Australia.
³ Centre for Youth Mental Health, University of Melbourne, Melbourne, VIC, Australia.
⁴ Telethon Kids Institute, University of Western Australia, Perth, WA, Australia.
⁵ School of Population and Global Health, University of Western Australia, Perth, WA, Australia.
⁶ Department of Public Mental Health, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Mannheim, Germany.
⁷ School of Psychology, University of Birmingham, Birmingham, UK.
⁸ Department of Psychiatry, Interdisciplinary Center Psychopathology and Emotion Regulation, University of Groningen, University Medical Center Groningen, Groningen, The Netherlands.
⁹ Child Health Research Centre, University of Queensland, St. Lucia, QLD, Australia.
¹⁰ Department of Child and Adolescent Psychiatry and Psychology, Amsterdam University Medical Center, Amsterdam Public Health Research Institute, Amsterdam, The Netherlands.
¹¹ Arkin Mental Health Care, Amsterdam, The Netherlands.
¹² Levvel, Academic Center for Child and Adolescent Psychiatry, Amsterdam, The Netherlands.
¹³ Child and Youth Mental Health Service, Children's Health Queensland Hospital and Health Service, Brisbane, QLD, Australia.
¹⁴ BioPsyC - Biopsychosocial Corporation, Vienna, Austria.
¹⁵ Medical Faculty, University of Basel, Basel, Switzerland.
¹⁶ Department of Psychiatry, School of Clinical Medicine, LKS Faculty of Medicine, University of Hong Kong, Hong Kong, Hong Kong.
¹⁷ LKC School of Medicine, Nanyang Technological University, Singapore, Singapore.
¹⁸ Institute of Mental Health, Singapore, Singapore.
¹⁹ Brain and Mind Centre, University of Sydney, Sydney, NSW, Australia.
²⁰ Melbourne School of Psychological Sciences, University of Melbourne, Melbourne, VIC, Australia.
²¹ Department of Psychiatry and Psychotherapy, Clinical Division of Social Psychiatry, Medical University of Vienna, Vienna, Austria.
²² Department of Psychiatry, Jena University Hospital, Jena, Germany.
²³ Department of Child and Adolescent Psychiatry and Psychotherapy, Psychiatric Hospital, University of Zurich, Zurich, Switzerland.
²⁴ Department of Psychiatry, Academic Medical Center, Amsterdam, The Netherlands.
²⁵ Mental Health Center Copenhagen, Research Unit (CORE), Copenhagen, Denmark.
²⁶ Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
²⁷ Office of Education, Duke-NUS Graduate Medical School, Singapore, Singapore.
²⁸ Institute for Mental and Physical Health and Clinical Translation, Deakin University, Geelong, VIC, Australia.

Abstract

The concept of ultra-high risk for psychosis (UHR) has been at the forefront of psychiatric research for several decades, with the ultimate goal of preventing the onset of psychotic disorder in high-risk individuals. Orygen (Melbourne, Australia) has led a range of observational and intervention studies in this clinical population. These datasets have now been integrated into the UHR 1000+ cohort, consisting of a sample of 1,245 UHR individuals with a follow-up period ranging from 1 to 16.7 years. This paper describes the cohort, presents a clinical prediction model of transition to psychosis in this cohort, and examines how predictive performance is affected by changes in UHR samples over time. We analyzed transition to psychosis using a Cox proportional hazards model. Clinical predictors for transition to psychosis were investigated in the entire cohort using multiple imputation and Rubin's rule. To assess performance drift over time, data from 1995-2016 were used for initial model fitting, and models were subsequently validated on data from 2017-2020. Over the follow-up period, 220 cases (17.7%) developed a psychotic disorder. Pooled hazard ratio (HR) estimates showed that the Comprehensive Assessment of At-Risk Mental States (CAARMS) Disorganized Speech subscale severity score (HR=1.12, 95% CI: 1.02-1.24, p=0.024), the CAARMS Unusual Thought Content subscale severity score (HR=1.13, 95% CI: 1.03-1.24, p=0.009), the Scale for the Assessment of Negative Symptoms (SANS) total score (HR=1.02, 95% CI: 1.00-1.03, p=0.022), the Social and Occupational Functioning Assessment Scale (SOFAS) score (HR=0.98, 95% CI: 0.97-1.00, p=0.036), and time between onset of symptoms and entry to UHR service (log transformed) (HR=1.10, 95% CI: 1.02-1.19, p=0.013) were predictive of transition to psychosis. UHR individuals who met the brief limited intermittent psychotic symptoms (BLIPS) criteria had a higher probability of transitioning to psychosis than those who met the attenuated psychotic symptoms (APS) criteria (HR=0.48, 95% CI: 0.32-0.73, p=0.001) and those who met the Trait risk criteria (a first-degree relative with a psychotic disorder or a schizotypal personality disorder plus a significant decrease in functioning during the previous year) (HR=0.43, 95% CI: 0.22-0.83, p=0.013). Models based on data from 1995-2016 displayed good calibration at initial model fitting, but showed a drift of 20.2-35.4% in calibration when validated on data from 2017-2020. Large-scale longitudinal data such as those from the UHR 1000+ cohort are required to develop accurate psychosis prediction models. It is critical to assess existing and future risk calculators for temporal drift, that may reduce their utility in clinical practice over time.

Keywords: UHR 1000+ cohort; Ultra‐high risk for psychosis; brief limited intermittent psychotic symptoms; disorganized speech; negative symptoms; prediction model calibration; prediction of transition to psychosis; psychosocial functioning; temporal validation; unusual thought content.