Using NLP to improve assessment

Improving and automating clinical assessment

In the late 1990’s - together with Peter Foltz - I (prof. Elvevåg) started applying natural language processing methods to improve assessment in psychiatry. The technology we used was similar to that used (by Peter) to automatically rate student essays. We applied it to clinical interviews to provide a second opinion about illness severity and to understand the underlying neurocognitive mechanisms of disordered thinking in clinical conditions that affect cortical function.

Highlights:

This was the first publication to apply natural language processing methods to psychiatric settings:

Elvevåg, B., Foltz, P., Weinberger, D.R. & Goldberg, T.E. (2007). Quantifying incoherence in speech: an automated methodology and novel application to schizophrenia. Schizophrenia Research, 93, 304-316. doi: 10.1016/j.schres.2007.03.001  Available at:  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1995127/

We built on this methodology to enable remote monitoring of psychiatric outpatients. This is the first publication to move this type of assessment out of the controlled lab or clinical setting:

Holmlund, T.B., Foltz, P.W., Cohen, A.S., Johansen, H.D., Sigurdsen, R., Fugelli, P., Bergsager, D., Cheng, J., Bernstein, J., Rosenfeld, E. & Elvevåg, B. (2019). Moving psychological assessment out of the controlled laboratory setting and into the hands of the individual: Practical challenges. Psychological Assessment, 31(3), 292-303.  doi: 10.1037/pas0000647

Stories are fundamental to our human experience and provide an effective way to organize information. Evaluating how stories are recalled provides critical information about the health of our brain. This publication shows how speech technology can automate such evaluations:

Holmlund, T.B., Chandler, C., Foltz, P.W., Cohen, A.S., Cheng, J., Bernstein, J.C., Rosenfeld, E.P. & Elvevåg, B. (2020). Applying speech technologies to assess verbal memory. npj Digital Medicine 3, 33. https://doi.org/10.1038/s41746-020-0241-7

See also ‘Remembering stories: Technology can help assess our memory’ https://healthcommunity.nature.com/posts/61733-remembering-stories-technology-can-help-assess-your-memory

This publication came out with a strong recommendation that the increased use of machine learning in psychiatry must ensure trustworthiness:

Chandler, C., Foltz, P.W. & Elvevåg, B. (2020). Using machine learning in psychiatry: The need to establish a framework that nurtures trustworthiness. Schizophrenia Bulletin, 46, 11-14. https://doi.org/10.1093/schbul/sbz105

This general idea of using technology and artificial intelligence for assessment in psychiatry generated a lot of interest in the popular press [see also tab concerning ‘Press’]: https://oxfordjournals.altmetric.com/details/69683984/news

RESEARCH GOALS

Building clinical decision support systems to support patients and clinicians

The promise of this work is to enable clinicians to monitor patients remotely, and alert them to issues or changes that arise between appointments that could be lifesaving. But… there remains a massive gap between the scientific success of detecting subtle differences or clinically significant change and the real-world implementations of such systems to assist clinicians by providing a second opinion. Our research aims to address some of these challenging issues.

For an introduction:

Chandler, C., Foltz, P.W., Cohen, A.S., Holmlund, T.B., Cheng, J., Bernstein, J.C., Rosenfeld, E.P. & Elvevåg, B. (2020). Machine learning for longitudinal applications of neuropsychological testing. Intelligence-Based Medicine, Vols.1-2, 100006.https://doi.org/10.1016/j.ibmed.2020.100006

For some reflections:

Elvevåg, B. (2023). Reflections on measuring disordered thoughts as expressed via language. Psychiatry Research, 322:115098. doi: 10.1016/j.psychres.2023.115098. https://doi.org/10.1016/j.psychres.2023.115098

https://en.uit.no/research/euresearch/art?p_document_id=795701#mod_795715