Mental Status Detection for Schizophrenia Patients via Deep Visual Perception
Journal
IEEE Journal of Biomedical and Health Informatics
Journal Volume
26
Journal Volume
26
Journal Issue
11
Journal Issue
11
Pages
5704
Start Page
5704
End Page
5715
ISSN
2168-2194
Date Issued
2022-11-01
Author(s)
Lin, Bing Jhang
Lee, Lue En
Chuang, Chih Yuan
Liu, An Sheng
Hung, Shu Hui
Abstract
Schizophrenia is a mental disorder that will progressively change a person's mental state and cause serious social problems. Symptoms of schizophrenia are highly correlated to emotional status, especially depression. We are thus motivated to design a mental status detection system for schizophrenia patients in order to provide an assessment tool for mental health professionals. Our system consists of two phases, including model learning and status detection. For the learning phase, we propose a multi-task learning framework to infer the patient's mental state, including emotion and depression severity. Unlike previous studies inferring emotional status mainly by facial analysis, in the learning phase, we adopted a Cross-Modality Graph Convolutional Network (CMGCN) to effectively integrate visual features from different modalities, including the face and context. We also designed task-aware objective functions to realize better model convergence for multi-task learning, i.e., emotion recognition and depression estimation. Further, we followed the correlation between depression and emotion to design the Emotion Passer module, to transfer the prior knowledge on emotion to the depression model. For the detection phase, we drew on characteristics of schizophrenia to detect the mental status. In the experiments, we performed a series of experiments on several benchmark datasets, and the results show that the proposed learning framework boosts state-of-the-art (SOTA) methods significantly. In addition, we take a trial on schizophrenia patients, and our system can achieve 69.52 in mAP in a real situation.
Subjects
Depression estimation | emotion recogni- tion | graph convolutional networks | transfer learning
Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Type
journal article