Developing a Performance-based Assessment to Measure Executive Functions in Patients with Schizophrenia
Date Issued
2014
Date
2014
Author(s)
Chiu, En-Chi
Abstract
Background and purposes: Executive dysfunction is a common cognitive deficit in patients with schizophrenia, which influences patients’ independence of carrying out daily activities. Common measures of assessing executive functions in patients with schizophrenia are the Wisconsin Card Sorting Test (WCST), Trail Making Test, Stroop Colour-word Test, etc. However, these three measures have two disadvantages: (1) score indices are unclear to represent components of executive functions; and (2) items are not functional activities, which may result in the scores not being able to reflect patients’ executive functions in real-life. Recent studies have started to use performance-based assessment that could reflect how executive dysfunction influences patients’ performance in real-life. Performance-based assessments that are currently being used in patients with schizophrenia are the Executive Function Performance Test, the Multiple Errands Test, and the Virtual Action Planning Supermarket. However, these three measures have two disadvantages: (1) the meaning of the test score is unclear, so users do not know what executive functions the scores represent; and (2) the items are based in a specific cultural context causing cultural differences and the items are not targeted at difficulties of functional activities in patients with schizophrenia. Therefore, this dissertation had three aims: (1) to develop a Performance-based measure of Executive Functions (PEF) based on an executive-function theory (Lezak model) for patients with schizophrenia; (2) to examine intra-rater reliability, inter-rater reliability, and practice effect of the PEF in patients with schizophrenia; and (3) to examine convergent validity and ecological validity of the PEF in patients with schizophrenia Methods: This dissertation contained three studies. Study 1 was to establish the PEF items, including three phases: (1) to conduct a literature review and administer interviews with patients with schizophrenia, caregivers, and experts to develop the preliminary PEF; (2) to implement expert consultation, cognitive interview, and pilot test to revise the preliminary PEF; and (3) to examine construct validity and delete items from the preliminary PEF using Rasch analysis to establish the final version of the PEF. Study 2 was to recruit two convenience samples of patients with schizophrenia. One convenience sample was for examining intra-rater reliability and practice effect. The other was for investigating inter-rater reliability. Each convenience sample was administered the PEF twice in a two-week interval. The intraclass correlation coefficient (ICC) was used to examine the extent of consistency between the two test scores administered by the same rater (intra-rater reliability) or two different raters (inter-rater reliability). The minimal detectable change (MDC) was estimated for the same rater or two different raters. Paired t-test was used for statistical analysis in practice effect. Cohen’s d was calculated to determine the size of practice effect. Study 3 was to recruit one convenience sample of patients with schizophrenia and administer four measures to examine concurrent validity and ecological validity. These four measures were the PEF, the WCST, the Activities of Daily Living Rating Scale III (ADLRS-III), and the Personal and Social Performance scale (PSP). Results: In Phase 1 of Study 1, the researcher compiled 47 instrumental activities of daily living (IADL) items to design an IADL questionnaire. The researcher used the IADL questionnaire to interview 32 patients with schizophrenia, 27 caregivers, and 12 psychiatric clinicians to develop the preliminary PEF. In Phase 2 of Study 1, the researcher conducted three expert consultations, three cognitive interviews, and five pilot tests to revise the preliminary PEF. In Phase 3 of Study 1, 200 patients with schizophrenia were administered the revised PEF. In Rasch analysis, the items with infit or outfit mean square < 0.6 or > 1.4 were deleted. The final version of the PEF contained 1 practice item and 13 test items to assess the four domains based on the Lezak model. The items of the individual domains were unidimensional. In Study 2, each of the two convenience samples had 60 patients of schizophrenia. For intra-rater reliability, ICC values of the four domains in the PEF were 0.88-0.92. The MDC values of the four domains (volition, planning, purposive action, performance effective) were 13.0, 12.2, 16.2, and 16.3, respectively. For inter-rater reliability, ICC values of the four domains were 0.82-0.89. The MDC values were 15.8, 17.4, 20.9, and 18.6 for volition, planning, purposive action, performance effective domains, respectively. Regarding practice effect, paired t-test was not statistically significant (p = 0.490-0.822) for volition and planning domains, but paired t-test was statistically significant (p ≤ 0.01) for purposive action and performance effective domains. The Cohen’s ds of the four domains were 0.01-0.24. In Study 3, 60 patients with schizophrenia participated. For convergent validity, the results showed moderate to strong correlations (r = 0.68-0.92) between the four domains. Patients who did not received a score in the index of “trials to complete first category” of the WCST were deleted. Moderate correlations (ρ = 0.43-0.69) were found among the four domains of the PEF and the four cognitive indices of the WCST (i.e., perseverative responses, perseverative errors, nonperseverative errors, and number of categories completed), except the low correlations between nonperseverative errors and two PEF domains (i.e., volition and planning). Regarding ecological validity, moderate to strong correlations (ρ = 0.42-0.86) were shown among the four domains of the PEF and the domains and total score of the ADLRS-III, except for the leisure domain. Moderate correlations (ρ = 0.31-0.56) were found among the four domains of the PEF and the domains and total score of the PSP. Conclusions: This dissertation was the first to develop the performance-based assessment to measure executive functions in patients with schizophrenia. The results of Study 1 supported construct validity (i.e., unidimensionality). That is, the items’ scores of each domain could be summed up to represent its domain-specific function. The results of Study 2 showed that the PEF had good intra-rater and inter-rater reliabilities, and trivial and small practice effects in patients with schizophrenia. The results of Study 3 showed that the PEF had sufficient convergent validity and ecological validity. As a whole, the PEF has good reliability and validity. The researcher provides the MDC values of the four domains in the PEF administered by one rater or different raters. The MDC values are useful for clinicians and researchers to determine whether the change of each domain in an individual’s score was real (beyond random measurement error). Therefore, the PEF is able to assist clinicians and researchers in understanding executive functions in patients with schizophrenia, and making treatment plans and clinical decisions to increase treatment efficacy.
Subjects
executive functions
schizophrenia
Lezak model
Performance-based measure of Executive Functions
psychometrics
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-103-D99429001-1.pdf
Size
23.32 KB
Format
Adobe PDF
Checksum
(MD5):7c1cdfc4f75e378bb742741a74e55fc2