The Inter-rater Reliability of Volitional Questionnaire Chinese Version
Date Issued
2016
Date
2016
Author(s)
Chen, Wei-Ann
Abstract
Background and Purpose Many people with mental illness often appear to low motivation which result in poor activity participation and reduce the effectiveness of rehabilitation. In this study, we use the Chinese version of Volitional Questionnaire (VQ-C) to assess the client’s motivational status, the level of volition, of the activity participation, and to understand how the environment affects their volition. The evidence of internal consistency, test-retest reliability, and construct validity of the VQ-C had been studied previously. Since the VQ-C is conducted by assessing the level of volition after observing clients’ performance, the score results from different rater should be consistent. Thus, the purpose of this study is to examine the inter-rater reliability of the VQ-C conducted by well-trained clinical occupational therapists, and with a view to establish a model which can be the foundation for clinical occupational therapists to assess the clients’ participation motivation objectively and effectively, and can be expected to be one of the indexes of the evidence-based for clinical treatment in the future. Methods There are 3 raters in this study and the professional work experiences are 18, 14, and 11 years accordingly. Before the assessment, the raters need to read the manual of the VQ-C and practiced assessing a standard client followed by comparison and discussion with each other. Through complementing the standard of assessment more detail, it can be the criterion of scoring the research clients. Data of 100 participations with mental illness were collected in this study. Each client participated in one operating type activity chosen by self and the other one assigned by therapist. Meanwhile, at least 2 therapists conduct the score according to the client’s performance, and then clients need to complete other scales. All clients were recruited from psychiatric wards of a medical center in northern Taiwan. Data collection was conducted from March 2014 to November 2014. The study was approved by the ethics committees of the Tri-Service General Hospital. We use the Multi-Faceted Rasch Measurement Model, ICC, and multiple rater kappa to probe into the inter-rater reliability, and the statistical analysis was conducted by FACETS (version 3.71.4) and SPSS (version 17.0). Results The average age of the 100 clients is 47.8 years old (SD=12.6), 59 clients are men (59%), 37% of clients are schizophrenia, and the average number of year of onset is 12.5 years (SD=9.8). The separation index of the VQ-C is 6.42 and the separation reliability is 0.98. The rater consistency is 89.1%, the ICC is 0.84 (95% CI = 0.7~0.93, F = 16.91, p = 0.00), and the kappa of 14 items are between 0.348 to 0.663. It shows the VQ-C has acceptable inter-rater reliability, but still has a significant difference in rater severity. There are differential item functioning (DIF) of the VQ-C in diagnosis and activity type. Conclusion The Chinese version Volitional Questionnaire has fair inter-rater reliability. It is a reliable and effective observational assessment tool of volitional function. The difference of the severity between therapists in rating content will need to be described in detail and concrete continually to improve the consistency of the therapist, but as a basis for future use, it also need to establish a specific item difficulty index in different diagnosis and types of activities. Activity types used in this study is limited (limited to operating type activities), and it will need further verify the scoring consistency of different forms of activity, and establish the standard procedures of rater training program to improve the reliability of the VQ-C.
Subjects
Activity motivation
Volitional Questionnaire
Inter-rater Reliability
Rasch measurement model
Rating scale
SDGs
Type
thesis
File(s)![Thumbnail Image]()
Loading...
Name
ntu-105-R94429008-1.pdf
Size
23.32 KB
Format
Adobe PDF
Checksum
(MD5):b32b6abd2a962ded6dd9cdc41e3114e2