The Validity and Expected Utility of the Two-Stage Screening Approaches for Infants and Toddlers with Developmental Delays
Date Issued
2007
Date
2007
Author(s)
Kuo, Yu-Ling
DOI
en-US
Abstract
Background and Purposes: It is important to apply reliable and valid developmental screening tests for the early intervention of children with developmental delays (DD) in clinics. The Simplified Child Developmental Screening Test (SiCDeST) is a simple and brief screening test used by some clinicians in Taiwan. However, psychometric properties of the SiCDeST are still limited. Based on evidence-based medicine, the multilevel likelihood ratios of a test are more powerful and useful than one single cut-off point. Therefore, first two purposes of this study are to investigate the test-retest reliability and the multi-level likelihood ratios of the SiCDeST. The problem of administering any single screening test in clinic is the low prevalence rate of DD. The two-stage positive or negative screening approaches are methods to increase the post-test probabilities to wait-test-treatment threshold. The third purpose of this study is then to estimate the validity of the two-stage positive/ negative screening approaches. In addition to validity indices, total expected utility (TEU) is used in the decision making for selecting screening stratgies or tests. However, the TEU of the two-stage screening approaches are not well investigated yet. Therefore, the fourth purpose of this study is to investigate the TEU of the two-stage positive/ negative screening approaches. Methods: Participants: Fifteen dyads were enrolled for the test-retest reliability of the 5 age groups of the SiCDeST, i.e. 6, 9, 12, 18, and 24 months. From the dataset of one previous longitudinal study, there were 266 infants who entered that study at 6-18 months and being followed up at 18-36 months. For data analysis, the children who received tests during 6-29.4 month-of-age and had complete data of the SiCDeST, CDIIT-ST as well as pediatrician’s diagnosis, were included in this study. Therefore, totally 406 data sets were included for analysis. Twenty-five professionals were recruited to fill the questionnaire for the utility estimation of 4 screening outcomes (true positive, true negative, false positive, and false negative) and cost. Procedure: The SiCDeST was filled by parents twice in the time interval of one week for the test-retest reliability study. For multi-level LRs of the SiCDeST, the raw scores of the SiCDeST were set at less than 60, 60, 70, 80, 90, and 100. The scores of the SiCDeST and Comprehensive Developmental Inventory for Infants and Toddlers- Screening Test (CDIIT-ST) were combined for validity analysis in two-stage positive and negative screening approaches. The reference criteria of DD was the diagnosis of one pediatrician who examine or observe the children with the information of two diagnosis tests, the Comprehensive Developmental Inventory for Infants and Toddlers- Diagnosis Test (CDIIT-DT) and Bayley Scales of Infants Development-II (BSID-II). One questionnaire with 5 visual analysis scale (VAS) was developed for estimate the utility values of 4 outcomes and the cost of the screening test. Statistics analysis: Quadratic weighted Kappa was used for the test-retest reliability analysis. Contingency tables were used to calculate the multi-level likelihood ratios of various score of the SiCDeST and validity indices of two-stage screening strategies. Screening validity indices of the two-stage screening approaches include: sensitivity, specificity, positive predictive value, negative predictive value, likelihood ratio positive, likelihood ratio negative, Youden index, and diagnostic odds ratio. Median values of utility for 4 outcomes and cost would be obtained from the questionnaire. Total expected utility would be calculated as the sum of the product of probabilities of four outcomes and its associated median utility, plus the utility of the cost. The α level was set at 0.05 (two tailed). Results: The test-retest reliability of the SiCDeST showed substantial to almost perfect agreement (κ = 0.63-0.94). The multi-level likelihood ratios of the SiCDeST at scores of < 60, 60, 70, 80, 90, and 100 were infinity, 7.02, 9.36, 2.40, and 0.66, respectively. Therefore, children with raw scores of the SiCDeST >= 80 were chosen for the validity and TEU analysis of two-stage screening approaches. The validity indices of the two-stage positive screening approaches were sensitivity 18%, specificity 98%, positive predictive value 27%, negative predictive value 96%, positive likelihood ratio 9, negative likelihood ratio 0.8, Youden index 16%, and diagnostic odds ratios 11.3. The validity indices of the two-stage negative screening approach were 53%, 82%, 12%, 97%, 2.9, 0.6, 35%, and 4.8, respectively. The TEU of the two-stage positive/ negative screening approaches were 0.91 and 0.73, respectively. Conclusions: The SiCDeST is a reliable and acceptable valid surveillance test filled by parents and it can be used in health care settings where the time is limited. However, its psychometric properties need further study. From the results of this study, clinicians may make appropriate decisions base on the multi-level LRs of the SiCDeST. From the results of this study, authors suggest children with raw scores of the SiCDeST < 60 need further intervention, with 60 or 70 need further diagnosis evaluation, of 80 need further second screening test (CDIIT-ST), and of 90 or 100 are arranged for next screening schedule. From the views of validity indices and TEU, two-stage positive screening approach is better than two-stage negative screening approach. Further studies in various setting are needed to clarify the feasibility and validity of the screening approaches before its widespread use.
Subjects
多階段篩選測驗
兒童
信度
效度
發展遲緩障礙
效用理論
Multiphasic Screening
Child
Reliability
Validity
Developmental Delay Disorders
Utility Theory
Type
other
File(s)![Thumbnail Image]()
Loading...
Name
ntu-96-R94428003-1.pdf
Size
23.31 KB
Format
Adobe PDF
Checksum
(MD5):9c7e792f00777dd7ea2f97467f1e6d5c