A permutation method to assess heterogeneity in external validation for risk prediction models
Journal
PLoS ONE
Journal Volume
10
Journal Issue
1
Date Issued
2015
Author(s)
Wang L.-Y.
Abstract
The value of a developed prediction model depends on its performance outside the development sample. The key is therefore to externally validate the model on a different but related independent data. In this study, we propose a permutation method to assess heterogeneity in external validation for risk prediction models. The permutation p value measures the extent of homology between development and validation datasets. If p < 0.05, the model may not be directly transported to the external validation population without further revision or updating. Monte-Carlo simulations are conducted to evaluate the statistical properties of the proposed method, and two microarray breast cancer datasets are analyzed for demonstration. The permutation method is easy to implement and is recommended for routine use in external validation for risk prediction models. ? 2015 Wang Lee.
Other Subjects
area under the curve; Article; breast cancer; controlled study; female; gene expression; human; major clinical study; microarray analysis; Monte Carlo method; permutation method; prediction; reproducibility; support vector machine; Breast Neoplasms; DNA microarray; gene expression profiling; genetics; risk; statistical model; Breast Neoplasms; Female; Gene Expression Profiling; Humans; Models, Statistical; Monte Carlo Method; Oligonucleotide Array Sequence Analysis; Reproducibility of Results; Risk
Publisher
Public Library of Science
Type
journal article