Assessing Laboratory Method Validations for Informing Inference Across Survey Cycles in the National Health and Nutrition Examination Survey

Abstract

Background and objectives Laboratory tests conducted on survey respondents' biological specimens are a major component of the National Health and Nutrition Examination Survey. The National Center for Health Statistics' Division of Health and Nutrition Examination Surveys performs internal analytic method validation studies whenever laboratories undergo instrumental or methodological changes, or when contract laboratories change. These studies assess agreement between methods to evaluate how methodological changes could affect data inference or compromise consistency of measurements across survey cycles. When systematic differences between methods are observed, adjustment equations are released with the data documentation for analysts planning to combine survey cycles or conduct a trend analysis. Adjustment equations help ensure that observed differences from methodological changes are not misinterpreted as population changes. This report assesses the reliability of statistical methods used by the Division of Health and Nutrition Examination Surveys when conducting method validation studies to address concerns that adjustment equations are being overproduced (recommended too frequently). Methods Public-use 2017-2018 National Health and Nutrition Examination Survey laboratory data were used to simulate "new" measurements for 120 analytic method validation studies. Blinded studies were analyzed to determine the final adjustment recommendation for each study using difference plots, descriptive statistics, t-tests, and Deming regressions. Final recommendations were compared with simulated difference types to assess how often spurious results were observed. Concordance estimates (concordance, misclassification, sensitivity, specificity, and positive and negative predictive values) informed assessments. Results Adjustment equations were appropriately recommended for 75.0% of the studies, over-recommended for 5.8%, under-recommended for 15.8%, and recommended with an inappropriate technique for 3.3%. Across simulated difference types, sensitivity ranged from 65.9% to 84.4% and specificity from 74.7% to 97.5%. Conclusions Findings from this report suggest that the current methodology used by the Division of Health and Nutrition Examination Surveys performs moderately well. Based on these data and analyses, underadjustment was more prevalent than overadjustment, suggesting that the current methodology is conservative.