External validity of DCEs: a case study of dental care

PhD: External validity of DCEs: a case study of dental care

[Back to PhD Projects]

This was a cross-theme PhD project, within Assessment of Technologies and Methods of Benefit Valuation.

Stated preference methods in health are sometimes criticised due to concerns over the external validity of the results. Most of these concerns relate to hypothetical bias, where respondents to a survey may not follow through on their stated choices if offered an identical choice in reality. The implication of hypothetical bias may be incorrect predictions of service uptake or biased estimates of WTP, leading to incorrect policy recommendations from cost–benefit analysis. The thesis focused on the challenge of hypothetical bias, and investigated several different mitigation techniques.

Ex ante mitigation methods focus on addressing hypothetical bias a priori, before respondents complete the choice task. Three different methods: oath scripts, consequentiality scripts and cheap talk scripts with an opt-out reminder were compared with a standard approach.

Ex post mitigation methods aim to calibrate choice responses, often based on certainty scales, to recode or statistically calibrate stated and revealed preferences based on the assumption that more certain responses are less likely to suffer from hypothetical bias. The thesis compared the use of different calibration approaches (recoding and elimination of uncertain responses) using quantitative and qualitative certainty scales. 

Predictions of service uptake (scale and polish and dental check-ups) from two DCEs were compared with revealed preference data, collected using retrospective questionnaires, to determine the methods generating the best predictive validity. Willingness to pay was estimated across groups. 

Outcome and Translation

DCEs can be reliably used in dentistry. They generate accurate service opt-in predictions for scale and polish and dental check-ups. However, it is less clear how accurately DCEs predict the uptake of specific dental care service configurations, especially check-up recall intervals. Ex-ante corrections (cheap talk, consequentiality scripts and honesty oaths) have little effect on WTP or data quality. However, it should be acknowledged that scope to show benefit is limited because the magnitude of hypothetical bias in the binary decision to consume a dental care service (i.e. the opt-in decision) was small. Consequentiality scripts and honesty oaths may have the greatest potential to address hypothetical bias, should it exist, and further research is required. It is encouraging that ex-ante corrections can be used without adversely impacting on DCE data quality, with no evidence to preclude their use in future studies.
Ex-post certainty corrections however should not be used routinely in DCEs to mitigate hypothetical bias in their current, most widely used (recoding) form. Ex-post calibrations require selection of arbitrarily selected threshold values to determine which responses are valid and which are not. The approach does not improve congruence between stated and real preferences but has serious implications for data quality and theoretical validity and raises equity concerns for using the method to inform policy recommendations. Recently published, and ongoing research is striving to find a better way to use certainty in DCEs (Beck, et al., 2016; Regier, et al., 2017). It is likely that these new and emerging approaches may pave a more positive way for the use of certainty in future DCEs, though this area of research is only in its infancy.

Papers arising from this work are currently being prepared for submission to peer reviewed journals.

PhD Student: Dwayne Boyers

Supervisors: Marjon van der Pol ; Verity Watson (HERU)

Publications

Boyers, D. (2020) 'External validity of discrete choice experiments: an application to the valuation of dental care', PhD Thesis, HERU, University of Aberdeen.

Presentations

Boyers, D., Pol, M. van der and Watson, V. (2016) 'Ex-ante corrections of hypothetical bias in a dental care DCE', [Poster] International Academy of Health Preference Research (IAHPR) Meeting, Nord Event Panoramadeck, Hamburg, Germany, 13 July 2016.

Boyers, D., Pol, M. van der and Watson, V. (2016) 'Using DCEs to estimate willingness to pay for dental care', European Health Economics Association Conference, 'Know the Ropes - Balancing Costs and Quality in Health Care', Universität Hamburg, Germany, 13-16 July 2016.

Watson, V., Boyers, D. and Pol, M. van der (2018) 'Using qualitative methods to design person-centred Discrete Choice Experiment surveys', 12th European Conference on Health Economics (EuHEA), Maastricht, The Netherlands, 11-14 July 2018.

Boyers, D.Pol, M. van der and Watson, V. (2019) 'External validity of Discrete Choice Experiments in dental care - the impact of ex-ante hypothetical bias correction methods', Health Economists’ Study Group (HESG) Summer Meeting 2019, University of East Anglia, Norwich, England, 3-5 July 2019.