Broader measures of value

Broader measures of value

While the Quality Adjusted Life Year (QALY) outcome metric is the most commonly used in economic evaluation of health care interventions, it is increasingly recognised as being insensitive, in some contexts, to what is valued by patients and the public.

We collaborate with the Methods of Benefit Valuation (MBV) theme to progress the application of broader measures of value in the economic evaluation of health technologies within a cost-benefit analysis (CBA) framework.

Current Projects

Effects and Safety of Testosterone in Men with Low Testosterone levels: an evidence synthesis and economic evaluation. The TESTES (Testosterone Effects and Safety) Consortium

Testosterone hormone is vital for sexual activity, bone growth and muscle development in men. Low testosterone levels may result in sexual problems, osteoporosis, weakness, low mood and tiredness. These symptoms can be treated with ART. Prescriptions of ART in men have nearly doubled within the last 10 years in the UK, yet the incidence of symptomatic men with low testosterone levels remains unchanged. Besides, there is conflicting evidence on the clinical effectiveness of ART with some studies suggesting a potential increased risk of cardiovascular events and prostate growth.

This project has three main components: 1) a systematic literature review (SLR) will identify relevant randomized controlled trials (RCTs), 2) a second SLR will be used to synthetize the existing qualitative evidence reporting men’s experience and acceptability of ART and 3) the development of a decision model to estimate the cost-effectiveness of ART for the treatment of symptomatic men with low testosterone.

As part of the project individual patient data (IPD) from relevant RCTs will be retrieved. The meta-analysis of these data will provide crucial inputs for the economic model.

HERU researchers involved in this research project: Rodolfo Hernández 

External collaborators: Jayasena, C. (Imperial College of Science, Technology and Medicine); Dhilo, W. (Imperial College London); Wu, F. (University of Manchester); Bhattacharya, S., Gillies, K., Brazzelli, M., Aucott, L. (University of Aberdeen); Quinton, R. (Freeman Hospital); Oliver, N. (Imperial College London).

PhD: Incorporating preference heterogeneity in economic evaluation: informing "realistic medicine"

The Chief Medical Officer in Scotland recently outlined her vision for realistic medicine, with key objectives to: (1) build a personalised approach to care; (2) promote shared decision making between patients and their doctors; (3) reduce unnecessary variation in practice and outcomes; (4) reduce harm and waste; (5) improve risk management; and (6) promote improvement and innovation. Some key aspects of this vision are shared decision-making and personalized care, where patients are guided to the treatment that provides the most value to them. This could potentially reduce healthcare costs by avoiding wasteful treatments that patients might not value.

Traditional economic evaluation methods focus on the costs and benefits of different alternatives treatment, using a measure of benefit that reflects population-averaged preferences for health outcomes. The aim of this thesis is to examine the effect of incorporating preference heterogeneity into the economic evaluation of health care interventions, and to see how policy recommendations might differ from those based on traditional evaluation methods. Incorporating preference heterogeneity into treatment choice could then lead to a more beneficial and more efficient allocation of healthcare resources.

PhD student:  Divya Mohan

Supervisors: Graham Scotland, Sebastian Heidenreich


PhD: Using existing data to incorporate broader measures of benefit in economic evaluation

This is a cross-theme PhD project, within Assessment of Technologies and Methods of Benefit Valuation.

The quality-adjusted life year (QALY) is the predominantly used measure of health benefit in economic evaluation. The QALY has many advantages, including readily available generic preference based instruments (e.g. EQ-5D) and its applicability across disease areas. However, these advantages are somewhat traded off by its narrow, health-oriented viewpoint and inability to capture benefits of health and healthcare outwith these generic instruments. For example, the QALY framework fails to adequately capture patient preferences for non-health attributes or the process of care.

A popular approach used in health economics to value all relevant benefits is the use of stated preference methods, where benefits can be measured in terms of willingness-to-pay (WTP). Despite their growing application in health economics, WTP measures of benefit are rarely used in economic evaluations. One reason is that measuring WTP is resource intensive (in terms of time and finances) because a new valuation study is required for each economic evaluation.

There are now many published monetary valuation studies that provide enough data to test if pre-existing WTP measures of benefit can be combined using benefit transfer (BT) as an alternative to conducting a new study. BT synthesises results from previously published studies and with adjustment, using all available and relevant information, predicts an estimate in a new study setting that is different in type, location or time from the original studies. This PhD will conduct two case studies to test the transferability of values. The first case study will focus across clinical areas (e.g. are values of the process of care the same across chronic and acute care settings?) while the second will focus within a clinical area (e.g. can values collected in perinatal care be transferred to post-natal care?).

To date, original monetary valuation studies have been conducted to value specific interventions, but the transferability of valuation results has not been explored. If BT can be successfully achieved in a healthcare setting, accessible methods to broaden the valuation space beyond the current QALY approach will be established.

PhD student: Emma Tassie

Supervisors: Verity Watson, Graham Scotland (HERU) and Stirling Bryan (University of British Columbia)


Recently Completed Projects

PhD: Broadening the valuation space in health technology assessment: the case of monitoring individuals with ocular hypertension


The economic evaluation (EE) component of health technology assessments (HTA) often defines value in terms of health related quality of life, with many HTA agencies requiring the use of EQ-5D based quality adjusted life years (QALYs). These approaches do not capture value derived from patient experience factors and the process of care. This thesis widened the valuation space beyond this limited perspective, taking account of such factors, using monetary values generated from a DCE, incorporating these into a discrete event simulation (DES) and conducting a cost–benefit analysis (CBA).

The case study monitored individuals with ocular hypertension (Project Number B2.21). Five strategies were compared using a DES: (1) ‘treat all’ at ocular hypertension diagnosis with minimal follow-up; (2), (3) biennial monitoring (either in primary or secondary care) with treatment according to predicted glaucoma risk; and monitoring and treatment according to the UK National glaucoma guidance (either (4) conservative or (5) intensive).

Outcome and Translation

DCE based WTP estimates for health outcomes (e.g. risk of developing or progressing glaucoma and treatment side-effects), patient experience factors (e.g. communication and understanding with the healthcare professional) and process of care (e.g. monitoring setting) were obtained. Conditional logit, mixed logit preference space and mixed logit WTP-space (rarely used within health economics) econometric specifications were used. These WTP valuations were aggregated in the DES, as fixed mean values or allowing variation between simulated individuals.

While the standard cost–utility analysis (CUA) using EQ-5D implied that ‘treat all’ was most likely cost-effective, CBA with broadened valuation space identified, consistently across different econometric specifications, ‘biennial hospital’ as the best choice.

This thesis proposed an approach to broaden the valuation space that can be promptly used for EE-HTA. Researchers should be attentive of the valuation space considered in their EE and choose wisely the EE approach to be used (e.g. CUA and/or CBA).

PhD student: Rodolfo Hernández

Supervisors: Mandy Ryan (HERU), Jen Burr (St Andrews University) and Luke Vale (Newcastle University)


PhD: External validity of DCEs: a case study of dental care

This was a cross-theme PhD project, within Assessment of Technologies and Methods of Benefit Valuation.

Stated preference methods in health are sometimes criticised due to concerns over the external validity of the results. Most of these concerns relate to hypothetical bias, where respondents to a survey may not follow through on their stated choices if offered an identical choice in reality. The implication of hypothetical bias may be incorrect predictions of service uptake or biased estimates of WTP, leading to incorrect policy recommendations from cost–benefit analysis. The thesis focused on the challenge of hypothetical bias, and investigated several different mitigation techniques.

Ex ante mitigation methods focus on addressing hypothetical bias a priori, before respondents complete the choice task. Three different methods: oath scripts, consequentiality scripts and cheap talk scripts with an opt-out reminder were compared with a standard approach.

Ex post mitigation methods aim to calibrate choice responses, often based on certainty scales, to recode or statistically calibrate stated and revealed preferences based on the assumption that more certain responses are less likely to suffer from hypothetical bias. The thesis compared the use of different calibration approaches (recoding and elimination of uncertain responses) using quantitative and qualitative certainty scales. 

Predictions of service uptake (scale and polish and dental check-ups) from two DCEs were compared with revealed preference data, collected using retrospective questionnaires, to determine the methods generating the best predictive validity. Willingness to pay was estimated across groups. 

Outcome and Translation

DCEs can be reliably used in dentistry. They generate accurate service opt-in predictions for scale and polish and dental check-ups. However, it is less clear how accurately DCEs predict the uptake of specific dental care service configurations, especially check-up recall intervals.

Ex-ante corrections (cheap talk, consequentiality scripts and honesty oaths) have little effect on WTP or data quality. However, it should be acknowledged that scope to show benefit is limited because the magnitude of hypothetical bias in the binary decision to consume a dental care service (i.e. the opt-in decision) was small. Consequentiality scripts and honesty oaths may have the greatest potential to address hypothetical bias, should it exist, and further research is required.

It is encouraging that ex-ante corrections can be used without adversely impacting on DCE data quality, with no evidence to preclude their use in future studies.

Ex-post certainty corrections however should not be used routinely in DCEs to mitigate hypothetical bias in their current, most widely used (recoding) form. Ex-post calibrations require selection of arbitrarily selected threshold values to determine which responses are valid and which are not. The approach does not improve congruence between stated and real preferences but has serious implications for data quality and theoretical validity and raises equity concerns for using the method to inform policy recommendations.

Recently published, and ongoing research is striving to find a better way to use certainty in DCEs (Beck, et al., 2016; Regier, et al., 2017). It is likely that these new and emerging approaches may pave a more positive way for the use of certainty in future DCEs, though this area of research is only in its infancy.

Papers arising from this work are currently being prepared for submission to peer reviewed journals.

PhD Student: Dwayne Boyers

Supervisors: Marjon van der Pol; Verity Watson (HERU)