What should we include in an economic evaluation of public health interventions and will policymakers listen anyway? This blog considers each issue in turn, and attempts to stimulate some feedback regarding the role and approach(es) that economic evaluation can best take.

What should we include in an economic evaluation? 

The premise of economic evaluation is to capture everything of value. So an economist should start by generating a ‘balance sheet’ that audits all the major outcomes, with benefits on one side and costs on the other. But then we may have a huge number of different outcomes – some positive, some negative. So, the key next step is to take this ‘social accountancy’ exercise further by valuing these outcomes where possible. For that, we need some generic measure of final outcome to map everything onto. This enables us to compare the overall value of different kinds of interventions, with the aim to inform policy where the ‘best buys’ are. That all sounds nice and logical, but which generic outcome measure(s) should we use? This is where the controversy begins and different economists get off at different stops!

If we take a ‘funder (or decision maker) perspective’, while recognising that some interventions are statutory in nature, then we make explicit the decision maker(s) objectives regarding what they wanted to achieve from the intervention. We then only value these benefits and costs and ignore the other outcomes on the balance sheet. This is narrow, but it’s exactly what health technology assessments (HTAs) do most of the time. The National Institute for Clinical Excellence (NICE) has a ‘reference case’ (essentially a ‘how to’ manual) which guides economic evaluators to look at quality adjusted life years and the costs to the NHS and personal social services.

But what about public health interventions that originate from, and usually impact on, multiple sectors? Does the opposite happen? For instance, a funder(s) of housing and regeneration interventions may be primarily interested in the number and quality of houses, as that is what is in the job description. If so, then health outcomes are really a by-product of the intervention, and may not influence the decisions to build new homes etc. On the other hand, in Glasgow there is a ten year research exercise underway, called GoWell, to look at the wider impacts (including health) of housing and regeneration and this is funded across multiple sectors including the Glasgow Housing Association, NHS, Justice and Scottish Government. This is promising news for a public health agenda!

A wider approach to economic evaluation is to take a ‘societal perspective’ and try to value all the different outcomes separately on the balance sheet. Time is really important in this context – in the sense of giving enough time for outcomes to manifest themselves. To then value all outcomes on a consistent basis, we canvass societal preferences. This approach is the way that mainstream economic evaluators go about their business (e.g. in transport, environment, welfare). Is this what those of us who are interested in further developing a public health agenda would like to see more of? If so, then in what sense are these ‘public health interventions’, as opposed to just ‘interventions’? Indeed economic evaluation takes place in these sectors anyway, so perhaps the role of the health economist is to ensure that health outcomes are sufficiently captured.  

Finally, we could take a ‘health perspective’. In this case, we would take the observed outcomes listed in the balance sheet and project everything onto expected long term health consequences. We’d also need to consider the long term consequences for net costs. Returning to the housing example, it may take a long time for health impacts to properly manifest themselves (even inter-generational). Research groups (e.g. MRC) spend considerable time trying to work out the impacts of different policies on health. So shouldn’t health economists also try to do the same? We think so. It also opens up interesting research opportunities regarding the possibilities for modelling and making more of our data, theories and ideas.

The great thing is that these approaches are not mutually exclusive and all fall out of the initial balance sheet. It’s not overkill either. All three approaches are practical, achievable and can serve to inform a range of stakeholders, as we briefly turn to now.

But, what about decision making?

Fine, let’s imagine we’ve developed our evidence base. The ‘so what’ question remains – how will this impact on future decision making? Perhaps there is a trade-off between the perspective taken and the immediate application of the information in policy circles.  If we take a ‘funder (decision maker) perspective’ then the information will be more relevant to immediate policy. As we move away from this perspective to either embrace the wider impacts of policies, or focus on the long term impacts on health, then perhaps we’re really entering the world of advocacy.

But then again, (economic) evaluation should also be used to challenge policy. If intersectoral consequences are important then we need to capture this. By demonstrating the impact of non-NHS sectors on health we can strengthen the public health agenda and help make the case for better coordination across sectors regarding the alignment of policy objectives. For instance, by capturing the health impacts of housing provision perhaps we can help strengthen the case for continued funding of new housing and inform the planning procedures. But how coordinated is policymaking in the real world: are budgets tightly ring-fenced?

……do we need institutional evolution?

It appears that real progress is being made in policy coordination at national and particularly at local levels (e.g. ‘One Glasgow’). However, the economists writing the blog all have past policy experience and it wasn’t so long ago that phrases such as cross-sector working were all-the-rage. The reality was often somewhat different. Perhaps that’s why we’re so intent on the so-what question. It’s ‘nice to know’ (that’s why we enjoy the elbow-patches of academia), but what information will decision makers/funders actually use?

Will adopting a range of perspectives (as described above) help or confuse? Economic evaluation is potentially too important just to be left to economists – and a recurring theme in future blogs will be the opportunity for interdisciplinary work as we try to improve our methods. The ‘economic’ part of the term economic evaluation should be there to add value to a coordinated evaluation effort (process and outcomes). There are really great signs that this is exactly what is happening in public health evaluation. Let’s discuss!