Scrutinising statistical claims and constructing balance: television news coverage of the 2016 EU Referendum

 

Section 2: Politics

During the 2016 EU Referendum campaign many people will have been confronted by a blizzard of facts and figures about the consequences of voting one way or another. So how well did UK television news interpret the statistical claims conveyed by both sides?

Cardiff University carried out a content analysis of evening television news bulletins over the ten week campaign. We examined bulletins on Channel 5 at 5pm, Channel 4 at 7pm and at 10pm on BBC, ITV and Sky News in order to assess not just the balance of issues, parties and personalities given airtime, but how broadcasters routinely scrutinised the claims made by the Leave and Remain campaigns.

There may have been exaggerated claims from both sides of the debate, but it was also the case that some of the statistics presented were more credible than others. In this campaign, however, there was a sense that broadcasters were afraid to make such a judgement. Objectivity, in this sense, was trumped by impartiality.

Overall, we found roughly half of the 571 items examined related to the process of the Referendum, notably about campaign walkabouts or the strategies of competing sides, as well as public opinion towards the EU and the way people would vote. There was also a focus on Conservative party infighting and speculation about the likely successor to David Cameron.

The economy was the major issue reported over the campaign, representing over a fifth of coverage, which included stories about economic forecasts and trade agreements. Immigration made up just over one in ten items, although this issue sometimes overlapped with stories about public services, housing and security.

Of the on-screen sources we examined, Boris Johnson made up 8.7% share of appearances – the most prominent figure for the Leave campaign – while 4.4% featured Nigel Farage. On the remain side, the PM and Chancellor were the most visible on television news (accounting for 7.2% and 4.1% respectively), with Jeremy Corbyn’s share of appearances – 2.4% – well behind Conservative sources.

But beyond the issues, personalities and party-politics involved in the Referendum coverage, it was perhaps the lack of scrutiny by non-partisan sources that was most conspicuous by their absence. If we exclude the 12% of sources that did not express a favourable or unfavourable opinion towards EU membership, Tables 1 and 2 show the overwhelming voices heard on television news were by Leave or Remain campaigners. More independent actors – from think tanks, say, or academics – made up a tiny share of sources used to inform coverage.

Since over 4 in ten items featured a statistical claim about the EU, the burden of independent scrutiny was thus left to journalists. But how did they communicate the competing facts and figures of the campaigns? About a quarter of news items involving statistics were either challenged or contextualised by journalists, such as questioning the claim made by Leave that the UK paid £350m per week to Brussels or explaining that much of this money was reinvested in the UK.. Put another way, 3 in 4 items involving statistical claims were not subject to either further analysis or additional context.

In effect, this meant much of the coverage about the EU Referendum was left to campaign groups to argue with each other about the relative merits of leaving or remaining. Indeed, we found almost a third of news items involving statistics were tit-for-tat exchanges between rival camps, where journalists did not intervene one way or the other about the veracity or credibility of competing claims. Without a great deal of prior knowledge, it would be very difficult for audiences to make sense of these claims and counter-claims, regardless of their veracity.

Why does all this matter? Although broadcasters have to abide by ‘due impartiality’ guidelines, this does not necessarily mean they have to be balanced when reporting facts and figures. The editorial goals of accuracy and objectivity involve challenging or questioning claims about being in or out of the EU. Our analysis of Referendum coverage suggests that, while broadcasters may have been even-handed in terms of giving both sides equal time, they could have more independently scrutinised, challenged or contextualised many of the facts and figures that were used repeatedly by both sides.

Despite the ten-week campaign, just days before Referendum day one representative survey found less than a third – 31% – felt well or very well informed about their EU vote. The tit-for-tat exchanges between rival camps and the trading of statistical claims would probably not have helped many viewers make better sense of the issues confronting them. There may have been exaggerated claims from both sides of the debate, but it was also the case that some of the statistics presented were more credible than others. In this campaign, however, there was a sense that broadcasters were afraid to make such a judgement. Objectivity, in this sense, was trumped by impartiality. More regularly drawing on independent analysis may have enhanced people’s understanding of the issues before they had cast their vote.

The research was carried out by Natasha Egan-Sjodin.