A misleading poll on the Budget tax grab in the Times

It is hard to imagine how Populus could have made their poll, given prominent coverage by the Times, on the budget tax changes less reliable.  They make just about every mistake in the book.

Misleading questions

Populus found support for tax increases on "large, so-called gas-guzzling" cars; the poll accepts the accuracy of the Chancellor's spin as a given.

The very term "gas-guzzling" is clearly leading as it carries undertones of greed and wastefulness.  Even if the question had been framed in a more neutral manner, though, that isn't what we had in the Budget.  The Telegraph reports the findings of our research on this issue:

"Analysis shows that over the next two years, millions of drivers will face soaring bills as road tax on some popular family models doubles.

However, duty on Rolls Royces and Porsches will rise by less than the typical family saloon - undermining claims that the taxes are meant for gas guzzlers."

Put that in your poll and see what result you get...

Polling before the measures in the Budget were really understood

All the fieldwork for this poll was done on Wednesday evening, not long after the Chancellor's speech and before the morning papers.  At that stage the main news source will have been the BBC, which largely accepted the Chancellor's presentation of his VED changes.  Since then the true picture has been emerging.  With the most popular broadsheet and tabloid newspapers slamming the changes and exposing how misleading their initial presentation in the Budget was do we seriously think that public opinion won't have changed?

Small sample size

The last TaxPayers' Alliance annual conference poll (PPT) by YouGov surveyed 2,162 people.  By contrast, this new Populus poll only asked 596 people.  That means its results are far less reliable.  This is true for all of the results that the Times cites so confidently but particularly when they focus on particular social classes.  The number of, for example, "professionals and managers" in the survey will be very low.

No past vote weighting

Mike Smithson of the respected PoliticalBetting.Com website notes that:

"The survey did not include a voting intention question and was almost certainly not past vote weighted. This is likely to have produced a smallish but significant nevertheless pro-Labour sample."

This will probably mean people look less favourably on the Budget changes than this poll suggests.

Comments

[Sean Brown]

Gas guzzlers?? - my wife uses large MPV as we live in the countryside with practically zero public transport and need to transport our four children to / from schools, after-school clubs, sports and non-school based activity clubs for four children!!
Why no allowance for CO2 emissions and number of passengers carried???
1 person (teacher) makes 10km round journey at low CO2 of 104g/km = 1040g CO2 per person
5 people (driver 2ways - 4 children 1 way)make same 10km round journey at high CO2 of 203g/km = Average of 609g CO2 per person
Which is lower?

[Peter Riddell]

Just because you don't like the message, don't blame the messenger. The Populus poll in The Times was intended to measure immediate reactions to the Budget, no more. The sample size is obviously insufficient for a voting intention question, which is why we didn't do one. The margin of error on a sample that size is about 4 points either way, which is only slightly higher than the 2/3 points either way on 1,000, or the 1,500 sample size in our normally monthly poll. The point about past vote weighting may make a slight difference to the outcome, but not sufficient to affect the balance of the answers which showed a substantial margin backing the Budget measures, as well as accepting that taxes would rise as a result of the Budget. You concentrate on the vehicle excise duty question, but on when asked whether people approved "big increases on the duty levied on beer, wine and spirits" 55 per cent agreed and 39 per cent disagreed. So to claim we make every mistake in the book is both tendentious and false. The truth is that the initial-- and I emphasise initial- responses were complicated, as the full poll results show. As I argued in my commentary in The Times, the public appears to be in rather a fatalistic mood, recognising that taxes will go up and they will be worse off, but not believeing that an alternative Government would do better. It is silly, and demeans you, to attack a poll in such an extreme fashion when you don't like the results.
Peter Riddell

[Matthew Sinclair]

Peter,

I do think a lot of mistakes were made in that poll and you don't seem to dispute that any of the problems I've identified exist. You have a point that "every mistake in the book" should be qualified a little, though. Apologies if I phrased my blog a little too aggressively, the blogosphere brings out the worst in all of us!

However, aren't you at least as guilty of overstating the case in your initial article?

As you acknowledge, there are necessarily severe limitations with any poll performed so soon after an event. I think I’ve identified several of them in this case. Shouldn't your article have been framed as something along the lines of "initial reactions to budget favourable" rather than "we're happy to pay more tax on drink and cars"?

Best,

Matt

[Peter Riddell]

Matt,
It's always good to blog... I didn't write that "we're happy to pay more tax on drink and cars". But I did write that the polls provide "an immediate public response"... The caveats about a quickie poll do not alter the thrust of the poll's conclusions, which, as I stressed, were ambiguous, both supportive and fatalistic. I certainly do not believe that this is the final word-- merely, as with all polling and journalism, a snapsot on one day. Let's see how the debate develops...
best wishes
Peter

[Northernhousewife]

IF you take the poll seriously... we should also consider these aspects re the reaction to the govt from Anthony Wells...

'26% think it will be bad for business, with only 9% thinking it will help. 23% think it will increase the threat of recession, only 6% think it help reduce one.'

Not sure that fits in with the idea that people are fatalistic about the budget, unless you mean fatalistic about it failing to acheive anything or making matters worse.

I have to say as a bystander, who has a great respect for Mr Riddell,that I got a more negative impression of the reaction to this poll from UK Polling than I did from the Times article.

Maybe we all somwetimes see what we want.

[Northernhousewife]

16 is Biggest Conservative lead with Yougov
16 is Biggest lead by any party with Yougov (back to 2002)
27 is lowest Labour poll with Yougov

Guess that does not say it all, but it says more