In statistics, we trust – but should we?
In this era of misinformation and fake news, it’s vital that we all understand how to assess credibility. Jon Andrews, Director for School System and Performance and Deputy Head of Research Education Policy Institute brought this into focus recently in a new report and concludes that one of the government’s “go-to statistic” is “flawed in several ways.”
The offending claim is that “there are now 1.9 million more children in good or outstanding schools than there were in 2010.” As well as being regularly used by the Department for Education (DfE) press office, the figure has been mentioned by two Prime Ministers, four Education Secretaries, and other ministers.
The first thing to say is that the statistic is factually correct. However, when deprived of context, it’s misleading. Rather than represent an increase in proportion, the statistic focussed only on numbers of students. The report shows us that the rise in numbers is misleading as it fails to account for population growth, changes to Ofsted grading, and the fact that many schools had not been inspected since 2010
The government use other misleading statistics too and another claim that is open to challenge is that they are spending more money on schools than ever before. Whilst technically true, this neglects to account for increasing numbers of students. In fact, the Institute for Fiscal Studies has revealed that per pupil spending has been cut by 8% since 2010, which is at odds with the intended message of the government’s claim.
Statistics are vital for understanding schools, and evaluating the effect of policy initiatives. Quantitative analysis can help us to build a picture of the truth, however, we must always be aware of the limitations of data, and what it can tell us. Our data analysts here at NCFE always present their findings with a long list of caveats and explanations, which helps us to ensure we use it appropriately and with integrity.
After such a thorough analysis and criticism, it was disappointing to see the DfE use the statistic again in their response. Though not wanting to accept criticism, the government should consider the importance of using statistics appropriately and building trust. Data and analytics is key to the success of policy making, and the government should be willing to listen to robust evidence and challenge.