The Association of Market and Social Research Organisations (AMSRO) today announced that the Inquiry into the Performance of the Opinion Polls at the 2019 Australian Federal Election (AMSRO Polling Inquiry) has released its final report, including recommendations for how to improve political polling in Australia.
The report follows AMSRO’s decision, along with the Statistical Society of Australia, to instigate an independent and impartial review of election polling in Australia to determine why all the published polls incorrectly called the outcome at 2019’s May Federal Election and how polling might be improved in the future.
As evidenced by the Inquiry, from 2007 until 2016, Australian pollsters had, in aggregate, a 96% success rate (that is, 25 out of 26 final week polls called the right result) across four Federal elections. The 2019 May election, however, was a notable anomaly and AMSRO and the Statistical Society of Australia, on behalf of the wider industry, wanted to know what went wrong. AMSRO invited pollsters, media organisations and others who commission election and political polling to contribute to the inquiry.
The independent inquiry report concluded that the collective performance of the published national election polls in 2019 constituted a ‘polling failure’ rather than simply a ‘polling miss’, as the polls erred in their estimate of the vote in a manner that was statistically significant and erred in the same direction and at a similar level; and found that the source of errors lay in the polls themselves and was not a result of a last-minute shift in preferences among voters.
The AMSRO Polling Inquiry Chair, Darren Pennay (Founder and former CEO of The Social Research Centre) said: “The first preference voteswere either underestimated (LNP) or overestimated (ALP) because of inadequately adjusted, unrepresentative samples. It is very likely the polls were skewed towards over-representing more politically engaged and better educated voters and this bias was not corrected. As a result, the polls over-represented Labor voters.
“As a consequence, pollsters should seek to better understand the biases in their samples and to develop more effective sample balancing and/or weighting strategies to improve representativeness, by looking at education or other variables.”
The Inquiry Panel also found some evidence that the reporting of the polls failed to consistently meet the basic disclosure guidelines for editors and journalists set out by the Australian Press Council.
The Inquiry report has made 10 key recommendations aimed at improving polling methods and their disclosure. Recommendations include calling for an enforceable Code of Conduct for Election Polling; consistent, enforceable disclosure standards; and recommendations to improve polling methods and better educate the media and public on polling.
AMSRO Board member (and AMSRO President when the Inquiry was launched), Craig Young, said: “Election and political polling are consequential and influential and play an important role in the democratic process. AMSRO welcomes this final report from the Inquiry Panel and is exceedingly grateful to all those who have volunteered their time and expertise to support this important Inquiry.
“We also welcome the recent formation of the Australian Polling Council and recognise that their stated objectives are aligned with the recommendations of the Inquiry report regarding the need for greater transparency, and the creation of a Code of Conduct and polling guidelines, as well as better education of the media and public about polling. We look forward to supporting Council members as they develop their Code of Conduct. A code that delivers greater transparency and disclosure regarding how the published polls are conducted, is critical to re-establishing public confidence in polling. As the research industry association, AMSRO supports extending such disclosure standards to cover all published market research, not just political polling.
“We hope that this report ultimately provides the polling companies with useful information, ideas and further lines of inquiry to assist them in their efforts to improve the accuracy – and public perceptions of the accuracy – of election and political polling in Australia. AMSRO looks forward to working with the polling companies and other important stakeholders to support the creation of appropriate standards to bring about these much-needed changes.”
The final report is released today and further information on the AMSRO Polling Inquiry can be found at: https://www.amsro.com.au/amsro-polling-inquiry-final-report
Notes for editors:
The Inquiry Panel:
The Advisory Board:
For more information please contact:
Executive Director, AMSRO
0417 665 144
Third Avenue Consulting
0408 774 577
The Association of Market & Social Research Organisations (AMSRO) is the peak body for the market and social research, data and insights industry. AMSRO works in partnership with its company members, plus privacy authorities, business, government, and the community to protect and promote the industry and uphold the highest ethical and privacy market and social research standards. Since its establishment in 1989, AMSRO has grown to more than 90 members, who employ over 5,000 people, representing 70% of the industry’s annual (data collection) turnover. In 2003, AMSRO pioneered its own privacy code for members which won an Australian Privacy Award in 2009. www.amsro.com.au
About the Statistical Society of Australia The Society was founded in 1962 as a national umbrella’ organisation to support and further the work of state statistical societies already in existence, to establish a national journal and newsletter and to host national conferences. The national Society now represents Australian and overseas statisticians with state branches in six states and territories providing the focus and organisation of local activities. The overall objective of the Society is to further the study, application and good practice of statistical theory and methods in all branches of learning and enterprise.