AMSRO Polling Inquiry 2019

Polling inquiry Frequently asked questions

Inquiry into the Performance of the Opinion Polls at the 2019 Federal Election

Why did AMSRO launch a polling inquiry?

The Federal Election result on May 18, 2019 caught almost everyone by surprise – the key reason being the mismatch between the result on the day and what the reported results of the leading polling organisations had been predicting over the preceding three years.

Since then, there has been widespread criticism of political polling, with no shortage of people inside and outside our profession providing their diagnoses of what went wrong and how to fix it.

Two things were clear:

  • There is no consensus from the polling companies, or any other quarter, as to ‘what went wrong.’
  • The credibility of opinion polling with the Australian public appears to be at an all-time low.

The Association of Market and Social Research Organisations (AMSRO) and the Statistical Society of Australia (SSA) believed it to be of vital importance to address public concerns about the accuracy of the polls because concerns about polling have the potential to undermine confidence in the wider market research industry.  

At the time (in Australia) there was no specific body representing opinion polling organisations. AMSRO is the closest thing we have in Australia to such an association (some, but by no means all, companies conducting political polling are AMSRO members).  It therefore fell to AMSRO and the SSA to examine the accuracy of the polls in the lead up to the election, determine the cause/s of any inaccuracies identified and suggest improvements to how polls are conducted and how their results are communicated to the general public.  In this way, it is hoped that the credibility of polling can be restored in the eyes of the Australian public.

Key to restoring credibility was an inquiry process that everyone can have confidence in.  In establishing the Polling Inquiry, AMSRO sought to proceed so as to maximise the prospects of achieving the ultimate aim of restoring credibility with polling and market research.  The view being that the inquiry process is impartial, robust, and comprehensive so that it can produce results that have credibility. 

On the Monday after the May 2019 election AMSRO announced that it would immediately commence an inquiry, with the initial tasks being to establish Terms of Reference and an Inquiry Panel.

On 7 June, AMSRO announced the formation of the Inquiry Panel and draft Terms of Reference.  The Terms of Reference and the make-up of the Inquiry Panel and Advisory Board are publicly available on AMSRO’s website:   

Further information is available here.

What is AMSRO’s role?

  • Appoint the Inquiry Panel and Advisory Board
  • Determine the Terms of Reference following feedback from the Advisory Board and Inquiry Panel, and other relevant stakeholders.
  • Encourage, promote and support the work of the review – provide (non-binding) comments on reporting, provide additional resources from AMSRO member companies as requested by the Inquiry Panel, reimburse travel and other reasonable expenses incurred
  • Once the Inquiry Panel provides its final report – to publish, promote and support the final report and its recommendations.
  • To field media queries during the Inquiry process and after the release of the report – so that the Inquiry Panel may conduct its work with a minimum of distraction.

 Who is on the Inquiry Panel?

The Inquiry Panel is comprised of technical specialists in polling, statistics, sampling, and related fields.  No individuals working for organisations currently conducting political polling in Australia are represented on the panel.  This is to provide the polling companies with the reassurance that they are not being judged by their competitors, as it is vital that they participate in the Inquiry.

Inquiry members

What the role of the inquiry panel?   

Darren Pennay is the Chair of the Inquiry, a task for which he is well qualified and equipped. Darren is the founder and immediate past-CEO of The Social Research Centre (SRC), part of the ANU, and one of Australia’s leading social research organisations.  Darren leads a panel of nine other members with relevant experience who are all highly regarded in their fields.  AMSRO is fortunate to have been able to access some of the best-qualified experts in polling and survey statistics, including international experts who worked on the Polling Inquiries undertaken in the wake of the 2015 UK General Election and the 2016 US Presidential Election.

Who governs the Inquiry?

The Inquiry will periodically report back to AMSRO on progress but will act independently of AMSRO.  The Inquiry Panel, not AMSRO, will determine what ends up in the final report and will determine the Inquiry’s recommendations.  The independence of the panel is important given that some polling companies are AMSRO members while others are not.

Why is there an Advisory Board?

AMSRO felt it prudent to establish an Advisory Board, whose role is to provide suggestions and advice to the Inquiry Panel.  The Advisory Board is comprised of senior figures with a range of relevant backgrounds.  They include experts in statistics, political science, commercial market research and the media.  We have also offered to Nine Media and News Corp that, as the main media stakeholders responsible for commissioning and supporting the polls and disseminating their results, they can have a representative each on the Advisory Board.

Who is on the Advisory Board?

AMSRO established an independent Advisory Board with diverse expertise in relevant areas for the Inquiry Panel to draw on for broad strategic guidance and input.

Inquiry members

The role of the Advisory Board is to: review and comment on Inquiry Terms of Reference; review and provide (non-binding) comments on Inquiry Panel reports (including preliminary reporting, if any); provide input to the Inquiry Panel on additional relevant lines of inquiry.

AMSRO also invited a Publisher representative from each of Nine Entertainment (Sydney Morning Herald/The Age) and NewsCorp to participate in an advisory capacity as part of a special Reference Group, however they declined.  Nine Entertainment offered to review and consult when the final report becomes available.   

 Why is there an interim Discussion Paper?

The Inquiry decided to prioritise the release of a Discussion Paper ahead of the Inquiry’s main report, so as to maximise the time available for AMSRO, the polling industry and other stakeholders to consider the report’s recommendations and take action before the next federal election polling cycle.

The Discussion Paper examines the following issues:

  • The existing codes of conduct/guidelines that apply to election polling in Australia;
  • International guidelines for election polling as well as examples of the disclosure standards in the USA, UK, Canada and New Zealand; and
  • Initial recommendations and issues for consideration by AMSRO.

What are the recommendations?

The key recommendation from the report is that a new and comprehensive minimum set of disclosure standards be adopted. The report lists 23 proposed standards, including some based on existing International Standard Organisations (ISO) and Australian Press Council (APC) standards and guidelines, as well as new recommendations.

The Inquiry Panel also recommends that any entity to mandate and regulate the recommendations should include a committee with industry, media and academic representatives (for example, political scientists), along with the polling companies and a chair who is independent from current polling organisations.  The Inquiry is keen for Australia to take a lead from the British Polling Council that has a sub-committee on Disclosure which has an independent Chair, academics and journalists among its members. The following is an extract from its website:

“The BPC is concerned only with polls and surveys that set out to measure the opinions of representative samples, for example the views of all adults, or all voters. Membership is limited to organisations who can show to the satisfaction of the BPC that the sampling methods and weighting procedures used are designed to accurately represent the views of all people within designated target groups (such as all adults, or voters etc.).  To these ends the members of the BPC will appoint as officers, journalists and academics with particular expertise in the field of polling to assist the BPC ensuring the highest standards are maintained at all times.”

What co-operation have we seen from the polling companies?

Ipsos, YouGov, Essential, and Lonergan have all participated with the Inquiry. The Inquiry panel is hoping for further input from the polling companies in explaining the methods they used in 2019 so that the final report can be as well-informed and comprehensive as possible.  It is hoped that by inviting comments on the Discussion Paper the panel can continue a dialogue with the polling companies which will enable them to produce a high-quality final report.

When was the Discussion Paper released?

An embargoed version of the final draft Discussion Paper was issued to the polling companies for comment on April 3 until April 17.  Ipsos, YouGov, and Essential responded.

The final Discussion Paper was released publicly on 20 May 2020 and there will now be a consultation period for interested parties that closes on 12 June 2020.

When will a final Report be released?

The target date for the delivery of the final report is October 2020.