August 28, 2018
This article was originally published in BusinessDay Newspaper, August 22, 2018
It was Falz the bahd guy who recently called out critics for “shooting down the messenger and missing the message”, as he responded to blacklash on his “This is Nigeria” video. Perhaps, this is new territory for Falz, but not so for social researchers and public opinion pollsters. We are often criticized for the results and findings of our polls; more so, by people who have neither read the research report nor understand the methodology adopted in conducting the poll, survey or research study. The rise or fall of any piece of research or public opinion poll lies in its methodology. In other words, what determines whether anyone should give a hoot about the findings of a poll or what confers credibility on any piece of research are in its ability to answer some questions – how was the poll conducted? how was primary data collected? how were the questions worded? how representative is the data? what’s the sample size of the poll? and what’s the margin of error, amongst other questions.
Public opinion polls are an essential part of politics, democracy, markets and social life in developed societies. From political predictions and approval ratings, to titles of Hollywood movies and song-of-the-week picks, polls are tools used to measure and gauge public opinion on issues affecting the society. Let’s take a step backward to define “public opinion”; it simply refers to the mind, thought and expression of the general population on a particular issue or subject matter.
Is public opinion polling new?
Certainly not, polling isn’t new. It was Jacques Necker, French statesman and Finance Minister under King Louis XVI, who in the 18th century, first emphasized the importance of L’opinion publique, as he advocated for the publishing of government accounts and budgets in order to boost public confidence in the years preceding the French Revolution. However, public opinion polling was popularized in America. It was George Gallup, founder of the renowned Gallup Poll and father of public opinion polling, who in the 1930s concluded that there was no difference between polling on toothpaste and politics. In recent times too, pollsters like Stanley Greenberg, have been hailed as the father of modern polling techniques and described as a man who doesn’t just have a finger on the people’s pulse, but has an IV injected into it.
How useful are polls?
For governance to be effective, it must be inclusive and participatory. Opinion polling is significant tool that can enhance inclusiveness and participation in governance. In the United States for instance, polling has become an entrenched part of democracy, as the need to understand what the public think is considered to be at the heart of governance. This disposition was aptly captured by Abraham Lincoln, who was quoted to have said “what I want to get done is what the people desire to have done, and the question for me is how to find that out exactly.” President Franklin Roosevelt relied on pollsters like Emil Hurja and Princeton professor Hadley Cantrill to shape strategies and public policy; same with President JF Kennedy. Likewise, President Ronald Reagan’s White House relied on polling support from Dr. Richard Wirthlin; while Presidents Bill Clinton and George W. Bush relied heavily on the polling gurus like Jeremy Rosner, Justin Wallin and Al Quinlan. Apart from the United States, there’s evidence to suggest that several global leaders and members of parliament (MPs) have in the recent decades relied on opinion research to help shape public policy and reforms in their countries – from Prime Ministers Tony Blair and Julia Gillard, to Presidents Nelson Mandela, Mikheil Saakashvili, and Viktor Yushchenko to mention a few.
Conducting scientific polls
Beyond media vox-pop and adhoc straw polls, which involve people responding to a set of questions without reference to how respondents are selected; there are certain indicators that can make a poll, survey or research study scientific. When we say a poll is “scientific”, it simply means the poll has been conducted through a set of rigorous processes and quality control mechanism to produce certain results; and that those processes can be replicated to produce about the same result, with some level of confidence and percentage of replicability. Several terminologies can be associated with polling, such as: Research population, Sampling technique, Sample size, Fielding and data collection, CAPI and PAPI, Questionnaire design, Question wording, margin of error, randomization and stratification amongst others. These would normally make up a full module in a survey research course, or an article for another day. My intention isn’t to make readers polling experts from this one article; but to help demystify the myth often associated with public opinion polls in Nigeria, and appreciate what makes good versus bad polls. Consequently, I would like to touch on a few of those terminologies that I consider fundamental to public opinion polling.
Research population
All polls are based on samples drawn from larger populations. The purpose of every opinion poll is to attempt to use a sample to make inferences or form conclusions on the larger population. With a population of between 180 and 200 million Nigerians, is it possible to access the entire population for a poll or survey? As far as social research is concerned, it is simply impracticable to have access to the entire 180 million Nigerians, unless you’re on a mission to conduct a census. Census studies don’t come cheap. Nigeria’s last census was conducted in 2006, and ought to be repeated every 10 years; however, due to financial constraints the country missed its 2016 census target and is still shopping for funds to conduct a fresh census. Notwithstanding, if the total population isn’t accessible, then we can rely on a research population. A research population is a well-defined collection of individuals or objects that have similar characteristics to the universe (total population), from which samples can be drawn for a poll. The next issue is then how the samples are drawn. Samples have to be selected randomly from the population. In other words, samples should be selected in such a way that every individual or object within the research population has an equal opportunity of being selected. This is called the principle of randomization.
The concept of sampling
As a pollster, one question I typically get asked borders on the issue of sample size – how can a mere 1,000 or 5,000 sample size poll tell us what over 180 million Nigerians are thinking on a particular issue? My constant response to those who ask is simple – it isn’t about the size of the sample, but the selection of the sample. As we often quip in polling parlance, when you fall ill and visit the hospital, the lab scientist doesn’t need to draw out your entire blood to check what’s wrong with you. He simply takes a tiny little blood sample for the test. This is the exact principle applied in public opinion polling. We mustn’t sample 1 Million Nigerians to know what Nigerians think on any matter. This concept has long been established in social sciences. In 1936, it was Gallup who demystified polling and dealt a major upset on the US presidential election prediction of Literary Digest. The magazine had sent out over 2 million dummy ballots from vehicle registration database and telephone directories, at considerable time and cost, in order to predict the election result. Gallup, on the other hand, used a sample of only 5,000 to predict that Roosevelt would take the lead in at least 40 states and carry the popular vote by 56 percent to 44 percent margin. Needless to say, it was a humiliating upset for Literary Digest, as Gallup succeeded in establishing that as far as survey sample size is concerned, more isn’t always better. Therefore, a scientifically selected sample of the general population was not only much cheaper and easier to handle, it would produce more accurate results.
Good vs Bad Polls
Without sounding academic, there are certain features that ought to be reported in poll reports to be ticked as good. Conversely, where they do not exist, the findings of such polls can simply be considered as bad polls. Organizations like American Association for Public Opinion Research (AAPOR) and the National Council for Public Polls (NCPP) have provided certain standards of disclosure to guide polling firms. While journalists and poll consumers may not be able to tell or assess the quality control criteria adopted on the poll; it is expected that good polls should provide information regarding: (1) Sampling (from what population was the sample selected? was data collected using a random or non-random sample? how many interviews were completed?), (2) Data collection (how was data collected? was data collected using face-to-face, telephone, web-based or SMS technique? (3) Question wording (were questions worded in a clear and neutral manner?), (4) Fielding dates (what were the specific dates of data collection?), and (5) Response rate (some sense of how many people were contacted and how many responded is always useful).
I find most poll consumers are unaware of these features, consequently poll results and reports are simply interpreted based on the reader’s premonition and worldview. In Nigeria for instance, politicians are quick to celebrate polls that seem to be in line with their predisposition, and shoot down polls that do not conform. They are partly responsible for the proliferation of fake or bad polls, which seek to support their political leanings. Interestingly, there’s usually a seasonal upsurge in opinion polling activities and report released during an election year; and as we build up to 2019 elections there would be many fly-by-night pollsters on the loose. The questions for discerning consumers of opinion poll reports should be – where have these pollsters been before now? Should we assume they are new or budding polling firms? And if they are, great! But would they continue their polling services after elections have come and gone?
So, when next you find yourself consuming the report from a public opinion poll, just before you hail it or shoot it down, depending on your socio-political leaning, be sure to spend a few more minutes to ascertain how it was conducted using some of the tips discussed above. That way, you would be better armed to judge for yourself if the poll should be given some attention or simply discarded as a mere academic exercise.
Dr. Bell Ihua is a public opinion polling expert and Chief Executive Officer at NOIPolls. He writes from Abuja.