WHAT THE NUMBERS SAY … OR DON’T
Surveys have room for improvement when it comes to credibility
Every day we hear the same thing: Americans believe this, or that, or something in-between.
Recently, yet another poll was released suggesting President Obama is considered by many to be one of the worst presidents in history. Really? There are many ways to arrive at that conclusion, but personally, I don’t believe Obama is the worst president we’ve had. I lived through Jimmy Carter and paid 16 percent interest on my Chevy Chevette loan during his presidency. That was tough — I was making $11,690 per year at the time.
Many Americans are deeply suspicious of polls no matter who does them. After all, a scientific poll is different that an unscientific poll. At our agency, we do a lot of scientific polls, complete with superior methodologies and impartiality. We do both qualitative (interviews, focus groups) and quantitative (numbers) surveys. Unscientific polls are not really worth much given the notion they are not representative of the entire population to be surveyed.
In public relations, we have a saying: “You want to make news? Do a survey.†You’ve seen these results. “Top Ten cars according to motorsports enthusiasts.†“Top-ranked hamburgers in America†(I heard that one on the way into work today). I believe (this is not research but my opinion) you can end up with just about any result if you manipulate the methodology enough. However, this is a bad idea and not an ethical approach to getting to the truth.
Since we are approaching a midterm election, we’re going to be seeing a lot of polls. Some of these will be diametrically opposed to each other. Recent examples: “Obamacare is working well,†“Obamacare isn’t working well.†How are these totally different outcomes reached? Well, it has a lot to do with the methodology I spoke of earlier. Who was surveyed? Everyone? Eligible voters only? Democrats, Republicans, independents or all three?
In addition, it’s possible that the questions were phrased differently, thereby resulting in different answers. For example: “Do you believe Obamacare benefits America?†or “Do you believe Obamacare benefits people who have had trouble finding health insurance?†Both questions ask about the effectiveness of Obamacare, though they might elicit totally different answers.
A number of years ago, I was dealing with a large university who, surprisingly, was voted a Top Ten Party School, even though it had never appeared among listings of Top 100 party schools previously. The administration was outraged and rightfully so. As it turned out, the survey was given to the student population immediately following the university’s appearance in the final game of the NCAA Final Four basketball tourney. The timing was perfect to reflect the party mentality of the student body — before they had to get back to the business of studying after two full weeks of partying.
No matter what you think, polls and surveys have high news value, whether they are true or not!