Media Researchers: Politics Ruins Everything, Including Media Research

Media Researchers: Politics Ruins Everything, Including Media Research

by Joe Mandese  @mp_joemandese,

Public perception about the inaccuracy of political polling during the 2016 and 2020 presidential elections has contributed to a crisis of confidence for the survey-based research industry, especially media researchers, a roundtable of industry experts said Monday.

The roundtable, organized by media research vets Tim Brooks and Charlene Weisler as part of an ongoing series of “media insights salons,” can be viewed in its entirety below, but the consensus was that negative attention surrounding the accuracy of political polling, has exacerbated concerns about media research in general.

Coming at a time when many in marketing and media already have begun moving away from survey-based research in favor of data science, as well as concerns that the efficacy of surveys has also deteriorated due to changes in the way surveys are conducted, especially the volume and ubiquity of them, the experts said.

“If we’ve learned anything over the past couple of years, it is that politics ruins everything,” quipped Betsy Frank, former research chief at Time Inc., Viacom, Zenith and other organizations, acknowledging that public perceptions have been tarnished. She blamed the news media, in part, for their coverage of political polling pre- and post the elections, but also acknowledged that the environment surrounding survey-based research has fundamentally changed, and that other methods -- including observational methods, or biometric techniques -- may be part of a longer term solution for measuring media.

“What’s happened now is the cynicism that we’ve seen in politics now has spilled over,” concurred Jack Wakshlag, former head of research at Turner, WB, CBS and currently an advisor to Simulmedia, adding, “There’s a decline in trust.”

Wakshlag noted there have always been issues with how survey-based research has been conducted, especially ensuring that they used proper sample sizes and representative ones, but he implied that many organizations have been let standards slide and that has contributed to a loss of confidence about research science overall.

He suggested some of the problems with sample representation -- including political polls -- could be offset by utilizing sample weighting techniques to compensate for the under-representation of key segments. That is one of the factors many researchers attributed to the inaccuracy of political polls in 2016 and 2020, because so-called “shy Trump supporters” may have been reluctant to participate in polls, or if they did, to give accurate answers about their voting plans.

Wakshlag noted that weighting has been an integral part of media research for decades, including panel-based research such as Nielsen’s, which has used weights to adjust for the under-representation of certain demographics, especially hard to reach ones.

Brooks, former research chief at Lifetime, USA Networks, and legendary ad agency N.W. Ayer, added that part of the problem with survey research is that methods have changed from the simple days of diary-based panel surveys and random-digital-dialing telephone surveys to online polls, which virtually anyone and everyone can conduct.

That latter point was something Paul Donato, Chief Research Officer of the Advertising Research Foundation, and Nielsen before that, said may be negatively impacting the quality of survey research, noting that many consumers -- especially those who were stuck at home during the COVID-19 pandemic -- may have become professional survey-takers in order to make a little money on the side from researchers and pollsters.

Donato said he doesn’t have any hard numbers to say exactly how much the survey universe has grown, but he noted that “response rates” for many forms of formal industry research soared during the pandemic, which is an indication that consumer behavior changed -- at least for a period of time -- during the pandemic.

Long-term, Donato suggested the best possible method of the media and marketing research industry might be a combination of techniques, including well-conceived and maintained panels that can control for the representation of various consumers, as well as data analytic methods that could be used to benchmark, and adjust each other.

He described this as a “proper balance between a well-curated panel and machine-curated data, and said that utilizing the two processes simultaneously would lead to the best “scientific combination” of research and data science.

 

Polling for Accuracy in the Census. An Interview with Steve Edgeller

As someone who is conducting genealogical research, I am aware of the discrepancies in past census records. Further, with the focus on the accuracy of recent political polls and the politicization of the recent census, especially during the pandemic, I was curious to know how challenging the data collection process was for a census taker.  

Steve Edgeller, who has worked with population data in his career, appreciates the importance of having a comprehensive and reliable census base for population/household demographics to inform his work in urban development. He was committed to working the census because, he explained, “Running my own business gave me the flexibility to work afternoons, evenings and weekends, which were the ideal times to catch people on their doorstep. The census period only lasted a couple of months, which gave me the opportunity to make some pocket money ($20 an hour).”

Charlene Weisler: Was this your first time working the census?

Steve Edgeller: This was my first time but a lot of my co-workers in the field were doing it for the second, third or fourth times, some well beyond their normal retirement age.

Weisler: Where did you work - what state, areas etc?

Edgeller: Most of my casework was located within half an hour drive from home in northern New Jersey, some in my own street!  I did need a car to get to the locations, and in most cases, to drive to the next address - which weren’t always next to each other. Some of my co-workers in city centers managed to do theirs on foot or by bus, especially when they were surveying whole apartment blocks at once.

As the census deadline approached however we were offered the chance to work out of state, and I joined a group of around 30 people who were flown from New York to Columbus, Georgia where there was a massive undercount that they needed to resolve. We spent three weeks in a hotel with rental cars, taking it from around 20% response up to nearly 80% response.

Weisler: What were you asked to do and how difficult was it to achieve at this time?

Edgeller: The job entailed knocking at predetermined addresses, where there had been no response received to date, and asking a household member to complete the full survey with me. I asked the questions and entered the answers straight into a smartphone app. They equipped me with an iPhone and language identification cards to use if English was not the first language. A simple survey could be completed in a couple of minutes, but it took a lot longer if there were more occupiers, children, relatives etc. Sometimes 15 minutes or more.

Weisler:  What data did you need to collect?            

Edgeller: The data collection was straightforward - name, date of birth, origin/race and relationship to other members of the household. There was a follow-up survey in some cases which went into employment, income, disability etc., although I wasn’t involved with that stage.

Weisler:  How did covid impact your work ... or not?

Edgeller: The Covid pandemic delayed the start of enumeration while new protocols were developed. It was a 100% no contact process though, and we were required to wear a mask at all times. We certainly couldn’t let respondents enter their names/data into the smartphone app, and it often took some time to ensure correct spelling. They did trial the option of enumerating by telephone, but this didn’t work as the numbers we were given were never accurate or up-to-date.

Weisler:  Did you see a pattern as to who cooperates and who doesn't? If so, what is the pattern?

Edgeller: I found on average that 50% of respondents were open and willing, or even insistent to ensure the survey was completed. The other 50% either didn’t answer (when you knew someone was home) or would refuse, often forcefully or with prejudice. The patterns weren’t related to age, gender, race or income, and I had some heartwarming interviews with busy single parents, people with very limited English and the elderly.  In others, there was deep distrust of the process, the government or just peoples' civil liberty.  Some people were polite and said they had already completed the survey (when I suspected they hadn’t) but others were extraordinarily rude or confrontational. We were encouraged to mark these respondents as dangerous to avoid repeated encounters by future enumerators.  Some of my co-workers were threatened with guns.

Weisler:  Do you think you were able to get truthful responses from people?

Edgeller: There were some people who were clearly reluctant to explain how many household members were staying there, particularly if they might not have been lawful residents. On the whole if someone was going to submit to the questions, they seemed to give truthful responses usually without hesitation.

Weisler:  Do you think your experiences were the same or different from other census workers?

Edgeller: We did sometimes meet our co-workers, and we were encouraged to participate in text based chat rooms to share (non-PII) information, experience and hints/tips.  In Georgia, we were staying in the same hotel and had team meetings several times a week, during which it was clear that everybody had very similar experiences, particularly with refusals to respond.

Weisler:  Do you think that you were able to achieve the most accurate information for an accurate census?

Edgeller: It wasn’t always easy to ensure information was 100% accurate, as people sometimes skipped family members if they thought there were too many to list. We were encouraged to at least record the total number of occupiers, even if someone refused to give names and ages etc.  Sometimes it could be difficult to complete the survey if somebody had moved since the census date (1st April 2020).

Fortunately, I had a lot of single family addresses which were easy to enumerate however some of my co-workers had multi-family apartment blocks or even areas with a significant homeless population.

It was very difficult to get into gated communities or secure apartment blocks without speaking to a superintendent or building manager.

Weisler: Did you get any feedback as to how the data looks?

Edgeller: As an enumerator our job was simply to obtain the data, which was very, very carefully controlled and treated as personally identifiable information (PII).  Retaining or writing down PII was a dismissible offence, and the smartphone app would transmit the data as soon as the interview had ended, it couldn’t be re-read, altered or amended. I enjoyed my time as an enumerator although it could involve long hours, and intense work on hot days. Despite all the knockbacks, I met some wonderful people who willingly shared their time, their stories and their information which made it all worthwhile. Roll on 2031!

This article first appeared in www.Mediapost.com