Measurement in an Age of Data and Fragmentation

Obviously cross platform measurement in all of its forms is a hot topic now as the de-accreditation of Nielsen’s local and national TV measurement services by the MRC along with a highly fragmented media environment creates uncertainty and challenge.

I was curious as to what the industry as a whole was thinking on the subject and reached out to the Research Wonks, a forum for those involved in media and advertising research, analytics and data science, last week. The response was so large that it required two articles on the subject – one appearing in this publication and the other in Mitch Oscar’s Hocus Focus newsletter.

Measurement Currency – Looking Back and Looking Forward

When they say that the past is prologue, they must be thinking of media measurement. We’ve had a measurement currency (and by currency I mean a standard metric that can be used as a comparison point between companies and platforms) since the beginning of media which has helped guide the valuation of inventory to this day. But as the industry fragments, it can leave us with a sense of uncertainty as the new solutions that best address the new environment have yet to be agreed upon.

For Jeff Boehme, Chief Executive, Executive Media Consultants, the future of measurement is as fragmented as the number of platforms. "I have observed over the past ten years the industry has evolved to create and accept multiple currencies for media planning and buying out of necessity,” he noted and added, “Going forward, successful media executions now must include metrics of performance beyond age and gender to measure outcomes defined outside traditional frameworks. I believe we will continue to embrace multiple currencies, already available through various walled gardens, but need transparency and audited standards to provide the necessary confidence for longer term use.”

Howard Shimmel, President, Janus Insights and Analysis, is concerned that we can easily go off track by focusing on the past while the future looms large. “I worry that we're distracted trying to solve for 2012 research needs- fixing linear TV currency measurement- when what we really need to be focused on is a future state integrated planning/activation/measurement research infrastructure to incorporates all video channels- linear, addressable linear, AVOD, CTV,” he warned. “Building that infrastructure is hard- requires panels, first party data from walled gardens, common target audience definitions across platform, ability to integrate RF forecasts for linear and addressable media, segmentation schema that reflects the way that media is consumed,” he added.

Media Currency – Getting to a Strong Place

There are those optimists, like Radha Subramanyam, President and Chief Research and Analytics Officer, CBS, who are ready for the measurement challenge. “The media measurement industry is ripe for real market transformation,” she informed, “There are now multiple providers of trusted and high-quality data measurement across video platforms, and our own data is critical as well. The measurement marketplace is undergoing massive diversification.”

Yet, the path to currency is still open to discussion. For Michael Vinson, Chief Research Officer, Comscore, the primacy of long standing panels that the industry has relied upon in the past may be up for discussion. “Recruited-sample panels can either be used as the foundational element of a measurement, or to add additional context to a measurement based on large transactional data sets,” he began. But, “When panel response/participation/cooperation rates are in the 10 to 20% range, we can no longer pretend that the panel is in any sense representative of the population. Moreover, media fragmentation puts severe pressure on the size of a panel, beyond what can be plausibly afforded. Therefore, panels should only be used to add context to currency measurements, which should instead be based on large-scale, passively-collected data sets.”

It stands to reason that the best way to formulate industry adoption of a media measurement currency is to gather interested experts together as a working group. Whether this is a consortium of industry organizations such as the MRC or something patterned along the lines of a European JIC, is up for discussion. As Andrew Brown of Andrew Brown Associates, noted, “All vendors no matter how good their data will need 3rd party verification. All players will need third party syndicated solutions to put their 1st party data in competitive context.”

This article first appeared in www.Mediapost.com

 

Is There a Future for Research? An Interview with Jeff Boehme

I have known media veteran, Jeff Boehme, from our days at NBC in the 1980s and since then, he has had a varied and interesting media career path. “I’m a veteran of local broadcast rep firms, NBC, ABC, NCC Media, Nielsen, Kantar Media, Rentrak and Comscore,” he explained where he concentrated on audience evaluations and processes for media currency acceptability. He has some strong opinions about where media is today and the role that research and data plays in it.

Charlene Weisler:  What role should data play in media today?

Jeff Boehme: Data always played a critical role in media. Content is now distributed on more types of technology than ever. Virtually all of these digital devices collect usage information and have been enabled in the marketplace by a multitude of companies. Content providers have taken advantage of technology by supplementing their traditional distribution infrastructure with streaming capabilities through over the top (OTT) platforms. Brand marketers realize the potential of reaching customers with far greater efficiency and effectiveness through addressable advertising across multiple platforms and content.

But defining the benefits of efficiency and effectiveness is not a standardized process; there are real issues surrounding the massive data sets collected from these digital devices and becoming ubiquitous as media currency. Ultimately data can and should be leveraged to maximize the effectiveness of the three basic pillars of brand advertising – creating awareness, reinforcing equity and driving purchases.

Weisler: What types of data are most important and what is currently missing?

Boehme: Over five years ago we understood the remarkable advantages of ‘big data’ expressed as the three V’s - volume, velocity, and variety. The sheer scale of anonymous, passively-collected user information provides much more statistically sound results than traditional small panels and surveys. However, most every big data set is incomplete and may not include essential data elements required for currency acceptance, making traditional tools still necessary to supply missing data points. I would add there should be a few more Vs to consider – the validation of the data (how accurate it is) and the ultimate V – its value. The value of the data ultimately answers the questions posed by the brand and can be accepted as currency on all sides of the ecosystem with confidence.

The good news is we now have more data than ever before - the bad news is that there are significant inconsistencies with the sources, collection techniques, methodology, standards, transparency and importantly – conclusions. All major cable MSOs are offering their tuning data to a variety of companies, as are virtually all connected TV (CTV) manufacturers. I have seen significant disparities on results depending on whom and how a company processes, manages, applies statistical corrections and matches census segments.

Weisler: Should age and gender still form the basis of currency?

Boehme: While age/gender metrics are still valuable criteria of value for brands and media, they have been supplemented with more relevant information including major census breaks and product usage. It was only in the late ‘70s when automotive brands finally looked at the data and revealed that women were the dominant influencer in car purchases. This transformed the industry in terms of understanding the real consumer, how to design new vehicles (think mini-van) and media investment placement strategies. Currency options now include actual auto ownership household impressions based on ‘auto intenders’ created by matching massive tuning and car ownership.

It really wasn’t until 1987 when Nielsen launched their people meter service that age/gender metrics became the de facto currency. However, many brand marketers learned that age/gender weren’t enough to efficiently plan or buy media – specifically for high spending categories such as automobiles. Most consumer purchaser data sets available today are household-specific and include information more relevant than just age/gender. Knowing that a household has pending lease expiration for a BMW is more valuable than simply counting adults 25-54.

Weisler: What is your opinion of the general state of attribution?

Boehme: Channeling Sergio Leone’s epic masterpiece western film “The Good, the Bad and the Ugly” - The Good is we now have a plethora of consumer-based intelligence and media companies are able to use attribution techniques to see a finer view of the customers’ behavior across screens and determine what components of media campaigns work (or don’t). The Bad is the complexity of data, multiple data sources, missing data points/deprecation and differing methodologies. The Ugly is there doesn’t appear to any consistent standards – resulting in significant outcome discrepancies.

Last year, CIMM completed a study on attribution which found the inconsistency of key television attribution inputs, not technology, is the main cause of variance in outcome measurements. They compared eleven different providers and determined, “more stringent media measurement standards are required to ensure attribution results that are consistent and comparable from provider to provider, with exposure data, more than occurrence data having the biggest impact on outcome results.”  I agree with their findings and with their report’s other recommendation requiring additional standardization, such as commercial IDs similar to Ad-ID, for identifying ad occurrences and in defining exposure and reach.

Weisler: What do you think is the most important issue facing Research at this time?

Boehme: Most research groups are a cost entry on a ledger, requiring investment without a direct responsibility for cash flow. Many successful researchers have learned to move quickly, adopt better data skill sets and provide actionable input into a sales process and discover how their company can be more profitable. Many companies see data scientists as a replacement for the research process but smart companies see the value of both, with complementary skill sets and valuable disciplines. The simplest distinction may be that the data scientist determines what could be accomplished with data and the researcher helps define what should be done with the data.

Weisler: Where do you see the Research function at media companies in the next five years?

Boehme: Data science has helped us improve our capabilities with disciplined scientific and technology-enabled approaches, beyond traditional research processes. However, Research is still a vitally imperative function as it is responsible for the objective analysis of the data with the clear communication of insights, business implications and recommendations. We have all witnessed the perils of utilizing large datasets without sufficient oversight in its contextual use case. Ultimately the most successful companies will discover research and data science are opposite sides of the coin – connected they bring greater value.

This article first appeared in www.Mediapost.com

 

 

The End of National TV and Local TV Measurement Siloes

There have always been separate local and national TV samples in television measurement—whether from Nielsen, Arbitron, or comScore. And, while the results of the two samples differed—weight-averaging local Nielsen NSI ratings don’t usually match national Nielsen NTI ratings—the industry accepted the separation of the two samples. Audience targeting is a powerful tool, but while its past has been siloed, its future must leverage local data into national.

Behind the Divide
So, you might be thinking: If the future is combined data, why the separation to begin with? The reasoning is baked into the history of television measurement. Back in the 1950s, radio measurement was fitted to the new technology of television. “Local samples were developed in the 1950s to serve a specific need, namely to measure 200+ individual markets in an affordable way (via paper diaries) as TV spread across the nation,” explains Tim Brooks, author and television historian. To fully compete with Arbitron, Nielsen launched a national sample of household meters, a more expensive venture than diaries (which were still used to supply national demographics), according to Brooks. As the technology advanced, people meters were introduced in 1987.

Read the full article on the Videa blog.

Insights, Meditations and Just Pure Happiness

I spent part of my week last week in the happiest place in the world – Disney in Orlando. I was not exactly in the Disney theme parks though. I was at the annual CTAM Insights Conference. Although no theme park, this year’s conference had a compelling theme – embracing change in research approach, methodology, marketing and even personal growth. 

In Sunday’s opening night dinner, Nat Geo Brain Games host Jason Silva rapped poetic about “Radical Openess.” Alternately described as a futurist, filmmaker, epiphany addict, ecstatic truth lover, techno optimist and performance philosopher, Jason is an evangelist of technology as a form of enlightenment.

According to his website, Jason believes that “the rapid, exponential growth rate of technological development is transforming our world, disrupting established industries and leading us towards a radical transformation in many key areas. There are three overlapping revolutions occurring in biotechnology, nanotechnology and artificial intelligence, all of which game-changers.” I see all three areas impacting how we conduct, analyze and receive research in the coming years. 

Jason’s frenetic talk on the subject set the stage for this year’s CTAM conference where speakers on neuroscience discussed how we can use bio metrics and eye tracking to monitor and gauge content.

Monday’s opening speaker, Shawn Achor from Good Think Inc, took us further along the path of corporate enlightenment. He spoke of a Happiness Advantage which is a modern take on the power of positive thinking. The basic premise is that “75% of our job success is predicted not by intelligence, but by your optimism, social support network and the ability to manage energy and stress in a positive way.” He offered daily exercises to raise our positive thinking for those of us who get bogged down in the stressful minutia of daily living. 

From there, much of the conference helped frame innovative ways to conduct and present research. ESPN’s Barry Blyn gave a welcomed presentation on the value of “research to know not research to show” which points out the intrinsic value of less-than-stellar research results as improvement tools. Negative research results from a study have enormous value – it highlights where things can be improved. Unfortunately there are some executives who only want to hear the good news so it is refreshing to hear that, at ESPN, “research to know” is treated as an opportunity and not as a report card.

A session on neuroscience, which included demonstration on biometric and eye tracking testing, demonstrated how the brain responds to content and how we can interpret these responses to form a more engaging viewer experience.  It could be the next step in content measurement. According to Laurie Kaman from EyeTrackShop,  “Eye tracking and other forms of biometric research will grow increasingly more important to advertisers as they seek to learn more about what consumers are actually seeing, how they are reacting to those stimuli and what they are taking away from those experiences.” 

So where do we think TV and research are headed in the next three years? What are the challenges / changes? I asked a group of attendees and their video’d responses are: 

Kaman believes that “the greatest challenge to television is and will continue to be finding creative, new ways to deliver integrated content that will be relevant to a variety of groups of consumers and that will work together across the multiple screens to further engage those audiences. It's going to become increasingly more important that consumers are able to interact in a meaningful way with that content and with the advertisers that are attached to the content.”

In looking at the industry environment, Cathy Hetzel of Rentrak summed it up: “I think the greatest challenge and also the greatest opportunity for television is ‘change’.  We are moving to a world of accountability and performance based metrics, including advanced demographics about the products that consumer use and the cars they drive.  The opportunity is to target and sell advertising in brand new ways, including branded entertainment, which for the first time can now be measured, but the industry has to embrace the change in order to capitalize on it!”