Is There a Future for Research? An Interview with Jeff Boehme

I have known media veteran, Jeff Boehme, from our days at NBC in the 1980s and since then, he has had a varied and interesting media career path. “I’m a veteran of local broadcast rep firms, NBC, ABC, NCC Media, Nielsen, Kantar Media, Rentrak and Comscore,” he explained where he concentrated on audience evaluations and processes for media currency acceptability. He has some strong opinions about where media is today and the role that research and data plays in it.

Charlene Weisler:  What role should data play in media today?

Jeff Boehme: Data always played a critical role in media. Content is now distributed on more types of technology than ever. Virtually all of these digital devices collect usage information and have been enabled in the marketplace by a multitude of companies. Content providers have taken advantage of technology by supplementing their traditional distribution infrastructure with streaming capabilities through over the top (OTT) platforms. Brand marketers realize the potential of reaching customers with far greater efficiency and effectiveness through addressable advertising across multiple platforms and content.

But defining the benefits of efficiency and effectiveness is not a standardized process; there are real issues surrounding the massive data sets collected from these digital devices and becoming ubiquitous as media currency. Ultimately data can and should be leveraged to maximize the effectiveness of the three basic pillars of brand advertising – creating awareness, reinforcing equity and driving purchases.

Weisler: What types of data are most important and what is currently missing?

Boehme: Over five years ago we understood the remarkable advantages of ‘big data’ expressed as the three V’s - volume, velocity, and variety. The sheer scale of anonymous, passively-collected user information provides much more statistically sound results than traditional small panels and surveys. However, most every big data set is incomplete and may not include essential data elements required for currency acceptance, making traditional tools still necessary to supply missing data points. I would add there should be a few more Vs to consider – the validation of the data (how accurate it is) and the ultimate V – its value. The value of the data ultimately answers the questions posed by the brand and can be accepted as currency on all sides of the ecosystem with confidence.

The good news is we now have more data than ever before - the bad news is that there are significant inconsistencies with the sources, collection techniques, methodology, standards, transparency and importantly – conclusions. All major cable MSOs are offering their tuning data to a variety of companies, as are virtually all connected TV (CTV) manufacturers. I have seen significant disparities on results depending on whom and how a company processes, manages, applies statistical corrections and matches census segments.

Weisler: Should age and gender still form the basis of currency?

Boehme: While age/gender metrics are still valuable criteria of value for brands and media, they have been supplemented with more relevant information including major census breaks and product usage. It was only in the late ‘70s when automotive brands finally looked at the data and revealed that women were the dominant influencer in car purchases. This transformed the industry in terms of understanding the real consumer, how to design new vehicles (think mini-van) and media investment placement strategies. Currency options now include actual auto ownership household impressions based on ‘auto intenders’ created by matching massive tuning and car ownership.

It really wasn’t until 1987 when Nielsen launched their people meter service that age/gender metrics became the de facto currency. However, many brand marketers learned that age/gender weren’t enough to efficiently plan or buy media – specifically for high spending categories such as automobiles. Most consumer purchaser data sets available today are household-specific and include information more relevant than just age/gender. Knowing that a household has pending lease expiration for a BMW is more valuable than simply counting adults 25-54.

Weisler: What is your opinion of the general state of attribution?

Boehme: Channeling Sergio Leone’s epic masterpiece western film “The Good, the Bad and the Ugly” - The Good is we now have a plethora of consumer-based intelligence and media companies are able to use attribution techniques to see a finer view of the customers’ behavior across screens and determine what components of media campaigns work (or don’t). The Bad is the complexity of data, multiple data sources, missing data points/deprecation and differing methodologies. The Ugly is there doesn’t appear to any consistent standards – resulting in significant outcome discrepancies.

Last year, CIMM completed a study on attribution which found the inconsistency of key television attribution inputs, not technology, is the main cause of variance in outcome measurements. They compared eleven different providers and determined, “more stringent media measurement standards are required to ensure attribution results that are consistent and comparable from provider to provider, with exposure data, more than occurrence data having the biggest impact on outcome results.”  I agree with their findings and with their report’s other recommendation requiring additional standardization, such as commercial IDs similar to Ad-ID, for identifying ad occurrences and in defining exposure and reach.

Weisler: What do you think is the most important issue facing Research at this time?

Boehme: Most research groups are a cost entry on a ledger, requiring investment without a direct responsibility for cash flow. Many successful researchers have learned to move quickly, adopt better data skill sets and provide actionable input into a sales process and discover how their company can be more profitable. Many companies see data scientists as a replacement for the research process but smart companies see the value of both, with complementary skill sets and valuable disciplines. The simplest distinction may be that the data scientist determines what could be accomplished with data and the researcher helps define what should be done with the data.

Weisler: Where do you see the Research function at media companies in the next five years?

Boehme: Data science has helped us improve our capabilities with disciplined scientific and technology-enabled approaches, beyond traditional research processes. However, Research is still a vitally imperative function as it is responsible for the objective analysis of the data with the clear communication of insights, business implications and recommendations. We have all witnessed the perils of utilizing large datasets without sufficient oversight in its contextual use case. Ultimately the most successful companies will discover research and data science are opposite sides of the coin – connected they bring greater value.

This article first appeared in www.Mediapost.com

 

 

Navigating the Data in the Linear and Addressable Landscape. An Interview with Dish’s Data Scientist, Camille Pickren

Camille Pickren, is on a career trajectory since joining Dish in 2016 moving from entry level to senior to lead data scientist to now the Business Operations Manager of Data Science. Her work takes her into the deep mechanics of data targeting for marketing, programming and sales. 

“As a team, we are in charge of all of the propensity modeling. So if they decide ‘this is the kind of audience we want to reach,’ we run a model and score all of the households with most presence of those most likely to want to watch your show,” she explained. But in addition to that, her team also does direct targeting and various post campaign reports. “We have a custom linear product, addressable reports and most recently, the Reach Booster,’ that analyzes how a campaign did in linear and then adding addressable ads to those homes that missed the linear messaging. 

Introducing Reach Booster

One of the challenges is that no matter how much you may spend in a linear campaign, you always tend to max out at 70% of the audience, leaving a substantial audience still to be exposed to the messaging with the right amount of frequency. How to best reach this additional 30%? “The idea was to find these linear ad schedules and find everybody who had seen the ad and then target those who didn’t.” This elegant solution enables advertisers to not only maximize their reach but also attain the correct amount of frequency. “We’ve run several beta campaigns at this point and they have been really successful,” she added, “reaching people who are not seeing linear campaigns.” Using Reach Booster, Pickren was able to increase reach to over 90%.

Finding the Best Data Points

In a world awash with granular first party data, we are faced with an embarrassment of data riches in trying to choose which sets are best for a particular sales campaign. “We have thousands,” she began, “People tend to stick with the traditional ways that linear was designed. But I have seen that slowly starting to shift.” So while possibly starting with age and gender, “people are starting to realize how many data points we have,” and are drilling deeper, leading to a far better advanced target.  A recent request started as, “we want men 18-45 who made over $75k and also had more than an acre of land,” resulting in a very adequate sample size of 200,000 households for an advertiser of farming equipment. This highly targeted data approach insured that there would be as little waste as possible. “There are so many datasets that it’s a little overwhelming for people to decide which things they want and don’t want,” she noted, but they are getting more sophisticated over time. 

Insuring Privacy

When asked about the biggest challenges facing the data business today, the CCPA looms large in both privacy and licensing. But, “Dish, very early on, formed an entire board – a privacy team – dedicated to privacy compliance. We are very protective of our data, more so than industry standard, so we haven’t had to change a lot,” to comply with the new privacy legislations in the U.S. and across the world, she explained. “So far what we have done is put into place process so that if someone calls and says, ‘I don’t want to be targeted’ we are ready for that.” Further, none of the data points that Pickren can access is connected to a name or an address. It is all anonymized, even internally. “Different teams have access to different things and no one person can put it all together.”

Education

This respect for the customer coupled with both linear and addressable advertising opportunities for advertisers places Dish in a strong position within the media ecosystem. And in addition to external sales efforts, Dish also focuses internally, educating the sales executives on all of the different choices of data available and how the data can be best used for an advertising campaign. “We have a lot of new people coming into the company and they may not know what data we have available,” she explained, “So it takes time.” Balancing both external and internal constituencies is both an art and a data science for Pickren.

 This article first appeared in www.MediaVillage.com

NCS's Leslie Wood on Monitoring Campaigns Using AI Causality

High on the wish list of most marketers is improving the measurement of campaign effectiveness. Fortunately, Leslie Wood, NCS's chief research officer, has been on a mission to perfect it. She has accomplished this with a combination of machine learning methodologies that interface with a campaign in flight, thus enabling marketers to modify and improve their ad placements in real time. This initiative, called Sales Lift Metrics, sheds light on the causal impact of specific campaign tactics by identifying the key sales drivers that amplify incremental sales for CPG brands.

"We have created a Super Learner, which includes multiple different machine learning methodologies, like 'random forest' and 'gradient boost,'" Wood said. "And when we execute the Super Learner, it runs all of those different models and sees which ones are winners — often up to three or four of them — and combines them into an ensemble model." It is apparently flexible enough to apply to any type of CPG advertiser and is constantly refining its application through the use of AI.

How It All Began
The idea and the efforts behind this initiative started about five years ago and, over that span of time, the methodology has evolved through testing and innovation. "When we first started, it was just television because you can't have a control if you have such high reach. After all, who is not exposed? People who were not exposed to the advertising look very different than TV viewers. They are very different kinds of people," she explained.

Wood discovered that this approach took the pressure off the researcher to constantly oversee the model. "Here was this machine learning method that allowed us to put this in places where data was siloed. We could set it up in 'free rooms,' places where there was no researcher," she said, adding that this allowed a door to open where Google, Facebook, and Apple could manage their proprietary datasets without a human interacting or overseeing it. "It is meant for computers to talk to computers."

In-Flight Optimization
Five years ago, the hardware just wasn't there to be able to apply in-flight optimization. But now, this can be executed on what Wood calls, "sales-lift metrics," which provides the ability to "use these machine learnings to quickly, in real time, deliver key performance indicators to tell you what about your campaign is working." It is now possible to compare multiple creative options, consumer targets, or sets of tactics for sales effectiveness and incremental causality.
"We are looking at the incremental value of each of them," she said, "and compare them" in a weekly cumulative sales report. An advertiser can make changes to the campaign as it rolls out and see the impact.

What Sets the Super Learner Apart?
"There is always a full set of statistics" within the Super Learner, Wood noted. This is unusual because "AI almost never has a full suite of statistics. Statistics were developed by professors on small data sets. For big data sets, we develop machine learning and AI methods and those were built by computer programmers. So, we don't usually have confidence intervals, significance tests, or standard error — those measures that you need to say that this is a good model. We built Super Learner, which does all of these steps."

In this way, as new machine learning methods are developed (and, according to Wood, they are being developed all the time), they are added to the system to see if there are new ways to improve campaign efficiency. "It allows us to continually improve without reprogramming or recoding," she said, adding that certain models never worked and others that worked too hard and delivered unreasonable answers. "We removed those."

Among the successful campaign indicators is incremental sales per household, which is a key performance metric. It also reports impressions delivery by target and by creative.

Going Forward
Since the Super Learner has the ability to integrate faster systems and new models, it advances every day. Thousands of new schedules have been run and rerun "to see what works and what doesn't work, what's the minimum threshold, what's feasible, what are indicators inside the statistics that let us know that it's not working."

Wood is committed to keeping up with "this fast-moving world" of data and machine learning with constant monitoring and asking why results change. In the fall, they will be adding more data, more brands, and a price/promotion feature.

And, according to Wood, this entire initiative has been a model for others. "We use the h2o machine learning language; they will tell you that we were the first to create a Super Learner and they followed suit and built one themselves," she said. "We have pushed the envelope on machine learning."

 This article first appeared in www.MediaVillage.com

The World of Outstream Video. Interview with Yoav Naveh

Yoav Naveh has a background steeped in building predictive systems for online advertising and internet security startups. Prior to launching ConvertMedia, he served as Captain in the elite technological unit of the Israeli Intelligence Corp and received multiple awards of achievement while there. 

Upon completing his service, Naveh went on to receive a degree in mathematics and computer science at Tel Aviv University. As a self-described expert in the programmatic industry, Naveh also has a passion for machine learning and big data.

As the co-founder and CEO of ConvertMedia, Naveh works in the world of outstream video which are, according to Cynopsis, video ad units that aren’t tied to content. (An out-stream ad can run between paragraphs of text, on the side of a page. They are presumed to be more valuable because they can guarantee 100% viewability.) ConvertMedia is an outstream video SSP that, according to Naveh, “enables publishers to strike an ideal balance between revenue goals, the exposure they afford advertisers and how they engage consumers.”

I sat down with him and asked him the following questions:

Charlene Weisler: What exactly is an outstream video SSP?

Yoav Naveh: SSPs have traditionally helped publishers connect to demand channels (advertisers via DSPs and ad networks) and sell real estate. An Outstream SSP makes that connection for outstream video supply. We believe that SSPs should evolve beyond demand management and offer user experience controls. In outstream, a high impact and effective format, publishers need to have tools to manage the user experience in a way that is respectful to the user, and measures not only the potential revenue, but also the user experience tradeoff.

Charlene Weisler: What type of video – length, origin etc, do you work with?

Yoav Naveh: Most ads are 15-30 seconds long. But we feel that outstream can be an opportunity for content marketers to play longer stories within outstream formats. This allows marketers to have a few seconds of high impact format that then stays active on the page only if users engage. If the user does engage, the marketer can now play a significantly longer story/ad (2-5 minutes).  This brings publishers the opportunity to deliver an experience similar to what Facebook created with in-feed with video.

Charlene Weisler: What is your definition of television?

Yoav Naveh : Television is professionally created video content that is made available for a mass audience through broadcasting or streaming

Charlene Weisler: How are you able to move TV dollars to digital?

Yoav Naveh: Advertisers are looking to reach their audiences, who are online, and in particular on their mobile devices. But they need to be able to reach those audiences at scale and in a viewable and reliable way if they are going to make the move from broadcast TV to digital. Outstream video opens up quality, viewable inventory at scale and will be the key to moving budgets so advertisers can be where their audiences already engage with content.

Charlene Weisler: Tell me about some of the data you collect and how you use it?

Yoav Naveh: We collect data on the effects of different video ad formats on the user experience and how it correlates with revenue to help publishers find the balance between revenue and user experience. We measure the load-time of the ads, the time the user spends on a page, how they move and interact with the page, where they came to the site from (i.e., whether it was from social media or organically) and if they immediately leave after the page visit. We also allow users to provide direct feedback on the ad with an opt-out option.

Charlene Weisler: What is your definition of programmatic?

Yoav Naveh: Programmatic is the automatic method of buying and selling digital media

Charlene Weisler: What happens when Smart TV’s gain critical mass? How will that impact your business?

Yoav Naveh: Smart TVs present an opportunity to create new video advertising experiences for users. We will need to develop smart and relevant ad experiences that enhance the user experience that such devices introduce beyond the pre-roll.