Measuring the Impact of a Covid Vaccine Campaign. An Interview with Brennan Lake, Senior Director of Research Partnerships & Data for Good at Cuebiq

How effective is the Ad Council and COVID Collaborative Vaccine Education Initiative? Cuebiq, a company that measures mobility will find out. Cuebiq and the Ad Council are partnering to ascertain how effective the ‘It’s Up To You’ campaigns are. They were created to educate Americans about the COVID-19 vaccines. 

Using foot fall and other traffic data, Cuebiq will measure the effectiveness of the campaigns in driving exposed audiences to vaccination centers. Metrics will include visit lift, the change in percentage of visitation to vaccine centers between those exposed to PSAs versus an unexposed control group, and incrementality, which represents incremental visits to frequently visited locations (such as pharmacies) above and beyond normal behavior. Results are expected by the summer. Brennan Lake, Senior Director of Research Partnerships & Data for Good, offered further details:

Charlene Weisler: Has Cuebiq done this type of research before?

Brennan Lake: Measuring the effectiveness of media campaigns in driving visits to brick and mortar locations is at the core of Cuebiq's commercial business. Alongside this, we've done extensive pro bono work through our Data for Good program in using privacy-preserving mobility data for analyzing behavior change throughout the pandemic. Therefore, it was a natural move for us to combine our work in COVID-19 research with our advertising technology to assist the Ad Council in measuring the effectiveness of their vaccine information campaign.

Weisler: What is the methodology for this study?

Lake: We utilize our sample of first-party mobility data, consisting of over 20 million anonymous and opted-in daily active users, in order to measure visitation trends to public vaccination centers, such as pharmacies and vaccine mega sites. Specifically, we created privacy-preserving segments of users who have been exposed to the Ad Council's PSAs, and compare their visitation behaviors to an unexposed control group that is demographically similar, in order to measure the visit "Uplift" resulting from campaign exposure.

We also use machine learning to compare 'normal' visitation patterns to current behavior in order to draw a causal link between exposure to a PSA and visits to vaccination centers - a metric known as "Incrementality".

Weisler: What are the challenges that you see?

Lake: As with any other campaign, we want to make sure that we can effectively verify real visits to the points of interest, differentiating them from the noise of 'passersby' within the mobility data. When attempting to infer something such as vaccination from visits data, this adds a layer of complexity, especially for multi-purpose venues such as pharmacies, where even if we can confidently point to visits, it may be the result of segments engaging in regular shopping activities rather than to get a COVID-19 shot.

In addition to applying campaign parameters - such as only looking at visitors who dwell at a location between 15 minutes and 1 hour - we can also use causal machine learning to distinguish between normal shopping behavior and true incremental visits that can be attributed to a campaign. For example, if I typically visit a pharmacy 4 times a month, but after being exposed to a campaign I make 5 or 6 visits, our neural network can pick up on these discrete changes over time, along with data from a control group, in order to reveal incremental visitation behavior.

Weisler: What is the main purpose of the study?

Lake: The main purpose of this study is to support the Ad Council in evaluating the effectiveness of their campaign in driving positive public health outcomes. While the main call to action of the Ad Council's "It's Up To You" campaign is to learn more about the COVID-19 vaccine so that you can make an informed choice to vaccinate, we also hope to see an increased vaccine uptake, especially among cautious, hesitant and skeptical segments.

Weisler: What do you hope to find?

Lake: Ultimately we hope to see a campaign that is successful in providing vital information to the general public, while also driving positive health outcomes through an increase in COVID-19 vaccine uptake. By better understanding which aspects of the campaign - such as content, creative, channel, geographic focus - are most effective at driving visits to vaccination centers, decision makers can better double down on what works, to get people the information they need to make an informed decision to get vaccinated, in order to protect them and their loved ones.

This article first appeared in www.Mediapost.com