Measurement at any time is filled with challenges. There are
so many different datasets and parameters to use to quantify an increasingly
complex consumer journey.
Now, during a pandemic where the entire landscape is
shifting, the task becomes even more complicated. Konstantinos Spetsaris, SVP,
Analytic Partners believes that we can apply specific methodology at this time to
make greater sense for future forecasting. His company, he explained, offers, “adaptive
solutions (that) integrate proprietary technology powered by the latest data
science delivered through our platform and high-touch consulting.”
Charlene Weisler: What goes into accurate measurement of and during a
pandemic?
Konstantinos Spetsaris: With so many forces at play, a
holistic econometric model is best suited to accurately measure and decompose
the impact of Covid-19 and its compounding impact on other business drivers
such as media, operations and direct to consumer marketing. A simplified
formulation of an econometric response model where all controllable and
non-controllable drivers are in included as predictors (independent variables)
would look like Response=f(Marketing, Non-Marketing and Macro Factors). The
model lends itself to quantification and decomposition of impacts, reporting of
core performance metrics (ROI, cost per acquisition, response/unit of support
etc.) and scenario planning (simulation and optimization).
Weisler: How can you maintain quality data and feedback?
Spetsaris: We are expert at auditing, cleansing and
validating data elements to be analyzed.
Our ROI Genome, an integrated database of benchmarking metrics, enables
us to sense-check data inputs, which is complemented by a series of algorithmic
validations, business logic, and human checks. Our data audits score all model
inputs for both aggregate (accuracy, consistency, granularity, completeness)
and user level data (coverage, quality, detail, integration). We work with our clients to identify and
resolve data gaps by supplementing (i.e. augmenting the partner ecosystem) or
by creating proxies for missing data.
Weisler: Does measurement vary by industry, consumer category etc? If
so how?
Spetsaris: Measurement varies in the sense of which KPIs are
most critical to any given brand within any given vertical, as well as what
data is available per industry. For example, there are industries with an
immense amount of 1st party data, like financial services, which allows for
extreme deep dives. Conversely, in industries like CPG there is a lack of 1st
party data, which calls for a different process to draw out insights.
Based on the industry, type of data available and KPIs, we
can align candidate data inputs to be tested empirically in the model. Our
fully specified model includes marketing, non-marketing and known external
factors, but given the atypical and disruptive nature of Covid-19, we further
explore other indicators which may have a significant impact on sales. These
factors may provide additional insights into how changes in consumer behavior:
e.g. reduced mobility, increased online shopping impact business performance.
Inputs are rigorously tested on significance, independence (multicollinearity),
in and out of sample fit (for predictive strength) and data source sustainability.
Weisler: What data is most important?
Spetsaris: What data is most important really depends on
what business question is being asked. For that reason, it is critical to have
a holistic measurement system in place that allows available data to be viewed
through different lenses and dimensions, in order to extract the most relevant
answer.
A few
examples of the most important factors to consider during the Covid-19 crisis
may include:
> Bayesian
Causal Impact – measures a signal in sales itself and the difference beyond
expected response variable (synthetic baseline)
> Human
Mobility data to account for restricted movement based on government
stay-at-home orders
> Macroeconomic
Indicators such as consumer sentiment, consumer confidence etc.
> Financial
Indicators such as the VIX (Volatility Index) for financial services firms
> Store
Closings and Operational Changes in Services e.g. no longer offering dining in
for restaurants or adding a service e.g. curbside pick-up, changes in business
model B2B to B2C etc.
> Category
Base Sales to capture shifts in consumer demand towards certain product classes
e.g. disinfectants, cleaners, shelf stable food
> Scaled
Indicator Variables to capture Out of Stocks and the impact of stockpiling as a
result of the initial panic mode buying at the onset of the pandemic
> Covid-19
Incidence as a leading indicator to government orders to shelter in place etc.
> Google Query Volume for specific search terms such as
“lockdown”, “virus” etc.
Weisler: How can we effectively capture the impact of Covid-19 with so
many other forces at play?
Spetsaris: We recommend starting with a holistic measurement
framework such as Commercial Mix Modeling that incorporates controllable,
non-controllable, and macro-factors in order to isolate the impact of Covid-19
on the business. From a measurement perspective, there are several factors to
consider including: time horizon (immediate vs. longer term impact), industry
(benefiting or negatively influenced and to what extent) and unique brand /
business dynamics (% of sales impacted, geographic footprint, etc.). As an
initial analytical objective, we recommend beginning with descriptive data
analysis to help define the impact window in terms of business units, sales
channels, consumer segments, and time. The goal is to identify where the impact
of Covid-19 may be manifested in the dependent variable and gauge the order of
magnitude vs. expectation. This helps refine our search for the right data
inputs for Covid-19 as businesses are impacted differently.
Weisler: Can we still leverage historical results to predict outcomes
given this unprecedented, non-regular event?
Spetsaris: In a word, yes. Covid-19 has disrupted every
business in some capacity, which has influenced business and marketing plans
and forecasted performance. In this chaotic state, data and analytics become
even more important and measurement approaches must adapt. It is critical to
update existing models to reflect new consumer behavior and continuously
refresh to assess how these changes impact business performance. But, without
an accurate understanding of historical insights and principle based learning
as a foundation for these updates, there is no way to measure progress or
success – nor is there a way to understand when and if consumer behavior and
other key factors have returned to “normal.”
Weisler: How will we know the lagging impact of Covid-19 as we shift to
stabilization / recovery and revitalization phases?
Spetsaris: In the current environment, it is not enough to just
know how much a business has been impacted by Covid-19. Brands need to know how
the underlying consumer behavior has changed, and how it may continue to
change, and better understand the pandemic’s impact on marketing and media
channels, shopping habits, competitive actions, and overall business
performance. Given the disruptive nature of Covid-19 its epidemic and economic
consequences, decisions taken in the short term will have significant ripple
effects down the road. With a robust and holistic model framework in place that
incorporates all business drivers including the economic and Covid-19 impacts
detailed above, brands will have a foundation to monitor business performance
on an ongoing basis as we shift into stabilization, recovery and revitalization
phases.
This article first appeared in www.Mediapost.com
