Getting into the Music Moods. An Interview with Eyal Golshani, Senior Director, Data Science, Vevo

It is thought that music can elicit certain moods or reflect the listeners’ moods. So a music-oriented company like Vevo might be able glean listener receptivity from the music they choose. Or so the theory goes. Eyal Golshani, Vevo’s Senior Director Data Science, set out to prove the power of music with Moods. 

“This is a reflection of the human story, providing us with color around how and why we consume certain content. This is especially true with artistic content, such as music videos, that particularly convey emotions. When we interact with music videos, we do so collectively, often in response to viral or cyclical moments,” he explained.

 

Charlene Weisler: What are the essential elements of Moods?

Eyal Golshani: We know that we turn to music videos to soundtrack a particular frame of mind or moment at a given point. Therefore, Moods serves to provide music fans with the most relevant viewership experience for how they are feeling at the moment -- whether it be the look, feel and narrative elements of a music video or experiencing relevant ads. From a technical perspective, there are two key elements in determining a mood: positivity (from sad to happy) and energy (from low to high energy). Each element has a range, so we can then create multiple combinations that each capture the varying degrees of these two elements. 

Weisler: What made you think of this?

Golshani: As we enter a cookie-free world, the industry overall is moving towards contextual targeting. Naturally, we wanted to provide advertisers with a proprietary option to do the same when it comes to advertising alongside music videos.  Additionally, we always want to make sure our content is relevant and engaging for each of the billions of music fans that watch Vevo. With Moods, we are not only expanding our advertising offerings, but also adding an additional dimension to the way we catalog our library, which helps us curate programming. 

 

Weisler: What is the impact for an advertiser matching their ad mood to the music?

Golshani: Ultimately, congruence between advertisements and content significantly increases the likeliness of viewers remembering the ad. Additionally, advertisers can select content that matches a brand’s image and that resonates with their target demographic. Beyond demographics and volume, it’s highly important that a brand can reach their audience in a way that’s consistent with their values and identity as a company or support the theme of a particular campaign. Vevo’s Moods product enables them to do that much more easily and at a much larger scale than ever before.

Weisler: What data is used and what is the methodology?

Golshani: Musixmatch provides us with data on the energy levels and positivity of the songs themselves. The energy and positivity values are collected from Musixmatch's proprietary crowdsourcing method - from their community of more than 50 million music curators and fans - which is then used to train an AI/ML model. Vevo then analyzes the energy and positivity values and clusters them together to find groups of similar videos. This allows us to figure out how videos sit in relation to each other -- as well as artists or genres -- based on the way they make our listeners feel. 

Weisler: How is AI involved?

Golshani: Musixmatch’s AI team has developed a cutting edge system to identify the general mood of a song, called Music Emotion Recognition (MER), which quantifies the positivity and energy of the song. The data gathered train the Deep Neural Networks models at predicting the energy/positivity levels of a sound recording.  On top of Musixmatch’s MER, Vevo’s clustering model takes the inputs from MER and finds combinations that correspond to mood types. 

 

Weisler: Is it possible to match the moods from music video to viewer? How can you tell their mood?

Golshani: We do not collect nor have individual viewer data. Moods works the other way around by examining the music and lyrics themselves, which is a non-intrusive approach.

 

Weisler: How do moods work in the US and how globally - are there differences in music mood by country?

Golshani: Local events and developments can impact viewing behavior, so we can see differences by market. Again, interactions with music videos tend to occur collectively, so we can identify trends at scale. When Joe Biden was announced the winner of the presidential election on Saturday, November 7, for example, we saw a spike in viewership around ‘fun’ and ‘impassioned’ music videos as people celebrated in the US. 

Weisler: Is there a case study you could share? 

Golshani: In addition to trends related to the presidential election, we saw some interesting trends around the Georgia runoffs and the Capitol insurrection.  For example, on Election Day, we saw an increase in viewership of impassioned music videos, such as The Black Eyed Peas’ “Where Is The Love?” and John Mayer’s “Waiting On the World to Change.” The next day, however, we saw a decrease in impassioned music videos, as votes were still being counted and there was lingering uncertainty. Instead, people began tuning to more heartfelt music videos in anticipation, such as Ray Charles’ “Georgia On My Mind” and Bruce Springsteen’s “Streets of Philadelphia”. Unsurprisingly, these songs also relate to Georgia and Pennsylvania which were going through recounts at the time.  

In January, we saw similar trends around the Georgia elections. As Georgia held its election, heartfelt music videos, such as Sam Cooke’s “A Change Is Gonna Come”, saw increasing viewership. Once the Georgia election was decided, we saw a spike in fun music videos, including Ludacris’ “Georgia” and Childish Gambino’s “This Is America”. The insurrection in DC right after drove steady viewership of impassioned music videos, such as Michael Jackson’s “Scream” and No Doubt’s “Just a Girl”. 

During the inauguration, we saw spikes in music video viewership overall, which is not surprising as people often turn to music videos during pivotal moments. Again, the biggest spikes were around fun and impassioned music videos. Some examples of fun music videos watched during this time were Beyoncé’s “Run the World (Girls)” and New Radicals’ “You Get What You Give”, while impassioned music videos include Bruce Springsteen’s “Born in the U.S.A. and *NSYNC’s “Bye Bye Bye”. We also saw a bump in empowering music videos like U2’s “Beautiful Day” and The Beatles’ “Here Comes The Sun” and heartfelt music videos, such as Florence + The Machine’s “Dog Days Are Over”. As you see by the titles of these videos, the attitudes of viewers on that day are clearly reflected!

Weisler: What are your next steps for Moods?

Golshani:  We are expanding our Moods product in a few ways.

Firstly, we are growing the number of music videos being tagged with a mood, as well as developing other types of moods. This will give us a more complete picture and better intelligence on the Vevo catalog, so that we can create more personalized programming in the future and drive increased value to our advertisers and great levels of monetization for our content partners and artists. 

Secondly, we are looking to expand Moods to also include Spanish language content.

Lastly, we are expanding our analysis from lyrics and audio to also include the visual components of music videos. By breaking these components down, we will be able to better understand the complexities of music videos and provide a truly advanced and nuanced experience.  

Moreover, we are layering Moods with our Moments product to identify viewership spikes around cyclical and viral moments to understand how people are reacting to events via the music they're consuming. Together, Moods and Moments can be a useful predictive tool and can decipher whether viewership trends reflect unique moments (such as a news event or an artist’s passing) or regularly recurring events (such as a holiday, a season or cultural event). 

This article first appeared in www.Mediapost.com

 

NCS's Leslie Wood on Monitoring Campaigns Using AI Causality

High on the wish list of most marketers is improving the measurement of campaign effectiveness. Fortunately, Leslie Wood, NCS's chief research officer, has been on a mission to perfect it. She has accomplished this with a combination of machine learning methodologies that interface with a campaign in flight, thus enabling marketers to modify and improve their ad placements in real time. This initiative, called Sales Lift Metrics, sheds light on the causal impact of specific campaign tactics by identifying the key sales drivers that amplify incremental sales for CPG brands.

"We have created a Super Learner, which includes multiple different machine learning methodologies, like 'random forest' and 'gradient boost,'" Wood said. "And when we execute the Super Learner, it runs all of those different models and sees which ones are winners — often up to three or four of them — and combines them into an ensemble model." It is apparently flexible enough to apply to any type of CPG advertiser and is constantly refining its application through the use of AI.

How It All Began
The idea and the efforts behind this initiative started about five years ago and, over that span of time, the methodology has evolved through testing and innovation. "When we first started, it was just television because you can't have a control if you have such high reach. After all, who is not exposed? People who were not exposed to the advertising look very different than TV viewers. They are very different kinds of people," she explained.

Wood discovered that this approach took the pressure off the researcher to constantly oversee the model. "Here was this machine learning method that allowed us to put this in places where data was siloed. We could set it up in 'free rooms,' places where there was no researcher," she said, adding that this allowed a door to open where Google, Facebook, and Apple could manage their proprietary datasets without a human interacting or overseeing it. "It is meant for computers to talk to computers."

In-Flight Optimization
Five years ago, the hardware just wasn't there to be able to apply in-flight optimization. But now, this can be executed on what Wood calls, "sales-lift metrics," which provides the ability to "use these machine learnings to quickly, in real time, deliver key performance indicators to tell you what about your campaign is working." It is now possible to compare multiple creative options, consumer targets, or sets of tactics for sales effectiveness and incremental causality.
"We are looking at the incremental value of each of them," she said, "and compare them" in a weekly cumulative sales report. An advertiser can make changes to the campaign as it rolls out and see the impact.

What Sets the Super Learner Apart?
"There is always a full set of statistics" within the Super Learner, Wood noted. This is unusual because "AI almost never has a full suite of statistics. Statistics were developed by professors on small data sets. For big data sets, we develop machine learning and AI methods and those were built by computer programmers. So, we don't usually have confidence intervals, significance tests, or standard error — those measures that you need to say that this is a good model. We built Super Learner, which does all of these steps."

In this way, as new machine learning methods are developed (and, according to Wood, they are being developed all the time), they are added to the system to see if there are new ways to improve campaign efficiency. "It allows us to continually improve without reprogramming or recoding," she said, adding that certain models never worked and others that worked too hard and delivered unreasonable answers. "We removed those."

Among the successful campaign indicators is incremental sales per household, which is a key performance metric. It also reports impressions delivery by target and by creative.

Going Forward
Since the Super Learner has the ability to integrate faster systems and new models, it advances every day. Thousands of new schedules have been run and rerun "to see what works and what doesn't work, what's the minimum threshold, what's feasible, what are indicators inside the statistics that let us know that it's not working."

Wood is committed to keeping up with "this fast-moving world" of data and machine learning with constant monitoring and asking why results change. In the fall, they will be adding more data, more brands, and a price/promotion feature.

And, according to Wood, this entire initiative has been a model for others. "We use the h2o machine learning language; they will tell you that we were the first to create a Super Learner and they followed suit and built one themselves," she said. "We have pushed the envelope on machine learning."

 This article first appeared in www.MediaVillage.com

The Impact of AI and Data Strategy. An Interview with Jacopo Bracco

Artificial Intelligence is becoming the hot additive to many aspects of media – from content curation to measurement. Jacopo Bracco, former president of DIRECTV PanAmerica, is currently focused on transforming the TMT (Technology, Media and Telecom) industry, as convergence across these businesses brings significant disruption.  What does the future hold? How will platforms evolve? Which jobs are safe? 

Charlene Weisler: What is your definition of AI and how do you see it impacting media, entertainment, content, data, and strategic insights?

Jacopo Bracco: At its core, AI is a technological tool that people can use to increase their productivity. Because AI conjures images of machines taking over the world and because it is referred to as "intelligence", it is often widely misunderstood and misrepresented.  Can people potentially lose jobs because of it? For sure, but just like the advent of the tractor put an end to a number of jobs in agriculture.

AI is already in use in a number of applications.  A great example is automatic captioning and metadata generation.  In the past, scores of editors, linguists and technicians were tasked with manually filling databases of textual information that would be used for creating content libraries. Today, this process is greatly assisted by AI tools. 

Weisler: Do you see AI replacing Creatives or Sales people?

Bracco: Not in the short term.  AI will make creative and sales people more productive.  Think about speech recognition.  Perhaps writers will grow accustomed to dictating their scripts or books and complete them faster.  But will the finish product be any better? Where I see opportunity is in assisting creative people in their research and organization of thought to produce increasingly rich and complex content.  Think about a journalist writing a piece about a company and using AI to identify relevant references about this company on the web.

Weisler: Netflix is an entrenched global platform. What do you see as the opportunities for competitive service for companies looking to launch similar streaming platforms?

Bracco: Netflix is global but consumer interest tends to be inherently local. Opportunities exist in speaking to local audiences.  The global battle will be between Netflix and the traditional global media companies such as Disney. Think about the decline of broadcast television and its correlation with the rise of reality television.  It was an enormous success when FOX first launched American Idol. Then it seemed all networks were doing reality.  Eventually consumers looked for other things.  Netflix is synonymous with binging.  I, for one, am already tired of watching a story for 10 hours when it could have been told in 2.  It could have been a great movie and instead it may be a slow TV show with -personal pet peeve- a cliffhanger in the end that leaves you wondering for next season.

Weisler: Major media conglomerates are impacting up-and-coming startups. What are the major media trends and do they impact established networks and up and coming startups differently? 

Bracco: The major trends are the proliferation of media and the resulting fragmentation of the audience.  There are thousands more sources of content now than 20 years ago.  This is a challenge to the major media companies who bet vast sums of money on the content they produce.  The fragmentation results in less audience even for the biggest hits. As a result of this, large companies are consolidating, which I see as the third big trend in media.  A perfect example of this is Disney's deal to acquire FOX; a deal that would not be seen as possible just a few years back.

These trends impact not only the networks but also the up and coming startups.  In fact, startups have an opportunity to establish themselves strongly with specific audiences.  If they do so and do it well, they will be interesting acquisition targets by the large media companies that must consolidate to remain relevant.  If one thinks about it, Netflix was a originally a product targeting the movie fan.  At first it was a niche but large enough and with the opportunity to grow into the mass market.

Weisler: How do you sell these audiences in to advertisers who are used to buying and selling off legacy demographics?

Bracco: Data is a fundamental part of the business model in Media.  But there is still a chasm between the potential of data and its actual effectiveness.  Think about the complexities: not only must you try to gather as much information as possible from your customer but also you must be sure it is the actual customer and not, for say, her or his family member.  Then you need processes to maintain the data up to date.  

Think about the many examples of companies you have dealings with and who still think you live in the other side of the country even though you moved 5 years ago.  Of course, we cannot ignore privacy and shifting legislation that affects your data operations but when you consider we live in a global world, a company must comply with data laws of hundreds of jurisdictions; or at least the major ones.

As with AI, there is a lot of hype with data, which is not to say there isn't a shift or a need to tackle it.  Eventually someone will get it right and the cascading effects will be major. As a result, Media companies must develop their data strategies and organizations.

My philosophy: by all means consider your data strategy, but use to give value to your customers first.  If you only are focused on your company's benefit you are bound to get in trouble just like Facebook did.  Think about Global Entry: I signed up and agreed to give a bunch of my personal information to the government; in return, I got the right to zip through immigration every time I come back to the country.  By all means, have my data.  It's so valuable to me that I will make it a point to update the data in your systems myself whenever there is a change in my status.  It's that valuable and that is a great data strategy.

This article first appeared in www.Mediapost.com
 

Using A.I. to Insert Ads into Programs. Interview with Mark Popkiewicz, CEO, Mirriad

Mark Popkiewicz, CEO of Mirriad, explains his background as, “the convergence of media, advertising, and technology,” working over the years in telecom, mobile entertainment and broadcasting. His company, Mirriad, embeds brands in programs using advanced technology. 

He views his work as a way to enable brands to break through the clutter and a way for media companies to de-clutter their air. “As networks begin to cut ad loads and experiment with shorter ad formats, there is less opportunity for quality advertiser exposure,” he asserted, “So we’ve developed technology that uses AI and image recognition to embed brand messaging directly into video content.”

Charlene Weisler: How do you track the ads?

Mark Popkiewicz: Advertisers can buy ad units and track delivery while being assured of quality of exposure, ensuring meaningful and contextually relevant ad insertion presented to the relevant audience every single time.

Charlene Weissler: How do you measure the ads?

Mark Popkiewicz: We have developed a set of metrics based on industry best practices from companies such as comScore and Nielsen together with our own research including eye-tracking, guaranteeing viewability and exposure for brands. These metrics, known in aggregate as the Visual Impact Score (VIS) have been verified to adhere to industry norms as practiced by leading ad effectiveness measurement businesses such as comScore. VIS allows us to manage which ad units are be billable. Algorithms using Computer Vision and Machine Learning generate a score between 0 and 2, and any segment scoring above a 1 is deemed ‘billable’ and those scoring below a 1 are discarded.

Charlene Weisler: How is this different from other in-program ad insertion companies?

Mark Popkiewicz: There are currently some companies offering high-end digital video solutions, but none have yet developed the comprehensive methodologies and measurement capabilities we provide. In-video advertising gives brands verifiable exposure and an ad format that’s plannable, reconcilable, and measured and sold on an audience basis, even though it’s integral to the content experience itself.

Charlene Weisler: Is it possible for the actors to interact with the advertising?

Mark Popkiewicz: We utilize a technique called ‘implied usage’ where a product is inserted into a scene suggesting it is actually in use, although it was never there.  For example an ad unit containing a bottle of wine would manifest with the wine being open and maybe half consumed placed next to a wine glass also containing the wine, all contextually relevant to the action.

Charlene Weisler: What do you see as the future of TV advertising in general?

Mark Popkiewicz: The economics of delivering content has vastly changed over the past several years. TV networks are cutting ads en-masse and a number of networks including NBC and Fox have vowed to cut ad breaks by as much as 20% over the next couple years. As the cord cutting trend continues, the video entertainment industry will need to find new ways to monetize content and remain profitable. Media ententes will need to get creative in relaying their messaging to potential consumers and with that will come new forms of advertising, which will be much more contextually relevant to the viewer based on what they watch.

Charlene Weisler: What do you see as the future of in-program advertising specifically?

Mark Popkiewicz: In order for in-program advertising to be successful, it will have to cater equally to brands, networks, and viewers and be engaging.  We have tested several methodologies to engage audiences and drive e-commerce transactions without damaging the audience experience. The results are staggeringly positive.  It’s clear that a blend of content marketing and engagement is part of the future of advertising - but at all times protecting the audience experience and adding value.

Charlene Weisler: Can this be bought programmatically?

Mark Popkiewicz: At the heart of In-Video is data.  Content is analyzed for brand fit and when combined with audience information we have the essential ingredients for programmatic.  The magic is about transforming content into data, the rest is straightforward.

Charlene Weisler: Can this be inserted on other platforms aside from TV?

Mark Popkiewicz: Our primary focus is on premium entertainment content wherever its consumed but our strategy is all about SVOD, and other OTT platforms wherever there are large audiences and a great deal of content in aggregate.