The Resurgence of Media Mix Modeling in Modern Times

Media Mix Modeling (MMM) has made a comeback in recent years but not without its critics. This decades-old approach to planning media channel investment and optimization was first popularized in the Mad Men era of the 1960s but took a backseat as digital analytics and attribution models rose in popularity. Let’s explore how MMM is re-emerging as the go-to channel planning tool again, key success principles that mitigate detractors' criticism, and how it can help marketing leaders build confidence with internal business partners (even CFOs) to approve investment within marketing channels.

MMM vs Attribution

First, let’s talk very high-level about the difference between MMM and attribution modeling. Both have a common goal to optimize the performance of your marketing channels, but the two techniques answer different questions.

Attribution is useful for answering the question, “how well are my various tactics performing when compared to one another?”, and it does this by analyzing the behavior of individual users. Attribution modeling requires direct user interaction data from across marketing channels, and then applies a defined model to determine which interaction had more influence against a conversion metric. It’s very much a look back approach and is only as accurate as the completeness of the data being used. Attribution cannot measure incremental lift, and so it cannot tell you if your advertising dollars are simply claiming credit for conversions that would have happened if you spent that money elsewhere.

Media Mix Modeling is useful for answering the question, “what incremental benefit did we get for our marketing efforts that we would not have seen if we did nothing?”, and it does this by analyzing the performance of your marketing channels as a whole. MMM is a more holistic approach that can take into account online and offline channels as well as external factors (i.e., seasonality and known business trends) to fit past spend and volumetric data to the effect on conversion metrics. Unlike attribution, the model is not dependent on user identifiers (i.e., cookies) but analyzes higher-order indicators of conversion and spend. Once the model is built and validated, it can then be adjusted for known events and trends and applied forward to determine optimal spend allocations and amounts within and across channels.

AI/ML has further refined the modeling capabilities and turn around time of MMM which has spawned an inrush of new SaaS MMM solutions, however AI/ML alone does not make MMM a turnkey or automated solution, as we’ll explore more later.

Extra Credit Reading

For a deeper dive into attribution and the fundamentals of mix modeling, we highly recommend viewing the great Tim Wilson’s Multi-Touch Attribution: Approaches and the Tradeoffs (And Fallacies) Therein presentation at the 2022 Superweek Analytic Summit.

Privacy Impacts

Another key factor shifting the tide from attribution to MMM is the changing attitudes about privacy and collecting data about individual users. Attribution only thrives in digital channels where cookies (and other identifiers) provide rich data about the behavior of individual users. As browsers and legislatures continue to restrict the collection of user data, attribution models are forced to operate on top of increasingly poor quality data. Coupled with the rise of false traffic (bots), the signal-to-noise ratio has greatly decreased, degrading the effectiveness of attribution techniques. MMM is able to sidestep the privacy impacts by looking at higher-level macro indicators, not dependent on user identifiers, to correlate investment to campaign/channel performance.

Perceived Issues with MMM

Media Mix Modeling has its share of detractors as it carries baggage from both its pre-digital years as well as new issues stemming from contemporary techniques:

1. Time-to-Value - MMM models are not real-time and further compounding this problem is the time required to generate the lookback results and to tweak the model to look forward. This time has been greatly reduced by AI/ML model generation but human, hands-on data model review and rationalization is still an essential component in producing effective results.

   
   A reasonable reporting and analysis window for MMM is on a monthly cadence with analysis available within a week after the end of the period. This meshes well with 4 - 8 week adjustment cycles for media planning.

2. Return-on-Value - Cost to run MMM is broken down into 2 buckets: tool/solution license (particularly when using an enterprise or SaaS solution) and the time needed to generate the model, analysis, and reports. AI/ML and reporting solutions have greatly reduced the manual labor involved while the traditional enterprise license costs are also under greater competitive pressure from SaaS solutions with much lower enter point budgets. Combined this has helped lower the total investment needed to run an effective MMM program.

   
   A generally accepted ROI for MMM is 5% - 30%, depending on size of marketing spend, inclusive of platform and labor.

3. Inaccuracy / Lack of Transparency - One of the greatest challenges we hear from clients is the lack of transparency within the modeling. This arises for a number of reasons with the two most prevalent being: a) the solution or agency partner running the MMM model does not share enough visibility into how the model works and b) insufficient investment of time by the end-client to make sure they understand the model and process. This in turn, raises the question of confidence in how accurate the look back is and whether the look-forward recommendations can be trusted.

   
   Related to the return-on-value issue, the more involved you are in the process of MMM as the client, the better it will perform and greater level of trust you will build within your business.

Recommended Principles for Effective Media Mix Modeling

To harness the full potential of MMM, we suggest these five success principles:

1. Balance Machine vs Human Involvement

Human review and rationalization of how MMM is applied and the modeling built within it is essential. AI/ML has greatly sped up the process of modeling but subject matter expertise about your business and the industry is paramount to ensuring the models are confident, make sense, and take into account known future exceptions and events. There’s also an experimentation aspect that needs to be managed to evolve the models over time and gauge how well they fit to your historical data. While there are solid new MMM solutions that have recently come to market, be wary of those promising full set-it-and-forget-it automation.

2. Data Requirements

The two key elements needed to run MMM are spend and impressions for each of the marketing channels and the target performance variable of choice. Return on Ad Spend (ROAS) is the most common performance indicator, however we tend to favor Return on Marketing Investment (ROMI) as a gold standard for analysis as it focuses on margin contribution and can provide the deepest understanding of your business. The historical data lookback window should ideally span 2 - 3 years with weekly granularity in order to capture seasonality and test model fitting. Shorter look back windows and monthly granularity is workable and still effective but we’ll lose some fidelity in the forward optimization recommendation.

3. Model Performance & Transparency

Media Mix Modeling has its roots in data science, well before we called it data science. We caution against the temptation to become enamored by the black box effect of AI/ML automated MMM solutions. Programmatic works well in many media applications but for omnichannel planning and look forward optimization, human rationalization is critical to iterating on model performance and driving fastest time-to-value.

To guarantee the reliability of the models, we prefer to adhere to a standard of model validation, which includes rigorous statistical testing against a hold-out data set. This process of accurately forecasting against a hold-out set empowers us to confidently project marketing outcomes 6 - 12 months into the future, ensuring our clients can plan with reliability and confidence.

Another key feature to look for in your MMM approach or solution is how well are you able to identify your saturation curve for each channel and then identify where your channel investment currently sits along that curve relative to the range of channel optimization to channel saturation (the point at which incremental working investment no long yields returns).

4. Scalability

When considering a new or replacement MMM solution, look for platforms (and services partners) that are able to start with smaller initial runs. These might be an initial DMA or even as a proof-of-concept. While you’ll need a certain volume of data and media spend to get meaningful results, it’s not always practical to jump into a full enterprise license for a solution. Surprisingly, many MMM platforms still require this high-level of initial commitment to test drive. Start focused, build confidence, evaluate, and expand at each step. This also becomes an effective approach to testing against existing solutions and approaches you may already have inflight. A solid MMM solution should be able to intuitively start as a proof-of-concept and scale up to handle all of your channels and regions.

5. Ownership

We believe it’s essential for the end-client marketing team utilizing MMM to fully oversee and manage the solution and process. It's common to find marketing and advertising agencies wholly managing MMM on behalf of their clients, often with proprietary, in-house solutions. The potential issues here are twofold: 1) end-client marketing teams tend to not be in the weeds as much as they need to be in understanding the model and help shaping it, and 2) we believe that agencies shouldn’t grade their own work. This is not to say that there are not a majority of agencies doing amazing work with MMM on behalf of their clients, but as a client, you should own the process that’s auditing your ROAS.

A Brief Case Study

We worked with a national retailer client to apply MMM across online and offline channels that comprised a total marketing budget of $30M annually. We began by establishing and accurately estimating the baseline performance across the channels. This is a crucial step in order to precisely calculate the return on marketing investment (ROMI) for each channel. The goal at this step was to identify (and to what extent) which channels were at saturation and as well as which channels were underinvested based on expected incremental return.

Channel Insights and Adjustments

Over Saturated Channels

Our analysis identified Facebook, Google, and Radio as the primary channels where spending had exceeded their effective threshold. There were millions in unnecessary expenditures that were not producing efficient returns within these channels, which could be preserved or reallocated to underinvested channels.

Underspent Channels Identified

Conversely, we discovered significant underspend in digital audio channels such as Spotify, Pandora, and Audacy, alongside display ads. These channels demonstrated untapped potential for driving incremental sales and we recommended some portion of the marketing spend be reallocated to these channels to move them closer to their saturation point. Additionally, some budget was distributed to new media channels to test their effectiveness.

Outcomes

After reallocating and optimizing the spend across these channels, we observed a $9M reduction in overall marketing budget which resulted in no impact to the baseline performance and provided greater margin contribution in terms of ROMI from the channels which were increased. As part of future interactions of the model, we continue to review model fit versus historical data and fine turn the allocations.

This particular case scenario is one of budget recovery and reallocation. In many of our other MMM applications, we typically find an underinvestment in overall budget and use this process to build confidence to help our marketing business owners make the case for additional incremental investment with the ability to build confidence overtime towards an optimal target budget.

Final Thoughts

Media Mix Modeling continues to gain renewed momentum driven by user-level signal loss in the current era of data privacy and accelerated modeling fueled by AL/ML. By addressing the concerns surrounding its application and embracing best practices, MMM can transcend its traditional role as a budget allocation tool to become a strategic partner that aligns marketing initiatives with broader business objectives. This alignment not only optimizes marketing spend but also cultivates a shared confidence among all stakeholders, from marketing leaders to CFOs, in the strategic direction and financial stewardship of the brand's marketing investments with improved time-to-value.

A very special thanks to our partners at MediaMix.ai who collaborated with us on this article.