When KPIs are Blinders: The Dangers of Local Optima

/in , , /by

Recently we encountered a strange situation.  One of our clients had a top of the funnel KPI that looked like it was going sideways with regards to efficiency, but their revenues were going gangbusters. We use this metric for optimization and it is the core number by which everyone from their Board of Directors on down judge the efficiency and success of the digital campaigns.

Suffice it to say, this was not good.

Interestingly enough, the revenue numbers were setting new records, which made us question what was happening with our long standing and go-to KPI. So we started checking off the boxes. Was the revenue coming from the current marketing investment? Check. Were we paying what our financial models said was appropriate to the audiences? Again check. Was the top of the funnel KPI consistent over time and audiences?

Hmm. No, no it wasn’t. Aha! Changes to user engagement paths had affected the top of the funnel over time, and the average KPI from the past was no longer aligned with revenue and profit.

Our go-to KPI had become a Local Optimum, and we needed to address it quickly.

I first came across the term Local Optima in Eliyahu Goldratt’s classic novel on business optimization The Goal.  Essentially, a local optimum is a metric that may apply to a “neighborhood” of criteria, but is uncorrelated with the entire system. Goldratt used the uptime of a single machine in a factory as his example, where running any machine at 100% capacity other than the one machine that throttles throughput for the entire factory will cause inefficiencies and losses for the entire system.

Goldratt and his Theory of Constraints thinking dictate that “Measurements of local optimum behavior should be abolished and replaced with holistic measurements.”  If we had a Local Optimum on our hands, attempts to “correct” for this KPI would not be mildly bad, it would almost assuredly hurt the profitability of the entire company.

So we threw out the KPI and went back to first principles: “Marketing is a financial investment in a financial outcome”. What could we look at that would be correlated to revenue production over time?

It turned out that slightly down-funnel was another metric that turned out to be 100% correlated to revenue production and, in fact, was the one to which we calibrated our bid models.  We were able to show that the sideways direction of the top-of-funnel KPI wasn’t a problem, but was actually a natural side-effect of using the “true” KPI to back-calculate to the top of the funnel. By shifting the conversation from our most visible KPI, the conversation then quickly shifted to how to take advantage of this knowledge. How could we change our reporting, conversation, and client-side metrics to align? We lucked out. Our client is extremely savvy in understanding their own cohort data and financial targets and rapidly escalated the conversation to the C-suite and their Board.

We talk consistently about using Profit as a KPI. In reality, as digital marketers we often have to use early engagement numbers as proxies for true Profit. It is always good to step back and assess the relationship to Profit in case a Local Optimum has snuck into the room as it did in this situation.

What Magritte Has to Tell Us About Marketing Data

/in , , , /by

In the winter of 1928-29 a Belgian painter living in Paris painted what looked like an advertisement for pipe tobacco. The painter, René Magritte, painted a caption below the large pipe stating “This is not a pipe” creating confusion for viewers and a bit of a stir in the Paris art scene at  the time.

The title of the painting is “The Treachery of Images” and Magritte intended it as a reminder that representational art is a reflection of the artist’s view of the object,  not the object itself. His interpretation was that  the painting of the pipe is not actually a pipe. You can’t smoke it, fill it with tobacco, or put it in your pocket; it is a representation of reality.

Magritte’s message is particularly apt in our view of marketing data. As marketers, we use our marketing data all the time in measuring and assessing user behavior related to digital advertising and engagement. But it is extremely important to remember that marketing data is an imperfect representation of user behavior and not a perfect simulation.

There are three main reasons that marketing data is imperfect. The first is in the nature of tracking. Tracking is technically limited in its scope and reach.  At best, tracking can measure engagement from the same device over time, or the same multi-device account over time. This only works if the tracking is implemented correctly in the first place and is not disabled by the end user. Because of the technical limitations of tracking there will always be engagement that is not tracked because of multi-device use, the amount of time for which the tracking is active (cookie window), or because of personal opt-out at the user level either by choice or by browser pre-set settings.

The second main source of imperfection in marketing data is in user behavior.  As humans we have a wide variety of choices and methods of interacting with advertising. Some of us choose to avoid interacting with advertising as much as possible while some of us behave in the opposite fashion. Some of us will choose not to click on ads at all, some of us don’t hesitate. Some of us need to heavily research purchases, some don’t. There is not one engagement path that is adhered to by all users yet often our data is interpreted through a “single funnel” lens that introduces inaccuracies in interpreting data.

The last main source of data imperfection is in data integrity. The cleaning and analyzing of data must align with the knowledge being sought. De-duping rules will differ by business model. Attribution will vary based on external factors such as affiliate payment rules (and will still never tell a 100% accurate story). Data dropout can cause interpretation issues.

So what is a marketer to do?

The best thing you can do is to fully utilize data to test your own model of reality.  The best data firms don’t blindly use data but instead use it to inform their perception of reality.  For example, if you know that a percentage of social users engage with your brand without clicking on ads and instead show up as brand search or No Referrer traffic, you should try to assess: how much, how can that magnitude be assessed and what impact does it have on my decisions as a marketer if I can hypothesize about non-measured (but real!) activity.

Magritte’s point  is well-made. Our data is not reality, but an imperfect reflection of reality biased by user behavior, technical limitations, and process. Our best response is to acknowledge and accept this and use it to our advantage.

5 Hidden Dangers in Annual Budgets for Marketing

/in , , /by

Ah, the smell of the end of the Fiscal Year! Budget Planning for Marketing is in the air and the smell of burning remaining budgets lingers, backed by the sound of inter-departmental standoffs.

It doesn’t have to be like that.

Annual Budgeting is a time-honored process, but I’d argue that in today’s world annual budgets for marketing do more harm than good. Flexible budgeting based on hitting financial KPIs and working within cash-flow limits is a much better way to go.  So roll up your sleeves and get that coffee, because today we’re talking about the five main reasons Annual Budgeting can harm your marketing program!

1. Cost vs. Profit

Annual budgeting attempts to control costs while managing resources for the continued profitable execution of the business. Unfortunately, the two common tools for reviewing financial performance, the P&L (past-looking) and pro forma (forward looking) are poorly equipped to understand marketing’s direct contribution to bottom-line growth as they both separate cause (investment in marketing) from effect (profit realized from marketing).  This is solely because marketing has always been treated as a fixed cost, along with salaries and rent. But quantitative marketing, and particularly quantitative digital marketing has changed that.  Done right, data-driven marketing creates a clear and predictable relationship between investment and return, often optimizing to financial KPI’s such as Contribution Margin (post-marketing gross profit) that, if scaled, will improve the profitability of the company. Annual budgets, by definition, cap the potential upside of profitable opportunities that might exceed the estimates of the company at the time of budgeting.

The Better Way: Utilize strict measurements on the generation of profit from marketing where marketing is expected to increase Contribution Margin over time (how much time is predictable, but is based on sales/evaluation cycles and will vary widely by company). Combine these with cash flow controls so as not to overspend in advance of revenue. For example, in a SaaS company we might be able to predictably increase future Monthly Recurring Revenue (MRR) in an acceptable payback period with a greater ad spend, but monthly spend would still be capped as a percentage of current MRR for cash management reasons.

2. Limits on Course Correction

Budgets are a plan for the future, but the more detailed they are, the more difficult it will be to change course, even in the face of compelling evidence that you should do so. Budgets are often difficult to change on purpose, as the control inherent in the budget planning process is one way that leadership executes the course for the company.

The problem is that detailed knowledge of opportunities doesn’t live at the top of the organization, it often lives deep inside, at the levels of tactical decision making. As opportunities surface, the entire budget planning process, people, and leadership control needs to be challenged even to make a minor change. It sounds crazy, but is extremely common to hear “We know we should make this change and doing so will increase profit, but it just isn’t in the budget.”

In a recent example, one of our clients realized that they had given us the wrong remarketing budget and that they had no more budget for the remainder of the fiscal year. And even though we have clear data illustrating the profitable results of the investment in remarketing, the line managers decided they would rather forfeit that profit than challenge the budget process. This does not help their company’s bottom line.

The Better Way: Put money behind goals with limits on expenditures. Educate line managers on P&L level financial KPIs and enable them to move resources to improve those financial KPIs within cash flow limits. This often means addressing local optima, KPIs that don’t correlate well with financial outcomes but are intended as measures of performance separate from revenue and profit. In marketing, most traditional measures (Bounce Rate, Time-on-Site, etc,) are local optima. Local optima are blinders to good prioritization of activities that improve the business and should be viewed with caution.

3. Arbitrary Channel Blindness

Marketing is complex. Actually, that isn’t nearly as true as the fact that the human interaction with marketing is complex. Yet internally we fall in love with single-source attribution to evaluate marketing channels. Nothing feels better than knowing that this one customer came from this single source. It makes us confident in crediting programs and teams, and makes it easy to allocate budgets.

Of course, it’s dead wrong.

We know from our own behaviors that people engage in interactions with marketing campaigns that are far more complex than our tracking can fully capture. We see ads in multiple locations over a period of time. We are remarketed to. We see an ad, don’t click on it, and then later do a brand search. We switch devices before engaging with a web site. We’re human and so are our customers.

Single source reporting creates a false sense of security, and like the CEO who exclaimed, “This Brand traffic is great! Let’s move all our budget to that!” it often masks cause and effect within the complexity of real human behavior. (If it is not clear what the issue is with that comment, please reach out to me.)

Budgets, of course, make this worse the more granular they get. In addition to making my last point worse in that they can limit course corrections to better performing areas, channel budgets reinforce the false view that channels act independently of one another, and that all channels act in similar fashion with similar functions.

The Better Way: Have your financial KPIs be holistic, evaluating black/white success only when looking at the whole picture of paid, unpaid, earned, unearned customer acquisition. The channel numbers are directional and need to be evaluated carefully. Your Cost-Per-Acquisition numbers in Search and Social shouldn’t be the same, as Social typically drives 5x to 10x more out-of-channel activity that breaks source tracking. Data needs to be used to its fullest extent, but with thoughtfulness, experimentation, and a constant understanding that data collection is limited with regard to user intent. When changes need to be made, or experiments run, the budgets should easily and flexibly accommodate those changes.

4. Slow Learning

In the Mad Men days of traditional media, learning about campaign performance was slow, when it could be measured at all. Annual budgeting caused no significant harm, because annual budgets often aligned with annual media buying.  In a world where success was often obtained by the most demographically-correct eyeballs for the lowest price, demographic targeting and price negotiation were everything.

No wonder people had time for all those martini lunches!

Modern digital advertising is a world of constant measurement and constant experimentation. The limits of the budget process in slowing change and limiting complex understanding have one extremely dangerous but often overlooked effect: They slow learning. Gains in digital media are often discovered through testing (certainly this is the financially responsible approach), so how many test cycles you engage in is one control over success. A process that by nature limits the speed of testing can have a highly detrimental effect on performance that is likely completely invisible to those engaged in the budgeting process.

The Better Way: Create the freedom for testing within the budgeting process. 10% of the marketing budget that is solely for experimentation and is not held to financial KPIs will pay off dramatically in rapidly finding ways to scale profits.

5. Incorrect Time-Frames for Budget Evaluation

Fundamentally, annual budgets operate at an arbitrary time frame that is out of sync with marketing decisions that need to be made in a financially optimized, highly data-driven campaign. Quarterly budgets are not much better but will be closer to the learning cycles in most campaigns.

While the longer the budgeting cycle, the worse the problem is, the fundamental problem is that line-item budgeting for marketing is an inherently bad practice. In a world of 24/7 live auctions, shifting competition, constant learning and improvement, budgets are sub-optimal.

The Better Way: There is a better way, and many organizations already do it this way. Financial KPIs tied to cash flow-based caps given to engaged teams with line-level decision making work far better than a classic budgeting process. The best, most nimble companies we work with work off of pro formas that are not set in stone but that are live working documents, recast constantly based on evolving data.

When done right, digital marketing contains an amazing set of tools for the predictable scaling of profits. What is required is for companies to support the financial practices that allow this to happen as quickly as possible, provide financial transparency and clear financial KPIs to their marketing teams, challenge their teams to be thoughtful about complex data and behaviors, and stand back. Maybe it is simple after all.

Profit: The Only Marketing Metric that Matters

/in , , /by

I recently wrote about the limitations of one of the most widely-held marketing KPIs: ROAS (Return on Ad Spend). The main limitations with ROAS are that it is a measure of efficiency and not magnitude and that the efficiency it measures is revenue generation, not profit generation.

Yet creating revenues without profit, or efficiency without magnitude is not creating success, and we want KPIs that are tightly aligned with success. In fact, that is the sole purpose of a KPI.

So if ROAS sucks at being an indicator of the magnitude of profit, what KPI does work?

I’m going to go out on a limb here and say all marketing metrics are poorly aligned with business success. Which is why your marketing KPI shouldn’t, in fact, be a marketing KPI at all.

It should be a business KPI.

In fact, it should be a really specific business KPI and that is a form of profit measurement called “Contribution Margin”. Contribution Margin is pretty simple to calculate, it is just Revenue – Variable Costs. In this respect it’s a close relative of a metric we like a lot: Gross Profit (Revenue – Cost of Goods). But the key to a good Contribution Margin calculation is what’s included in the variable costs. For us, the biggest variable cost is Ad Spend. But in actuality, we also add in cost of goods and any other per-unit costs if they exist.

If it sounds complicated, you can probably simplify it. If you want to call it Post-Marketing Gross Profit that works, and in many cases may be close to the same number as Contribution Margin, depending on what other Variable Costs exist. Frankly, we find the hardest part is to get this information from clients, as it’s unusual for Marketing to know these numbers and Finance may need an explanation on why it’s important to share them.

So why is Contribution Margin a good KPI? Well, first, it doesn’t just align with the magnitude of profit, it IS the magnitude of profit. Second, if you’re looking at actual profit, a false read on efficiency can’t cause you to make bad decisions in the name of efficiency (this is called the local optima effect and I’ll post about this in the future). If you use Contribution Margin as a KPI, you actually see the drop in profit. And lastly, it lets you do two very exciting things in growing a business, and those are that the investment in marketing becomes a profit center and not a cost, AND it allows you to have a business metric that transcends channel complexity, but how that works is a topic for another time.

5 Marketing Data Mistakes Most Companies Make

/in , , , , /by

Even the most data-savvy of marketing teams can make mistakes in thinking about the use of data in optimizing campaigns for financial success. Let’s face it, marketing data is rife with issues and is never perfect, and it is easy to put on blinders based on the data you have available, the systems you use, the marketing channels you work with, or even the directives of senior executives. Despite that, everyone wants to be able to connect the dots from ad impression to profit. Here are a few common data traps we see even smart companies easily fall into. How do you rank on this list?

1. Using Average Value CPA Targets

Cost-Per-Acqusition targets are fertile ground for data issues. For example, are your targets even based on customer value, or are they merely a “seems reasonable” guess? Smart data-centric companies will base target CPAs on actual customer value to ensure that their marketing programs don’t risk over-paying for customer acquisition. However, even smart companies can fall into using a single average CPA target for all their customers. In doing so, they underpay for audiences that provide higher-than-average value customers and overpay for low-value customers. Companies can avoid this by using targets tied to segmented customer values, not averages.

2. Using Last-Click Attribution

The attribution question has long been mired in a false discussion of “who gets credit?” when there is more than one user touch leading to a sale or lead. Some companies still use last-click attribution, often in a mistaken belief that this is somehow a “truer” view of acquisition, or that it just allows them to avoid thinking about attribution at all. Google hasn’t made things easier by offering multiple views of attribution with little guidance on when and where the different options should be used. Here is our take: Avoid last-click attribution at all costs unless you are evaluating retargeting assists. Last-click attribution will severely over-inflate your brand and direct numbers and cloud your ability to see high-value first-mover channels.

3. Not Considering Out-Of-Channel Effects

Everyone looks at their channel-specific numbers, but an astonishingly few companies continually examine their direct and brand channels, looking for influence from other areas. While everyone logically understands that users did not wake up with magical knowledge of a company’s brand, it often feels like that’s the assumption of marketing teams who put on “channel blinders” when evaluating their programs. Smart companies view their data holistically, looking for out-of-channel trends that increase or decrease direct and brand engagement. While only 5% of users in a typical search campaign are likely to bounce over to direct or brand, it is not unusual for a whopping 50%-90% of sales from social media or display campaigns to come through direct or brand traffic.

4. Over-Valuing Metrics Not Tied to Revenue

What is the value of a Like? Most data-driven marketers have moved on from directly equating social media engagement as revenue-related events, but many metrics that don’t correlate well to revenue are still held as sacred cows. Any metric used for campaign optimization should be well understood in how it relates to revenue before it becomes a key KPI. Data-driven marketers with their eyes on the profit prize quickly realize that Time-On-Site, Impression Share, Cost-Per-Click, or other common metrics are not as tightly aligned with profit as they might think when other factors such as volume, customer value, out-of-channel influence, or profit margin are taken into account. Easy rule of thumb: Use post-marketing profit as your marketing KPI.

5. Assuming Traffic Equals Sales

We’re two decades into the digital revolution, and it’s still incredibly common for people to assume that eyeballs equal profit. Back in the days of traditional media the best shot you could make in media buying was the most eyeballs for the lowest cost. That approach doesn’t work in digital because of the competitive auctions, and yet “Let’s get more traffic to this page/product/site” is not at all an uncommon marching order, particularly from executives who don’t understand the auction effects in digital media buying. Smart data-driven marketers know that their job is as much about when NOT to buy traffic as it is in finding the areas of success, and continually evaluating how to increase the quality of an audience by peeling away the “eyeballs” that are not their target audience. This allows them to compete more aggressively in the auctions while protecting the bottom line. Sometimes less really is more.

These are samples of marketing data traps that are very easy to find in almost any campaign. Most of these issues can be avoided through three core practices: 1) By adhering to a holistic financial lens in optimizing the entire marketing program against financial targets; 2) By working backwards through the path that led from advertising to revenue and; 3) By not ignoring revenue that falls outside of the “channel buckets”.

Soren Ryherd to Speak at SMX West 2016 – Chain-Based Attribution

/in , /by

Working Plant Co-Founder, Soren Ryherd, will be speaking at SMX West 2016 in San Diego on March 2nd as part of the panel on “Attribution Beyond the Last Click”. He will be discussing the merits of Chain-Based Attribution.

Panel Overview: Giving all the credit to the last click of a conversion is like saying the scorer is solely responsible for the goal. Attribution is much more accurate when factors such as in-store visits, phone calls, and offline conversion uploads are taken into account. In this session, panelists examine various attribution metrics and show you which are best for the most important things you need to measure in understanding conversions.

Soren will be joined on the panel by Dave Roth of Emergent Digital and Adam Proehl of NordicClick Interactive. The panel will be moderated by Brad Geddes of Certified Knowledge.

Soren Ryherd to Speak at SMX Milan 2015 Conference – Mad Scientists of Paid Search

/in , , /by

Working Plant Co-Founder, Soren Ryherd, will be speaking at SMX Milan in November!  Soren will discuss why Out-Of-Channel measurement is critical in to the success of CPA-based marketing given the high levels of out-of-channel user engagement outside of Search, as well as the increasingly common cross-device behavior seen in mobile search. 

SMX Milan will take place at the Meliá Milano hotel on Thursday and Friday November 12 & 13 2015. Please catch up with him after the session, if you plan to attend! 

What “The Next Level” Looks Like for Digital Marketing

/in , , , /by

“We want to take our marketing to the next level.”

We hear that often. But with digital marketing becoming more complex by the minute, what does the “next level” even look like?

1. It’s Holistic

With cross-device and cross-channel behavior becoming more the norm than the exception, marketing programs that are siloed by channel are going to be increasingly inefficient. For example, Mobile ad exposure can drive desktop engagement. Video ads may well drive brand searches.  But when value created in one channel is realized in another, it creates disconnects for budgeting and performance measurement. Striving for neat and tidy single-channel performance numbers is likely detrimental for your business. Marketers who break through to the next level are looking holistically at engagement across channels, across engagement points, and over time.

2. It’s Even More Data-Driven

Data-driven reporting, or simply looking at Google Analytics reports won’t cut it any more. Data used just for creating reports is data that is being wasted. True data-driven marketing feeds core metrics from advertising, engagement tracking, and customer databases back into ongoing optimization on a daily basis. “Next-level” data will focus on supporting complex modeling with hard data from multiple sources. The days of “do, measure, done” are gone.

3. It’s Financially Rooted

If you’ve been optimizing for marketing metrics without an underlying financial model, it’s time to step it up (take notice “Time-On-Site”). Marketing at its core is a financial  investment in a financial outcome. “Next level” marketing is explicit about the financial model and how engagement and customer value support revenues and profits. To do this, Key Performance Indicators must be tightly aligned with profits.

4. It’s Predictive

The ad dollars you spent today may have no relationship to the sales you made today unless you have the world’s shortest sales cycle. Today’s sales were likely to be largely, or maybe even entirely, from past advertising.  Tomorrow’s marketing relies on predictive models that are savvy about time. An understanding of sales cycle, latency and engagement process is critical to the financial assessment and efficient optimization of your program.

5. It’s Complex

Yes, life would be easy if marketing were simple. But today’s big opportunities in Digital lie in mastering complexity. The better you are at understanding complex user behaviors and tightly optimizing those behaviors for a clearly-stated financial result, the better chance you have to beat your competition. This means understanding the limitations of your ad management tools, reporting, tracking systems, and customer data and moving beyond those limitations.

The “next level” is an exciting place. Welcome to it.

Moving Beyond Search

/in , /by

Data-driven marketing in the last decade has largely been thought of as a search marketing phenomenon. The growth of search fueled many of the tactics and tools adopted for digital since the launch of paid search in the early 2000’s. But now is the time for companies to step back and re-evaluate the digital landscape. If you’ve been heads-down in search marketing, you are in for a big surprise!

Paid digital advertising is both better and more varied than ever. Innovations in targeting, retargeting, ad formats, and measurement have broken open the dam, and advertisers should be embracing both new media formats in their digital campaigns as well as more advanced forms of measurement, evaluation, and modeling. Opportunities abound for companies willing to look beyond search, but to maximize these opportunities, some coveted metrics and tools will need to be cast by the wayside.

New Opportunities

If you have been heavily reliant on desktop-based search, then welcome to the party! From video to internet radio, to promoted tweets, pins, and posts, from pre-roll to rich-media display ads, the digital opportunities are almost endless for connecting to a targeted audience. Some favorites of ours at Working Planet include more sophisticated retargeting display ads, YouTube ads from text overlays to pre-roll video, Google Shopping product listings, and LinkedIn B2B paid advertising. All of these have proven to be remarkable areas of growth for clients willing to break out of the search box.

New Measurement Required

If you are moving out of search, be aware that your measurement and optimization program will need to mature as well. Many, if not most, non-search forms of paid digital media involve a more complex pattern of user behavior than the click- visit-buy approach that is common in desktop-based search advertising. Often, this will require looking holistically at your sales across channels, and looking for cause and effect in different channels. Be prepared for customers who engage without ever clicking on an ad, and don’t be locked into looking only at known channel behavior.

The good news is that there are sophisticated ways to assess and optimize multi-channel campaigns that take advantage of complex user behaviors. Attribution Modeling exlores the path of multiple visits across channels in the creation of value. Media Mix Modeling looks at the influence of marketing on cross-channel and cross-device behavior. Tools like these allow for the same kind of efficient optimization in non-search media, reducing risk and maximizing value.

We’re excited about the unceasing pace of innovation in digital media. Every new ad platform creates new opportunities. So look beyond search. Search is great, but there are so many other audiences to tap into.

Soren Ryherd to Speak at SMX West 2015 – Conversion Tactics for SEM

/in , /by

Working Plant Co-Founder, Soren Ryherd, will be speaking at SMX West in San Diego on March 4th! As part of an exciting panel on Conversion Tactics for SEM, Soren will be focused on how embracing the complexity of user behavior can help in your conversion improvement tests. Soren will be introducing the concept of the “Interruption Curve” and what that means for the proper analysis and testing of your digital campaigns.

Soren will be joined on the panel by conversion guru, Tim Ash, as well as by Luke Alley and Matt Van Wagner. Catch up with him after the panel to chat and ask questions!