Blog Post By: Chris Graham

A Tool That Reallocates Your Ad Budget For Cost Efficiency


A prominent E-Commerce client came to HYFN as a sophisticated but nascent social advertiser. Their brand and direct response initiatives lived with separate teams, they had fixed budgets for specific countries and initiatives, and most importantly, they had fixed budgets for individual segments within those initiatives, such as interests or purchasers of similar products.

We knew this wasn’t going to work. Why? Because we’re navigating a massive number of variables, meaning consumer behavior is changing constantly. Some of these variables include:

  • People buying our product, and therefore no longer being in our addressable audience
  • Competitors entering the space
  • Other forms of advertising influencing purchase
  • New competing advertisers in the Facebook auction
  • New devices changing the way users consume advertising and social sites
  • Those same devices changing the way users purchase products

Because of this, we shouldn’t expect to be able to predict, with perfect precision, what our budget allocations should look like before we start a campaign. Instead, we should begin with an idea of what those budget allocations will be, but be flexible enough to change them throughout the campaign.

‘Doc Oc’ is a product HYFN developed to manage the reallocation of spend for clients with several initiatives across segments. Very simply, it tells us where we should be allocating your budget to achieve the best results.

That makes sense! So it’s A/B testing, right?

No! A/B testing is an overused term that applies to a specific kind of testing:

https://en.wikipedia.org/wiki/A/B_testing

A/B testing is a great tool, but it has an inherent flaw when applied to advertising: we run a specific sample for each segment, observe the results after the fact, and then make decisions accordingly. While this is fine for some scenarios, it also comes with serious limitations:

  1. It wastes money! If we know after two days that B is definitively better than A, why would we keep spending the same amount of money on each? Instead, we could start moving budget towards B, and see if the results continue to hold.
  2. It assumes we should be running roughly even samples to start, which we often know not to be the case (think remarketing versus prospecting; the second audience is a whole lot bigger!).
  3. It doesn’t help us achieve a mixed strategy.

What is a mixed strategy?

A mixed strategy in Game Theory is any strategy in which we don’t continually make the same decision. If you always kick the ball left in a penalty kick, the keeper will eventually figure it out and be able to block your kick, so sometimes you should shoot right. This perfect proportion of kicking right and left is called a mixed strategy Nash Equilibrium.

https://en.wikipedia.org/wiki/Strategy_(game_theory)#Mixed_strategy

Hey! Like that one movie, A Beautiful Mind, right?

Yes, with less misogyny and psychosis.

OK, but how does that work in advertising?

So, we have a certain number of segments we’re trying to reach (let’s say three) and a goal we’re trying to achieve (let’s say a fifteen dollar cost per sale).

  • Segment 1, Remarketing
  • Segment 2, Lookalikes
  • Segment 3, Interests

Now, if all of these people were exactly the same (i.e., equally like to see an ad and buy our product), and the size of each group was exactly the same, we would simply apply budgets equally:

  • Segment 1, Remarketing: $100 Budget
  • Segment 2, Lookalikes: $100 Budget
  • Segment 3, Interests: $100 Budget

We’d expect the exact same result from each segment, which would be great and our work would be over!

Hey, wait a minute! There’s no way each of those segments is exactly the same size. Also, Lookalikes always perform better than interests, because the intent signal is higher. Same with remarketing. So there’s no way that’s true!

Alas, you’re right. So in reality, we’re going to need to allocate budgets a little differently:

  • Segment 1, Remarketing: $10 Budget
  • Segment 2, Lookalikes: $190 Budget
  • Segment 3, Interests: $100 Budget

Here, you can see that we’re making some assumptions about how likely people are to buy in each segment, as well as how many dollars we can spend against each. But, as soon as we start the campaign, we’re going to see that we’re wrong:

  • Segment 1, Remarketing: $10 Budget. CPA: $5
  • Segment 2, Lookalikes: $190 Budget. CPA: $10
  • Segment 3, Interests: $100 Budget. CPA: $20

Some segments are outperforming others, which means that our budget allocations aren’t exactly perfect. This is why we don’t A/B test: we’d wait until the end of the test to move money around accordingly, whereas now we can move it within the test.

Exactly! So move all the money to Remarketing, which has the lowest CPA!

Yeah, about that. It won’t work, because that audience is finite. Particularly, in relation to the amount of money we’re spending.

See, what our client wants to do is acquire as many people as humanly possible at a given price that they know works within two parameters:

  • Their overhead structure: what does it cost to maintain our supply chain? Servers? Customer Support? Snacks? Etc.
  • The lifetime value of those customers: sure, I may have bought a $10 subscription today, but how likely am I to renew? To buy incremental services? What is the likely dollar amount I’ll spend by the end of our engagement?

This means that for any given group of people, we have a specific constraint: the cost of acquisition. So, while we may have a remarketing group of 100,000 people, even if we can acquire all of them for $15 (not likely!), we’d still want to advertise to other groups, as long as we can acquire some number of them for $15 on average.

This is best understood through SOM/SAM/TAM, a classic approach to understanding the markets your business serves:

https://www.thebusinessplanshop.com/blog/en/entry/tam_sam_som

While SOM may be easily accessible and cheaper to acquire, that doesn’t preclude TAM and SAM, and especially small portions of those markets, from being areas where we can acquire customers. In fact, for a business of our client’s size, it’s essentially a requirement.

Why? Because if we allocate all of our budget to Remarketing, this will happen:

  • Segment 1, Remarketing: $300 Budget. CPA: $30
  • Segment 2, Lookalikes: $0 Budget. CPA: N/A
  • Segment 3, Interests: $0 Budget. CPA: N/A

What happened? We over-allocated budget to a saturated market and found that the cost of acquiring incremental customers there was well over our goal. Worse, we now have no idea how much to put in Lookalikes or Interests.

OK, that makes sense. But what should our budget allocation look like?

Let’s go back to our campaign performance:

  • Segment 1, Remarketing: $10 Budget. CPA: $5
  • Segment 2, Lookalikes: $190 Budget. CPA: $10
  • Segment 3, Interests: $100 Budget. CPA: $20

What we left out here was magnitude. Without understanding how many conversions each of these campaigns represents, we can’t make any real decisions about how to reallocate money. So let’s see what those look like:

  • Segment 1, Remarketing: $10 Budget. CPA: $5. 2 Conversions.
  • Segment 2, Lookalikes: $190 Budget. CPA: $10. 19 Conversions.
  • Segment 3, Interests: $100 Budget. CPA: $20. 5 Conversions.

Now we’re getting somewhere! We’ve established:

  • That all segments are valid if they meet our goals
  • That we’ll be allocating budgets differently over time as variables affect performance
  • That allocating all of our budget to one segment won’t work

This means that we need small, incremental changes to get to that perfect balance. For that reason, we use a cap on Doc Oc to ensure we only reallocate 10% of the total budget at any time.

Why 10%?

Because we will be able to reallocate almost half of the overall spend by the end of the month, which is nearly always enough time to get to the right mixed strategy. We could do it faster, but would risk over correction.

What does performance look like after we change our budgets?

Great question! Let’s take a look. We reallocated $30 (10% of our total budget of $300), and the new performance is:

  • Segment 1, Remarketing: $40 Budget. CPA: $8. 5 Conversions.
  • Segment 2, Lookalikes: $190 Budget. CPA: $10. 19 Conversions.
  • Segment 3, Interests: $70 Budget. CPA: $17.50. 4 Conversions.

This is excellent! We got two more conversions (28 versus 26), and the prices for our three segments are coming much closer together. Once those CPA’s are equal (assuming the lifetime value of each customer is the same), we’ll have the right allocation of spend.

Why did the CPA for Remarketing go up, and Interests go down? Did we change anything besides budget?

We did not. This is a case of supply and demand at work:

https://en.wikipedia.org/wiki/Supply_and_demand

See, when we’re ramping up budgets for our Remarketing audience, what we’re really doing is buying more of a finite quantity: Facebook advertising inventory for that audience. As we buy more, we both reduce the supply and increase the demand, which has the effect of increasing costs. Sometimes, our impact is so low that we don’t notice, and prices (CPA’s) don’t change. Other times, they will.

Conversely, the opposite is happening with our Interest audience. By reducing demand, we increase the available supply and lower costs, meaning CPA’s drop.

That makes sense, but how are we deciding how to reallocate money? What determines where dollars come from and where they go?

The multi-armed bandit algorithm:

https://en.wikipedia.org/wiki/Multi-armed_bandit

The algorithm was originally designed to figure out what the optimal allocation of slot machine pulls is based on their respective payouts.

Google popularized this algorithm when they started using it for testing things like websites instead of traditional A/B testing. By reallocating spend (or in their case, the percentage of time a given landing page, or search ad, was shown) throughout the test, they ensured that the best solution was getting leveraged more often, consequently reducing waste and improving performance midtest.

We apply this algorithm to social advertising by using impressions and website clicks as trials, with conversions representing successes. By running millions of simulations to determine the probability of supremacy between various segments, we’re able to determine the optimal allocation of spend.

Because the algorithm doesn’t have a sense of finite supply (I could use the same slot machine any number of times), we limit reallocation to better align with advertising objectives.

Incidentally, we usually use a binary multi-armed bandit or Bernoulli multi-armed bandit algorithm, but cart size can be factored into the trials as well, making it work exactly like the original algorithm with variable payouts.

So Why Doc Oc?

Because Doctor Octopus from Spiderman lore is the original multi-armed bandit.

https://en.wikipedia.org/wiki/Doctor_Octopus

Wow. Just wow.

Right???

Moving on…

Right. So, going back to our campaign, let’s say we’ve been running for a month, and have kept making those same reallocations based on performance. What might it look like at the end of the month?

  • Segment 1, Remarketing: $72 Budget. CPA: $9. 8 Conversions.
  • Segment 2, Lookalikes: $174 Budget. CPA: $9. 19 Conversions.
  • Segment 3, Interests: $54 Budget. CPA: $9. 6 Conversions.

We’re now at 33 conversions, 7 more than our original 26, for a 27% increase in performance with no incremental budget, creative changes, or other optimizations; just incremental reallocation of spend. We’ll have to keep fine-tuning budget allocations, but we’re now free to focus on improving other areas of our campaign.

Can we use this to optimize for…

Sorry to cut you off, but yes, we can use it for anything. If you isolate a variable in an ad set and change that variable across other ad sets, we can optimize for the appropriate allocation.

That was a lot of information.

I know. Sorry, but it’s necessary, as the roots of the problem are as deep as the solution. Fortunately for you, you now have a graduate level class in advertising theory under your belt, and you understand why HYFN has been able to achieve such amazing results for its clients.

That was shameless.

Absolutely. But also true.

Subscribe Here!

Recent Posts

Let's Do Something Great

Smart thinking, compelling design, and a process that flourishes with your contributions.