Bid Management Algorithms Demystified
Published: October 1, 2008
Author: David Rodnitzky
During every bid management software sales pitch, the PowerPoint deck inevitably gets to the section on bid management technology. The sales reps proudly proclaim that their brilliant engineers have developed an incredible technology that will not only increase your ROI by 40% in days, but also cure cancer and solve world peace. As proof of the sophistication of this technology, they’ll wow you with complex words like “algorithm”, “portfolio management,” “patent-pending”, and “proprietary code.” Most search marketers at this point in the meeting either nod their head and feign understanding, or start to daydream about their weekend plans. Indeed, if you actually deigned to ask a question about the technology to the sales rep, he probably would have no idea how to answer your question anyway. In sum, most discussions of bid management technology are lost on all parties to the conversation – both marketers and salespeople.
It doesn’t have to be this way! My goal in this article is to outline the basics of bid management science, so that you can a) know what different terms mean; and b) ask the right questions to get the right answers to help you choose the right bid management software for your business.
At a high level, almost every bid management technology falls into one three basic approaches: portfolio management, rules-based optimization, and positional optimization.
Portfolio management is perhaps the oldest and most-well known approach. A portfolio management bidding program optimizes your entire search program against a specific business objective. For example, let’s say your goal is to maximize your company revenue but with a constraint of at least a 20% profit margin. A portfolio bid management system might set bids to achieve 40% margin on some keywords, 10% on others, and 5% on others, with the goal of giving you a blended average of 20%.
You might be asking “if I want a 20% margin, when would it ever make sense to bid for a 5% margin?” Consider a keyword that might only get 10 clicks and $10 of revenue at a 20% margin, but 500 clicks and $5000 of revenue at a 5% margin. In the first case, you would only end up with $2 of profit dollars but in the second (due to the fact that you are bidding higher) you would get $250 of profit dollars. If you could then offset this keyword by another keyword that gets 40% margin, you would end up with more revenue and still hit your profit dollar goals.
In my experience, portfolio management technology works best when you have a specific, non-profit based goal for your search marketing. This would include goals like “maximize revenue within a given budget”, “maximize leads within a budget”, or “maximize revenue with a specific margin constraint.”
The next type of bid management technology is rules-based optimization. Unlike a portfolio-based approach, a rules-based optimization strategy is centered on keyword-level bidding. As with portfolio management, you start by establishing your business objectives for your search campaign. A rules-based algorithm (note: algorithm is basically a fancy word for the overall program that determines bids) then applies your objectives to every keyword in your account. Initially, it starts with keywords that have lots of clicks – enough to make them “statistically significant” (i.e., there is enough data to be reasonable confident that the keyword’s past performance is indicative of its future performance). If you have an objective of 20% margin, and you have a keyword with 1000 clicks at $1 each and overall revenue from these clicks of $2500, the system would perform a calculation like this:
Total revenue from clicks = $2500; Revenue per click (RPC) = $2.50; Margin Goal (MG) = 20%; Bid = RPC(1-MG). Bid = $2.50(1-.2). Bid =$2.5(.8). Bid = $2.00
The system then might move on to groups of keywords that , taken alone don’t have statistically significant data, when grouped together can be bid collectively. This is known as “clustering”, and most clustering occurs either as result of “semantic clustering” (the keywords are similar words) or “behavioral clustering” (user respond to these keywords similarly).
Rules-based bid management is probably the most common technology found in bid management systems today. This is probably because it is relatively easier to develop and generally works well across many different business objectives.
The final bid management approach is called positional optimization. Positional optimization (as the name implies) attempts to find the ideal position in the search results to achieve your business objective. This is far and away the most complex bidding system and deserves a little extra explanation.
If you think about all major paid search programs today (Google, Yahoo, MSN, Ask, etc), they all determine position on their search result page (SERP) based on three variables: Maximum cost per click (CPC), click through rate (CTR) and Quality Score (QS). If you remove QS from the equation, you are left with CPC X CTR, which actually results in a cost per thousand impressions (CPM) system. In other words, the advertiser that shows up in the first position on the page is the advertiser that generates the most revenue per impression for the search engine, not the most revenue per click.
Not surprisingly then, if you looked at the amount an individual advertiser needed to pay for positions 1 through 10 on a SERP, you would see that the effective CPM (eCPM) drops almost exponentially by position. For example, you might have to pay an eCPM of $500 for position #1, $300 for position #2, but at position #5 the eCPM may only be $20. That’s because both the Max CPC and CTR are much higher in the higher positions.
Positional optimization algorithms combine an understanding of eCPM by position with the amount of revenue the advertiser makes by position. Revenue by position is determined CTR multiplied by conversion rate multiplied by revenue per conversion. When you multiple these three factors together you get your revenue per thousand impressions (RPM). For example, if in position #1 you have a CTR of 10%, a conversion rate of 10%, and a revenue per conversion of $50, you would end up with an RPM of $500 (1000X.10 /10 X $50). If the same numbers in position #2 were 5%, 5%, and $40, your RPM at position #2 would be $100 (1000X.05 /5 X $40).
When you subtract your eCPM from your RPM, you are left with your earnings per thousand impressions, or EPM. For example, if the eCPM for position #1 was $90 and the RPM was $500, your EPM for position #1 would be $410. If in position #2 your eCPM was $30 and your RPM was $460, your EPM would be $430. In such a case, it would be smarter to set a bid to achieve second position on the SERP, as the overall profit is higher.
This sort of optimization is very difficult to perfect for many reasons. First, it is hard to predict the right bid to achieve a specific position. Second, the system must account for competitors changing their bids. Third, the system must understand the difference in volume by position. Fourth, most keywords do not have data at every position (or even most positions) which requires the system to make predictions based on limited data.
At the end of the day, however, I believe that this is the most powerful algorithm of the three discussed, simply because it makes bid adjustments based on how the search engines themselves optimize results. In a traditional rules-based platform, you would assume that reducing your bid would increase the difference between your revenue and cost, thus increasing profit. But it is very possible that a bid reduction might move you into a lower position where the RPM dropped at a faster rate than the eCPM, resulting in a decline in EPM. If you don’t optimize the same way the search engines optimize, this scenario is bound to happen.
9 Questions to Ask A Vendor About Bid Management Technology
To complicate matters further, the truth is that most bid management systems today combine elements of more than one of the above theories into their bid equations. As a result, if you ask a vendor “are you rules-based, portfolio-based, or position-based”, they may very well explain to you that they are a hybrid of all three (though again, if you are talking to a sales rep, you are more likely to just get a blank stare anyway!).
Nonetheless, I think you need to ask the question and see what comes back. Here’s my list of nine questions you should ask to determine whether a specific bid management technology is right for you:
1. What type of bid management algorithm is this (portfolio, rules-based, positional, hybrid)?
2. How do you deal with keywords that don’t have enough statistical data?
3. Do you take into account day-parting and geo-targeting?
4. Do you factor in Quality Score? How?
5. Can I set objectives based on revenue? On profit? On leads? On budget? On a combination of the above?
6. Can your system accept historical data (prior to implementation) and use this to “learn” so that it can make smart bid adjustments from day one?
7. Can I manually override your technology for specific keywords/accounts/days?
8. If I wanted you to optimize for profit, how would your system do it?
9. Can you have one of your engineers walk me through the algorithm?