Google’s Chief Economist, Hal Varian, noted in a post on the AdWords blog today that conversion rates are pretty much the same across positions on AdWords. To quote the post:

… on average, there is very little variation in conversion rates by position for the same ad. For example, for pages where 11 ads are shown the conversion rate varies by less than 5% across positions. In other words, an ad that had a 1.0% conversion rate in the best position, would have about a 0.95% conversion rate in the worst position, on average. Ads above the search results have a conversion rate within ±2% of right-hand side positions.

In the great scheme of things, I suppose this is possible. If you are analyzing millions or billions of pieces of data, I imagine that the data just normalizes down to small percentage differences between different positions.

The problem here, however, is that the data set is too big to draw real conclusions. For example, if I told you that 95% of Americans make between \$0 and \$166,000, and that 60% make between \$19K and \$91K you might conclude from this data that wealth is evenly distributed in the US, right? But then if I told you that 10% of the US population controlled 71% of the wealth in America, and 1% controlled 38%, you might think of the data in a different light.

Here’s a different way to think about the Google conversion rate data that I think would probably have a much different outcome. Do a conversion rate analysis by “token length”, which is search engine language for the number of words in a search query. If someone types in “baseball” for example, what’s the conversion rate differential between position #1 and position #10, versus a query for “buy Louisville slugger size 28 wooden baseball bat.”

I suspect that the conversion rate for the first query is going to be very low for the first position, simply because you are going to have a lot of browsers who simply click on the first ad that they see. Any browser who eventually makes it down to position #10’s result may very well have turned into an actual shopper after clicking on all the other ads. Conversely, if you already know the exact product you want – and your search query indicates that intent – you are much more likely to convert on the first ad you see that actually offers the specific product you want.

Suffice to say, I’ve seen plenty of examples where conversion rate does vary dramatically by position. To make a general statement that position does not factor into the equation is something that, well, a smart economist who loves statistics but doesn’t really understand SEM might be apt to conclude.

Postscript: for those of you wondering about the picture at the top of the post, it’s a scientist explaining why the Space Shuttle Challenger exploded. For a great explanation of why statistics and the presentation of statistics failed to alert NASA that this disaster would occur, check out Edward Tufte’s book, Visual Explanations.

2 Comments

1. tdwhalen August 24th, 2009

David, I think you are probably on to something with your idea to parse by keyword length.Overall, I think you need an absolute TON of data to look at this – I don’t know of many agencies outside of maybe EF that have enough data. Part of the reason you need a ton of data for this is that you have to negate all the other reasons for changes in conversion rates, including ad rotation, changes on the landing page or to the funnel, conversion tracking changes, etc. With enough data from enough different accounts all the other variables can kind of cancel themselves out. But with smaller datasets, you’re likely to come to erroneous conclusions.In addition, I think the only data that really matters is the data of the specific advertiser. It may be true that in aggregate, conversion rates don’t vary a whole lot by ad position. But for specific advertisers, or possibly for specific categories, this may not be true. The problem is that for specific advertisers, there usually isn’t enough data from the account to form any conclusions about conversion rate by ad position.

2. David Rodnitzky August 25th, 2009

I think that's my problem with this data – it is making a conclusion that is supposed to apply equally to all advertisers in all verticals. That would be like assuming that consumers act the same way when they are buying a luxury yacht versus signing up for a free newsletter – you can't make blanket statements that fit all like that.

David Rodnitzky is founder and CEO of 3Q Digital (formerly PPC Associates), a position he has held since the Company's inception in 2008. Prior to 3Q Digital, he held senior marketing roles at several Internet companies, including Rentals.com (2000-2001), FindLaw (2001-2004), Adteractive (2004-2006), and Mercantila (2007-2008). David currently serves on advisory boards for several companies, including Marin Software, MediaBoost, Mediacause, and a stealth travel start-up. David is a regular speaker at major digital marketing conferences and has contributed to numerous influential publications, including Venture Capital Journal, CNN Radio, Newsweek, Advertising Age, and NPR's Marketplace. David has a B.A. with honors from the University of Chicago and a J.D. with honors from the University of Iowa. In his spare time, David enjoys salmon fishing, hiking, spending time with his family, and watching the Iowa Hawkeyes, not necessarily in that order.