Your Geo Reports are Lying to You
Published: November 2, 2011
Author: Sean Marshall
What if I told you the data in your favorite AdWords report was wrong? (As in, not close, not delayed…just plain wrong.) What if I told you it could be happening to you right now and you wouldn’t know any differently? Well, if you are running any geo-specific KWs in your account today, your geo reports are lying to you.
A few months ago, PPC Associates started working with a start-up looking to advertise a San Francisco-based service. The service has appeal nationally, but we were limited to run on “San Francisco” KW variants to ensure we could fulfill requests. This set of campaigns was part of a pilot program – if we could make it work in the Bay Area, the rest of America was next.
Campaign-wise, we chose a simple structure – one campaign targeting Bay Area residents and another targeting the rest of the country. You can see the geo settings for the national campaign below:
Nothing crazy here, right? We didn’t bother with intent targeting (for now), so the settings are straightforward – advertise everywhere in America except the Bay Area.
Imagine my surprise when I started pulling geo reports and saw this:
What you are seeing here is the geo data for National campaigns in the last 2 months. In case you missed it, the SF Bay Area represent the overwhelming majority of our spend…even though the region is blocked. Furthermore, the geo data looked almost identical in analytics. Two platforms (thought-connected), same problem.
As you can imagine, I was startled by this and immediately reached out to Google. I fancy myself an AdWords expert, but it’s easy to overlook a settings error. I asked the Google team to review the account and verify I hadn’t missed some setting. I hadn’t.
There were only two options left at this point.
1) Google ignored our geo-settings and based distribution on query intent (remember, EVERY query has some reference to San Francisco). This would mean a major targeting bug and a big ol’ refund from Google.
2) The targeting was correct, but the report was wrong. Given the nature of the queries, it could simply mean that the Google geo report was now IGNORING location and categorizing exclusively on intent.
In order to eliminate option 1, I pulled conversion data from our 3rd-party tracking system and started mining IP data. Though I didn’t include conversion stats in the screengrab, the conversion numbers coming from SF were proportionate to our spend.
The results were pretty clear – well over 50% of conversions were coming from outside California. Only 20% came from San Francisco, with the rest coming from Southern California and Sacramento. While IP data is fickle, it confirmed that traffic and conversions were not really coming from the Bay Area. This only left option 2.
A couple of emails with Google later, and it was confirmed: the geo report was not using physical location, only query intent. The only reason 100% of the traffic wasn’t reported as Bay Area was that we used tokens like “SF” in our keyword set, and this doesn’t signify enough intent to be re-categorized.
I’m wondering how many of you are running geo-modified terms in your accounts right now. If you are, you’re experiencing the same problem. Assuming you’re running them as part of a more diverse set, you would never know the difference. You would happily go on optimizing against false geo data, making the wrong changes based on incorrect information from what is supposed to be your #1 data source.
When targeting a specific market, losing insights into geo data is crushing. While there are always things to do on an account, the ability to tailor your campaign based on traffic metrics and user intent in different geos is critical. This is especially true for a lean start-up trying to run as efficiently as possible. Given the service is based in San Francisco, users’ location relative to SF is critical. Until this is fixed, we’ll never know how to take advantage. Google won’t acknowledge this as an error, but categorizing intent as physical location is grossly misleading. Make no mistake about it – this is something that needs to be FIXED.
At the end of the day, there is no solution. Your geo report is lying to you, and any help article you find will confirm this. The closest thing I could find to an explanation was on a random help page – buried under physical location exclusion (see below)! I have yet to find a page about the GEO report (under dimensions, fyi) that says anything other than it reports location. If you find otherwise, let me know.
– Sean Marshall, Director of Search Engine Marketing
– Questions? Comments? Email us at bloggers at ppcassociates dot com.