What if I told you the data in your favorite AdWords report was wrong? (As in, not close, not delayed…just plain wrong.) What if I told you it could be happening to you right now and you wouldn’t know any differently? Well, if you are running any geo-specific KWs in your account today, your geo reports are lying to you.

A few months ago, PPC Associates started working with a start-up looking to advertise a San Francisco-based service. The service has appeal nationally, but we were limited to run on “San Francisco” KW variants to ensure we could fulfill requests. This set of campaigns was part of a pilot program – if we could make it work in the Bay Area, the rest of America was next.

Campaign-wise, we chose a simple structure – one campaign targeting Bay Area residents and another targeting the rest of the country.  You can see the geo settings for the national campaign below:

Nothing crazy here, right? We didn’t bother with intent targeting (for now), so the settings are straightforward – advertise everywhere in America except the Bay Area.

Imagine my surprise when I started pulling geo reports and saw this:

What you are seeing here is the geo data for National campaigns in the last 2 months. In case you missed it, the SF Bay Area represent the overwhelming majority of our spend…even though the region is blocked. Furthermore, the geo data looked almost identical in analytics. Two platforms (thought-connected), same problem.

As you can imagine, I was startled by this and immediately reached out to Google. I fancy myself an AdWords expert, but it’s easy to overlook a settings error. I asked the Google team to review the account and verify I hadn’t missed some setting.  I hadn’t.

There were only two options left at this point.

1)      Google ignored our geo-settings and based distribution on query intent (remember, EVERY query has some reference to San Francisco). This would mean a major targeting bug and a big ol’ refund from Google.

2)      The targeting was correct, but the report was wrong. Given the nature of the queries, it could simply mean that the Google geo report was now IGNORING location and categorizing exclusively on intent.

In order to eliminate option 1, I pulled conversion data from our 3rd-party tracking system and started mining IP data. Though I didn’t include conversion stats in the screengrab, the conversion numbers coming from SF were proportionate to our spend.

The results were pretty clear – well over 50% of conversions were coming from outside California. Only 20% came from San Francisco, with the rest coming from Southern California and Sacramento.  While IP data is fickle, it confirmed that traffic and conversions were not really coming from the Bay Area. This only left option 2.

A couple of emails with Google later, and it was confirmed: the geo report was not using physical location, only query intent. The only reason 100% of the traffic wasn’t reported as Bay Area was that we used tokens like “SF” in our keyword set, and this doesn’t signify enough intent to be re-categorized.

I’m wondering how many of you are running geo-modified terms in your accounts right now. If you are, you’re experiencing the same problem. Assuming you’re running them as part of a more diverse set, you would never know the difference. You would happily go on optimizing against false geo data, making the wrong changes based on incorrect information from what is supposed to be your #1 data source.

When targeting a specific market, losing insights into geo data is crushing. While there are always things to do on an account, the ability to tailor your campaign based on traffic metrics and user intent in different geos is critical. This is especially true for a lean start-up trying to run as efficiently as possible. Given the service is based in San Francisco, users’ location relative to SF is critical. Until this is fixed, we’ll never know how to take advantage. Google won’t acknowledge this as an error, but categorizing intent as physical location is grossly misleading. Make no mistake about it – this is something that needs to be FIXED.

At the end of the day, there is no solution. Your geo report is lying to you, and any help article you find will confirm this. The closest thing I could find to an explanation was on a random help page – buried under physical location exclusion (see below)! I have yet to find a page about the GEO report (under dimensions, fyi) that says anything other than it reports location. If you find otherwise, let me know.

Sean Marshall, Director of Search Engine Marketing
– Questions? Comments? Email us at bloggers at ppcassociates dot com.

9 Comments

  1. Sean November 4th, 2011

    Speaking with a Google product manager today….stay tuned!!

  2. Ariel November 6th, 2011

    Its own note makes the setting rather pointless, doesn’t it? I can think of so many things going wrong when you combine cookies, dynamic IPs and Google’s supposedly “smart” location attribution based on a variety of factors (as hinted in the note on your screenshot).

    This is far from the only inaccurate mode of operation of the Google AdWords system, the Display Network has keyword based targeting problems from what I’m seeing as well.
    It could be a bug based on the discrepancy between keyword state changes and saved placement data in the system but the again it could simply be loosening the matching algorithm (the end result of which is a greater number of -irrelevant- clicks).

  3. Sean November 7th, 2011

    I’d put this in a completely different category as Display network matching.

    The Google display network (GDN) is a blackbox, plain and simple. As SEMs, I think we’re accepting a certain degree of risk (especially compared to targeting methods on google.com) when we turn our targeting over to an algorithm.

    The saving grace of the GDN is that we can get visibility into where our ads our showing up.
    The closest equivalent to this geo-reporting issue would be if your placement reports were telling you your traffic was coming from youtube.com when it was really coming from nytimes.com.

  4. Alex November 6th, 2011

    Hello Sean,

    First of all, I’m happy to see that you’re shining some light on this and quite frankly surprised that there hasn’t been more fuzz about this before. Google has been aware of this fact for a very long time and understand that they need to find a solution. I think it’s ok to say that they are working on an advanced location targeting alpha, that will enable you to target by physical location only (for real this time).

    As an advertiser who targets almost exclusively location based keywords, we are very eager to see if this experiment works. Like yourself, we use a third party analytics vendor which provides us with the necessary data. However, it would be a lot quicker if we could access accurate data in AdWords.

    I should be able to follow up with more information end of next week to share how the test is going. Since it’s a reporting issue and not a serving issue, it’s more of a nuisance than a violation, which is a clear distinction. However, every advertiser should have the ability to accurately track where their clicks are coming from. I think we’re very close to getting more accurate geo reporting.

    Thanks,
    Alex

  5. Sean November 7th, 2011

    Bit confused by what you mean when you say “they are working on an advanced location targeting alpha, that will enable you to target by physical location only (for real this time).” but then follow with “Since it’s a reporting issue and not a serving issue”.

    My understanding is that my ads ARE being served based on location only and that this is only a reporting issue where KW intent overrides location. I don’t know that this necessitates a special location targeting alpha, the system already behaves this way.

    This isn’t a violation but I’d call this more than a nuisance. Calling it a nuisance under-emphasizes the importance of geo data. I think SEMs have a different expectation of what the geo-report is meant to provide vs what it actually does. This is incredibly misleading.

  6. Alex November 10th, 2011

    You’re right, that was a bit confusing. Let me clarify. Google has been providing conflicting information about the actual issue. I just got confirmed that it’s both a serving and reporting issue. Google still conducts query parsing, meaning if you search for new york plumbers, even if you’re in Texas, Google will serve the ad to you even if the advertiser specifically had New York as “physical location” only in settings. In Google reporting, it would say that the users location was New York if that was included in the search query.

    Does that make more sense?

    Based on this knowledge I understand that advertisers are frustrated and angry. If you’re a small business with a tight budget, geo targeting is essential.

    What the new test does is it removes the query parsing out of the equation and actually serves based on location only and reports on click location properly. This is pretty major stuff.

  7. Sean November 10th, 2011

    Major stuff indeed which is why I was shocked that 1) I was not aware of it until now, 2) someone didn’t prove me wrong.

    I think the kicker here was that I was excluding the very region I was supposedly getting traffic from. For this, query intent matching didn’t really matter to me since I was open to get traffic from the entire US.

    If I were a local only advertiser…I’d be less than pleased by this development.

    Here’s hoping for a resolution (posted a follow-up in today’s blog FYI)

  8. Edwin November 7th, 2011

    This is ridiculous – how long has this been going on?

  9. Sean November 7th, 2011

    If I understood Google correctly – always.

Leave a Comment

Sean Marshall
Sean Marshall is the CEO of Intended, an SEM agency founded in 2013 to provide industry-best service for SMB clients. Before Intended, Sean was the VP of Business Development of PPC Associates (now 3Q Digital). He is a huge Cal fan and has been known to win a buck or two playing online poker.