new-quality-score-300x294

Larry Kim recently posted on the Wordstream blog about Quality Score having more importance than ever. Typical of Larry’s posts, the article was thorough, thoughtful, and compelling.

And I’m going to tell you why I’m ignoring it.

Quality Score is a Google-driven metric that is only loosely related to performance. Basically, it’s the wrong metric to chase – and on top of that, the metric itself is flawed. Let me break this down, bullet-point style. (Brace yourselves.)

Why is QS a flawed metric?

  • Google openly says that the “true” QS is more granular than the visible one.
  • Although Google says that QS is evaluated in real time, they don’t say that the UI-displayed QS is evaluated in real time, nor do they give the frequency with which the visible one is updated.
  • The above two points in isolation mean that whatever number you’re using to calculate “QS findings” has some flawed, inexact numbers for the QS itself. It’s fuzzy math, and I don’t need to tell you that’s one of the most toxic terms in SEM.

Why is QS the wrong metric to chase anyway?

  • It’s inarguable that QS and performance are related. There is a correlation. But the rigid assignment of any metrics showing lower CPCs is false data. Adding assumptions based on account data is faulty logic; it’s like saying, “Hey, there’s a correlation between higher-impression keywords and higher-conversion keywords” and coming up with numbers showing that driving more impressions on everything will get you X more conversions.
    Correlations are likely driven by your bidding strategy. So you’re combining flawed numbers with faulty logic and getting drivel.
  • If you smooth results with a big data set, the assumed correlation between higher QS and lower CPC is there. (Here comes the but.) But if it doesn’t work on the level of single keywords or keyword sub-sets, you’re just creating bigger pools of faulty data when you apply predictive numbers.

Here, by way of example of a false correlation, is a sub-set of my highest-performing keywords in a certain campaign. These are “super alphas,” in our Alpha Beta method of campaign structure (fill out a little form and download it; it’s worth it), which means they’re golden keywords that show in top positions 100% of the time. And they show a reverse correlation between higher QS and lower CPC or CPM:

reverse qs correlation
  • The stronger correlation is between higher CTR and higher QS…and this is so close that I propose the QS follows the CTR. In most cases (but not all), you should be aiming for better CTR anyway, so just go after CTR.
  • Except when: You have those cases when lower CTR is desirable – e.g. you’re eliminating unwanted clicks through exclusionary ad text focusing on high costs, OR you’re chasing tangential keywords with low CTRs but great conversion rates. If you’re optimizing for QS, you’re not using those highly valuable strategies.

The idea that QS is directly tied to true performance metrics is flawed. It’s like calories; it’s a correlated measure of something real that exists in a complex system of variables.

If you aim to lose weight, lowering calories is a good thing to do; likewise, increasing QS is a good tactic for secondary optimization benefits (more impressions, lower CPCs). But, like calories, QS is not formulaic. There’s some metabolism factor at play there, and just as you can’t say that eating 1,900 calories makes you gain half a pound, you can’t say a QS bump of 1 correlates to any specific lower CPC.

Yes, do the best practices (quick load times, good CTRs, highly relevant LPs). But understand that chasing QS – and especially creating flawed formulas on the impact – is a not only a waste of time but is selling clients on a bunch of hot air.

Susan Waldes, Director of Client Services
This entry was posted in Featured, Google. Bookmark the permalink.

12 thoughts on “Why I don’t optimize for Quality Score

  1. I was recently on a call with our agency Google rep in which he mentioned a very strong correlation between quality score and bounce rate on website – with the new columns for GA data it’s possible to see a comparison for that as well as CTR

  2. Quality score is definitely a factor that should be part of your optimization efforts. But is by no means the end all, be all metric for PPC. At the end of the day it is all about ROI. Is the traffic you are driving converting profitably? If so, than you are on the right track.

    A perfect example of this is a campaign I ran for a large auto insurance provider. We bid heavily on competitor brand names and this resulted in extremely low quality scores. I am talking straight 2’s. Since there was low competition around these KW’s our CPC’s averaged around $11. Unbranded keywords like ‘cheap car insurance’ or ‘auto insurance company’ could have CPC from $18-$40. At the end of the day, we found our competitor campaigns (with low QS’s) to provide better ROI than some of our unbranded campaigns.

    Sure QS’s were low, but they worked for the business. Which ultimately what advertising is about.

  3. @Justin Love it! Competitor keyword strategies are another great example of when low QS (and usually low CTRs too) are tactically “good”.

  4. QS is the main factor I take into account when optimizing my campaigns and because of this I was able to increase profit per unit for a client who worked with 3 other companies before finding me. In my experiences QS updates within a 24 our time period. And it really does go up. Interestingly enough the other consultants didn’t know about DKI so a combination of reorganizing the campaigns and using DKI did the trick. We went from 3/10 to 10/10 overnight.

  5. I have a love/hate relationship with QS. You make some GREAT points in this article about the reasons the metric is flawed.

    I think the way to properly treat and utilize the QS metric is to watch the overall improvement over time to tell show that you are optimizing. Another excellent way that I have found QS helpful is to check it against cpc and conversions to see which keywords to focus on after I have statistically significant data to back up my decisions.

    I totally agree with Justin that you should never pause a keyword based on the simple fact that you have a low QS and have to pay more per click than the next guy. Ya gotta look at other metrics.

    QS simply helps you determine where you can optimize for relevancy. Over time it should always improve.

  6. Hi- i already acknowledged the differences between real and visible QS. the chart was meant to be an illustration, so not sure why this is being introduced into evidence. I’ve looked a few billion of dollars of spend and there is a very strong relationship between ROI in terms of metrics like CPC, cost per action, conversion rates, etc. and Quality Score.

    Agree that sometimes if you look at a measly 2900$ of spend (which by the way, seems like a ridiculously small data set to be making sweeping conclusions about PPC strategies, but anyway) that the relationship doesn’t always hold true for every keyword in every instance, and i certainly didn’t mean to imply that that was the case so it sounds like we can agree on that. I’m just saying that if you take a step back, on average, Quality Score is the game that Google is playing, and on average it is the path to ROI (via lower avg. CPC’s higher avg. conversion rates, lower CPA’s, etc.)

    • The way I often think about Quality Score is that if you do everything you’re already supposed to be doing, then you are working to improve QS. It’s possible to do all the right things without ever really focusing on the visible QS metric at all.

  7. @Larry Kim
    “there is a very strong relationship between ROI in terms of metrics like CPC, cost per action, conversion rates, etc. and Quality Score”

    But the question is, is this causation or correlation?

    • Yeah, to me, there’s really not much clarity in statements about correlation.

      We can take a bunch of keyword data, and of course there will be a correlation between high QS and high ROI (and/or low CPAs, high conversion rates, low CPCs). You will get that all the time – but that isn’t saying much. It may be the case that QS=10 keywords do great – that’s because – to take an extreme, but very prevalent scenario – the majority of those keywords are *brand* keywords. They have high CTRs, low CPCs, (hopefully) high conversion rates, and low CPAs. They are brand. There is great continuity between user intent and what the advertiser delivers.

      It may also be the case that QS=7 (sorry, not many 8s or 9s to be had – heh) keywords might be keywords like ‘car insurance’, where the intent is targeted and commercially-focused, and where there are advertisers who offer exactly what the user is looking for – car insurance. And if the car insurance advertiser happens to have a compelling offer or promotion, then of course – without thinking about QS – that advertiser should put their best foot forward, and include the compelling promo offer in the ad text, and point to a fast-loading page that delivers what the ad promised and what the user was looking for. The QS correlation just isn’t that interesting, and it doesn’t bring to mind a bunch of optimization action items that we wouldn’t already be thinking about were there not a discrete metric called quality score.

  8. If QS is defined by a series of other metrics – CTR and Bounce Rate – then putting any sort of emphasis on those metrics can be defined as “optimizing to QS”.

    Every marketer should be looking for ways to minimize bounce and increase CTR without damaging CVR. You can call that process your secret sauce to optimizing QS but I find it to be a total misnomer.

    You’re optimizing CTR, BR and CVR and just happen to be tracking QS improvements in the process.

  9. I agree that quality score is the wrong metric to chase when looking at short term performance but I have done some research that very strongly points to a connection between a lower quality score and a higher CPC (and vice versa).
    I tracked daily quality score for many keywords and also tracked the min recommended CPC for page 1 and for top position. This way I dont have to account for bid strategy and position effects on CPC. If you run a linear regression
    % change in Top of PAge CPC = alpha*change Qs + C
    you get a very stable estimate that change in Qs of 1 leads to a 7.7% change in CPC . Note that I am tracking the same keyword’s change in QS. So I am convinced that change in QS does lead to a change in CPC. Now the question is if it affects performance. Here things get very hazy. For Google QS is a function of CTR (85% of QS is explained by CTR if you run the numbers for Google. For Bing the CTR /QS connection is not strong… an interesting finding worthy of a blog post :) ) . However,high CTR does not lead to high conversion rate all the time. Hence the connection between QS and ROI is not strong. My POV on all this is, do not look at QS for bid strategy, but for long term account improvements QS is a good metric to look at. I speaking about my findings at SMX Advanced :)

Leave a Reply

Your email address will not be published.


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>