You’ve probably seen reports that tell you how your website ranks on Google for specific keywords. Some use these to measure the success or otherwise of their SEO efforts. While these reports can help us understand trends and detect serious issues, it is important to remember that they’re based on lies.
To understand why the reports are lies, we need to know where the numbers come from. There are two common sources.
Search on Google
One approach is to actually type the keyword into Google, then look through the results to see where your website is mentioned. There are many programs and services that can automate this for you.
On the face of it, this seems reasonable. And a decade or more ago, it was a perfectly sensible thing to do. But search has changed. The results you see are influenced by many things specific to you, such as your location and what other searches you’ve made.
So searching yourself is not going to show you what other people experience. Indeed, there’s a risk that your website will be ranked higher when you search than when others search: Google may have worked out you are interested in your website, and therefore be more likely to rank it higher when it’s you searching.
Some people work around this issue by using a proxy and changing certain browser settings when searching. This changes their location and hides their history, removing the "it’s me" search effect. However the results still don’t reflect reality; Google’s showing results customised for a searcher who has never used Google before, searching from a new location. Which probably isn’t your target audience.
Assume your business is a local business, drawing its customers from a handful of suburbs in a major city. Also assume Google knows your business is based in My Suburb.
You obtain your ranking report, showing you rank number 11. You are unhappy about that. But that rank is measured while pretending to be some distance away from My Suburb, perhaps even on a different continent (those who are aware of the issues normally make a point to search from somewhere near My Suburb, but in Australia at least it is often difficult to target more specifically than continent).
At the same time, you may well be ranking number 1 for searches by people who live in My Suburb. Which is exactly what you want. Adjusting your website to improve your ranking based on the ranking reports carries the risk of worsening your ranking for searches made by your target audience.
You are told that your website is finally ranking number 2 for a specific keyword. With excitement, you search yourself but can’t see your website on the first page of the results.
You assume it’s because of where you’re searching from, or your personal search history, so you decide to obtain a current ranking report. It says your website is now ranking at position 8.
Which report was correct? Are you number 2, 8, or invisible?
Search engine indexes are dynamic, meaning they are constantly updated. Other inputs into the search results algorithm can also change minute by minute. A ranking report is a snapshot at a moment in time. Therefore there is no guarantee that you’ll obtain the same results even a minute later.
If you ensure as many inputs as possible are kept the same, you’ll minimise the variation. But it will still be there.
What this means
If your keyword ranking reports are gathered the way most are, by searching on Google from a different location while pretending to be someone who’s never before used Google, then the results will not reflect reality. They will be lies.
Does that mean these reports are worthless? Certainly not. For one thing, they are very useful for detecting serious issues. By always fetching the data with a known stable setup that isn’t influenced by you going on holiday or searching for new things, you obtain a baseline. Should something go wrong with your website, such as a Google penalty, unintentionally blocking Google, malware installation or speed issues, you’ll see the rankings in this type of report drop significantly, and not recover quickly.
But resist the temptation to base your website optimisation decisions solely on a keyword ranking report.
Search Analytics in Search Console
Another source of ranking data is based on the Search Analytics section in Google’s Search Console.
Search Analytics tells you all sorts of helpful information about how your website is faring in Google search. For instance, how many times your website was displayed in what position on what day and device for which search queries from which country. And which page on your website was the one suggested to the searcher.
At Coherent Digital we spend a lot of time with this tool, because it can tell us so much about how Google views your website, and what Google thinks each page is about.
But like all data, it has limitations. One of the limitations can become a serious issue when one focuses on the overview numbers, such as average position in search results for specific general keywords. That’s where the lies start to appear.
For one month, Search Analytics reports an average position of 18.2 for an important keyword. Clearly this isn’t good enough. Or is it?
Digging further shows that on any one day during the month, the position varied markedly. For example, on the seventeenth of the month the website’s home page was displayed at position 2 for some searchers, and position 96 for other searchers.
Investigating even further revealed that the website actually ranked 2 to 4 more than half of the time on that day. Checking other days at random revealed a similar story.
In fact, across the month, two thirds of the reported searches had the website in the top half of page one of the search results. The other third mostly had the website ranked position 45 or worse. It was rare for the website to appear anywhere between 5 and 45.
My belief is that most of the searchers who saw the website at position 45 or worse were very unlikely to buy from the website. Perhaps many of them were automated searches and not human, while others were people looking for other reasons. I think Google can often (though not reliably) tell the difference between a genuine search looking for a relevant website and a search intended to check ranking or research competitors.
By the way, the numbers quoted here are from a client’s Search Analytics data.
What this means
Summary numbers, such as averages, can be very misleading if the underlying data contains a few outliers.
Which would you prefer, 9 out of every ten search results at position one, or an average position of 11?
It’s possible to have both, at the same time. All you need is for that tenth search to put you near position 100.
Remember that when looking at the overview numbers in Search Console (or anywhere else, really). To get maximum value from the Search Analytics data, you’ll need to dig deep. And understand what you’re looking at. Because although the data in Search Analytics may not be lies, it can still lie to you.
Search engine ranking reports are an important tool in SEO. They help in many ways. They can be used to monitor the health of the website. They can provide a valuable insight into a search engine’s understanding of a website’s topics, as well as the search engine’s idea of the importance of specific pages. In short, ranking reports are an important tool in an SEO practitioner’s tool box.
It’s when they’re treated as a conclusion, or their data is the major measure of the success of a website, that a ranking report can become a serious liability. Because when used for those purposes, it becomes a report of lies. Decisions based on lies are no better than guesses, and may even be worse.
We have collected, and continue to collect, many examples of the lies and traps one can find by not looking properly at Search Analytics data. I plan to discuss these on this blog over the coming months.