When your search numbers disagree, which one wins?
Some of October's readings were faulty
If your website gets a monthly search report, October's has probably just arrived, or will soon. Before you start new work, celebrate because of something in it or set a budget, it's worth knowing that some of the tools behind many of these reports gave faulty readings last month. The faults aren't in your website, and neither Google nor OpenAI has yet said anything publicly about any of them.
For a week, many visits from ChatGPT were counted without their source
For about a week from 8 October, visits from links in ChatGPT's answers often arrived without the two things that would have told Google Analytics where they came from. One is a tag that ChatGPT adds to its links, and the other is the referrer, the address of the website the visitor clicked from. Without them, Analytics files the visit under Direct, its label for a visit it can't trace. In fact, one measurement company, Profound, tracked how many of ChatGPT's answers with citations carried at least one tagged link: about 98% on 8 October, and 66% on 14 October.

Cedric Huesler, who measured the shift in visits from ChatGPT to Direct, has since said it was a bug, and that OpenAI told him so. He says the tags started coming back on 15 October. OpenAI hasn't said anything publicly. But the visits that were already counted as Direct are unaffected, so that week is still in your October figures. If your report shows ChatGPT visits falling and Direct rising on the same pages, much of that is a change of label, not lost visitors.
That isn't the same as AI quietly sending you visits you can't see. The first is a count of visits that changed label. The second is a theory: that AI tools send you visits Analytics can't trace back to them, so it files them under Direct (they're never counted as AI). Kevin Indig went looking for that traffic in the Google Analytics data of 12 websites, over August and September, and he couldn't yet detect it. It's the question I asked about a much-quoted 60% in September: what was counted, and out of what?
Some rankings are the tracker's, not yours
Many reports include positions from a rank tracker, a tool that checks where your website appears in Google for a fixed list of searches. Since late summer, several of these tools have been collecting results pages where the usual websites vanish and then come back. Semrush, one of the companies that make them, has acknowledged the problem, though it says the effect on rankings is limited and mostly beyond the first page. Google hasn't commented.
Look out for a drop that reverses the next day, or one where most of your usual competitors vanish at the same time—it's probably the tracker.
Ahrefs, which also makes a rank tracker, says it's cleaning up any results it has already collected that may have been affected. So if your rankings come from Ahrefs and October's report compares them to September's, the September figures may not match the report you were sent last month.
Which source wins when two disagree
Search Console, Google's dashboard for website owners, isn't infallible either. It was reported running more than two days behind on 11 October, though it had caught up by the next day, and Google hasn't posted a notice about it.
Before anyone looks at the figures, it's worth agreeing with whoever prepares your report on the source you'll both trust. Once a number has moved, whichever source shows the reading you'd prefer starts to look like the reliable one. It may not be.
For anything to do with Google search, the best candidate for most websites is Search Console's clicks, Google's own record of people clicking through from its results to your website. Google counts these clicks itself, so they don't depend on a rank tracker or on the tool your website uses to count visitors. Its impressions, how often your website was shown in its results, belong beside them, with a caveat. They include your appearances in Google's AI answers, as I explained in the last issue, so a rise may come from more of those appearances rather than from your website being shown more often in Google's ordinary results.

What's worth asking for
- A ranking drop, checked in Search Console. Before anyone acts on an alert from a tracker, it's worth asking whether Search Console shows clicks and impressions falling too, for the same search, on the same date, in the same country and on the same device. If they don't, the alert may not be trustworthy.
- ChatGPT and Direct, side by side. It's worth seeing both for each of your main pages, over periods that compare like with like, such as the week from 8 October and the week before. If one fell and the other rose by much the same amount, probably nobody went missing.
- A dated note beside every figure these faults could have touched. On the report itself, whether or not the fault has been confirmed. It's the same habit as in my post on August's Search Console numbers: writing the event down against its date, while you still know what it was.
Thanks for reading
David
If you want to talk about what your search numbers are actually telling you, please contact me at [email protected].