It’s Time to Start Thinking of Google Search as Infrastructure, not a Product
Hi friends –
It’s hard to imagine now, but back in 1998, there were so many search engines on the market that I couldn’t be bothered to meet with the Google founders, Larry Page and Sergey Brin when they offered to come by my offices at the San Francisco Chronicle.
I was a young reporter who had been hired by the Chronicle business desk in part because I had grown up as a computer nerd. Newspapers like the Chronicle needed a new class of tech savvy journalists to understand the Internet revolution.
Web startups were constantly pitching me to write about their offerings for my weekly feature called “website of the week” where I profiled interesting websites such as Craigslist.
“Who needs another search engine?” I remember thinking, as I turned down the Google PR person’s pitch to meet the founders, who were then Stanford grad students. “There are so many good search engines already,” I thought. My favorite was Alta Vista, but I also used Lycos and Infoseek and Excite and many others.
Of course, we all know how this story ends. Google won the search engine competition – at first, by providing better faster results – and later, by protecting its market dominance with business deals that blocked other search engines from the market. Now, there is virtually no competition in search - Google controls an estimated 90 percent of the market.
Last year, it seemed that Google’s monopoly might be curbed, when a judge ruled that Google had illegally abused its monopoly in search. But such hopes were dashed earlier this month when the judge proposed the meekest of punishments for Google, further entrenching its dominance.
In my latest piece for New York Times Opinion (gift link), I argue that this is a perilous time to be granting one company so much control over global access to knowledge.
What Happened?
Probably the most shocking revelation of the antitrust trial against Google was the fact that the company had paid $26 billion for just one year of guarantees that its search engine would be the default on iPhones, Samsung phones and other devices.
In his opinion last year, Judge Mehta repeatedly referenced the payments as evidence that Google had illegally abused its monopoly power to block competitors. But then, in a shock to court watchers, Judge Mehta did not follow up this year by banning Google from making these payments.
The judge’s reasoning? Competition from AI bots. “The emergence of GenAI changed the course of this case,” Mehta wrote in his 230-page opinion. In the trial last year, he wrote, no witness testified that AI bots posed a threat to Google. But a year later, when the court debated remedies, AI was “front and center as a nascent competitive threat,” he wrote.
As a result, Judge Mehta wrote, his goal was largely to ensure that “Google’s dominance in search does not carry over into the GenAI space.”
To spur AI competition , Judge Mehta proposed a very narrowly tailored program that would require Google to offer competitors access to some of Google’s data - but only in limited infrequent batches. And he gave Google control over three of the five votes on the committee that oversees the data spigot.
His proposal has been widely criticized as inadequate. Jonathan Kanter, the former head of the antitrust division at the Justice Department, called it “deflating.” “The message to other companies is plain: It pays to break the law,” he wrote in a New York Times op-ed.
What’s the Problem?
On the surface, it can look like Google faces competition from AI in search. OpenAI, Microsoft, Anthropic, and X have all added real-time search results to their AI products, while Perplexity, an AI search engine, has claimed to process more than 400 million queries per month.
But that’s a mirage. AI search is built on top of existing search capabilities. AI models are built on static piles of data that allow the machine to “learn” patterns from the data. Search engines are built on dynamic and constantly updated piles of data that is ranked using information from years and years of search queries.
AI companies cannot build up the same type of updated data piles because many websites block their scrapers. Websites welcome Google’s scrapers because they want to show up in Google results.
And Google’s advantage is not just the website data that it scrapes, but all the information Google has learned about those sites over decades. Do people linger on certain websites longer? That’s a sign of a high quality site. Do people click onto a site and immediately bounce away? That’s a sign of a potentially disappointing site.
The judge wrote in his opinion that it would take Microsoft’s search engine Bing 17 years to collect the same amount of user data about website quality that Google collects in 13 months. This is Google’s real secret sauce – and it is this valuable data that the judge largely declined to offer to competitors.
Without frequent access to Google’s data and special sauce, it’s very unlikely we will see meaningful competition to Google arise.
Why Does it Matter?
This is a dangerous time to entrench Google as the gatekeeper to the world’s knowledge. We are in the midst of a shocking governmental assault on free speech.
The Trump Administration is distributing lists of banned words, detaining and imprisoning students for speech it disagrees with, and persecuting the media and comedians for insufficiently sycophantic coverage.
It's likely just a matter of time before the administration starts pressuring Google to remove objectionable words or suppress political dissent. Already Google has removed the autocomplete suggestion for “impeach Trump.”
Imagine if the administration pressures the company into suppressing search results for groups that it deems “domestic terrorists” such as the Jewish billionaire George Soros’ foundation that the Department of Justice has just launched an investigation into.
This is not a far-fetched scenario. Google does exactly this type of suppression in other countries. In Vietnam, where the law prohibits content “opposing” the government, Google removes all sorts of material critical of the government. In Thailand, Google complies with laws prohibiting content that is critical of the monarchy.
And Google, like all companies, is vulnerable to pressure from the government. It relies on work authorization visas that the government has just placed huge fees on. It uses tax shelters abroad that the government could investigate. It has a small but growing stable of defense contracts with the government. Any of these could be used for leverage.
What We Need
What we need is a robust information landscape that is not concentrated in the hands of one company.
Not only would it make it harder for the government to use pressure to shape what we know, but innovation in search is also just long overdue.
Just for starters, I want to be able to search the web for the best merchandise rather than just pulling from the sites that list items with Google. I want to be able to ban certain untrustworthy outlets from showing up in my search results. And I want to be able to do more specialized searches, for scientific papers, or tree identifications, without getting spammed with commercial results.
My wish list goes on and on. And there are probably even cooler things that entrepreneurs will dream up that I haven’t even thought of yet.
Unfortunately, the judge’s ruling isn’t going to get us there. But we can still get there if we recognize Google search for what it has become – infrastructure, not a product. Its underlying data is impossible for competitors to match. But if we meaningfully open up access to its data, we could have an array of awesome search tools built on top of it.
That would likely require Congressional action – which, of course, is always a long shot these days. But given the high stakes, we must try.
As always, thanks for reading.
Best
Julia