As the US government weighs its options following a landmark \u201cmonopolist\u201d ruling against Google<\/a> last week, online publications increasingly face a bleak future. (And this time, it\u2019s not just because of severely diminished ad revenue.) Bloomberg<\/em> reports<\/a> that their choice now boils down to allowing Google to use their published content to produce inline AI-generated search \u201canswers\u201d<\/a> or losing visibility in the company\u2019s search engine.<\/p>\n The crux of the problem lies in the Googlebot, the crawler that scours and indexes the live web to produce the results you see when you enter search terms. If publishers block Google from using their content for the AI-produced answers you now see littered at the top of many search results, they also lose the privilege of including their web pages in the standard web results.<\/p>\n The catch-22 has led publications, rival search engines and AI startups to pin their hopes on the Justice Department. On Tuesday, The New York Times<\/em> reported<\/a> that the DOJ is considering asking a federal judge to break up parts of the company (spinning off sections like Chrome or Android). Other options it\u2019s reportedly weighing include forcing Google to share search data with competitors or relinquishing its default search-engine deals, like the $18 billion one it inked with Apple<\/a>.<\/p>\n Google uses a separate crawler for its Gemini (formerly Bard) chatbot<\/a>. But its main crawler covers both AI Overviews and standard searches, leaving web publishers with little (if any) leverage. If you let Google scrape your content for AI Overview answers, readers may consider that the end of the matter without bothering to visit your site (meaning zero revenue from those potential readers). But if you block the Googlebot, you lose search visibility, which likely means significantly less short-term income and a colossal loss of long-term competitive standing.<\/p>\n iFixit<\/em> CEO Kyle Wiens told Bloomberg<\/em>, \u201cI can block ClaudeBot [Anthropic\u2019s crawler for its Claude chatbot<\/a>] from indexing us without harming our business. But if I block Googlebot, we lose traffic and customers.\u201d<\/p>\n Another problem with combining the two is that it gives Google an immeasurable advantage over smaller AI startups. The company gets a plethora of free training data from publishers eager to remain visible in search. In contrast, AI companies are forced to pay publishers for access to their data \u2014 and, even then, it wouldn\u2019t add up to the motherlode Google gets (essentially) for free.<\/p>\n From that perspective, it isn\u2019t surprising to read that, according to Bloomberg<\/em>, Google is spurning publishers that try to negotiate content deals. (Reddit has been the lone exception<\/a>.) Why waste money on content deals when they get all the training data they want in exchange for the search results most publishers need to survive?<\/p>\n \u201cNow you have a bunch of tech companies that are paying for content, they\u2019re paying for access to that because they need it to be able to compete in any kind of serious way,\u201d Alex Rosenberg, CEO of AI startup Tako Inc., told Bloomberg<\/em>. \u201cWhereas for Google, they don\u2019t really have to do that.\u201d<\/p>\n It comes down to leverage, which Google wields over desperate publishers. On top of the industry\u2019s existing financial troubles (online ad revenue has fallen off a cliff over the past eight years), AdWeek<\/em> reported<\/a> in March that Google\u2019s AI-generated search answers could lead to a 20 to 60 percent drop in organic search traffic.<\/p>\n The ball is now in the Justice Department\u2019s court to figure out where Google \u2014 and, to an extent, the entire web \u2014 goes from here. Bloomberg<\/em>\u2019s full story is worth a read<\/a>.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n