When can you expect your query to trigger an AI-generated summary of the results? “AI Overviews appear for complex queries,” says Mallory De Leon, a Google spokesperson. “You’ll find AI Overviews in your Google Search results when our systems determine that generative AI can be especially helpful–for example, when you want to quickly understand information from a range of sources.” During my initial tests, it felt like the AI Overviews popped up almost at random for queries, and the summaries appeared for simple questions as well as more complicated asks.

According to De Leon, the AI Overview is powered by a customized version of Google’s Gemini model that’s supplemented with aspects of the company’s Search system, like the Knowledge Graph that has billions of general facts.

One of my core hesitations about this feature as it rolls out is the continued potential for AI hallucinations, more commonly known as lies. When you interact with Google’s Gemini chatbot, a disclaimer at the bottom reads: “Gemini may display inaccurate info, including about people, so double-check its responses.” There’s no such disclaimer added to the bottom of the AI Overview, which often simply reads, “Generative AI is experimental.”

When asked why there’s no mention of potential hallucinations for AI Overviews, De Leon emphasizes that Google wants to still offer high-quality search results and mentions that the company did adversarial red-teaming tests to uncover potential weak points for the feature.

“This implementation of generative AI is rooted in Search’s core quality and safety systems, with built-in guardrails to prevent low-quality or harmful information from surfacing,” she says. “AI Overviews are designed to highlight information that can be easily verified by the supporting information that we surface.”

Knowing this, you might still want to click through the webpage links to double-check that the information is actually correct. Though it’s hard to imagine many users, who are often looking for quick answers, will spend extra time reading over the source material for Google’s AI-generated answer.

Liz Reid, Google’s head of Search, recently told my colleague Lauren Goode that AI Overviews are expected to arrive for countries outside of the United States before the end of 2024, so over a billion people will likely soon encounter this new feature. As someone whose job relies on readers actually clicking links and spending time reading the articles, of course I’m apprehensive about this change—and I’m not alone.

Beyond concerns from publishers, it also remains unclear what additional impacts might trickle down to users from Google’s AI Overviews. Yes, OpenAI’s ChatGPT and other AI tools are quite popular in Silicon Valley tech circles, but this feature will likely expose billions of people, who have never used a chatbot before, to AI-generated text. Even though AI Overviews are designed to save you time, they might lead to less trustworthy results.