Articles — How shortlists work
How shortlists work
AI can give stupid or dangerous advice
The glue-on-pizza line came from a joke with eight upvotes. A packing list for Mount Shasta ended in a rescue. In both cases a bad source was used as though it were a real one.
Packing lists, routes, and invented places fail the same way shop catalogs do when the source is never weighed. News like this is part of why makeaivisible.com exists, because I wanted to understand how an assistant could treat a joke, or a bad packing list, as a source. The failure is simpler than a missing SKU. The system took a scrap of text somebody posted online and used it as though it were serious, without ever checking whether the source deserved that much weight.
2024: an 8-upvote joke
May 2024, days after Google’s AI Overviews launched in the US. The query was about cheese not sticking to pizza, and the Overview recommended about 1/8 cup of non-toxic glue.
The source was an 11-year-old Reddit comment from an obscure thread, posted as a joke by a user named fucksmith, with 8 upvotes. The same week, another Overview cited The Onion as a real source on how many rocks a person should eat per day. 404 Media traced the glue line, and contemporaneous coverage in the New York Times and elsewhere filled in the rest.
Google called the results extremely rare and said the vast majority of Overviews were high quality, and those specific queries were cleaned up. What stayed was the underlying habit of treating unweighted forum posts as though they sat level with everything else.
2026: a packing list with a rescue
The answer came with a citation. Nobody checked whether the source was a joke, or whether that packing list held enough food for a mountain.
Three hikers were stranded on Mount Shasta, California, on 30 August 2026, and rescued the next day. They told the Siskiyou County Sheriff’s Office they had relied heavily on Gemini for the route and for what to pack.
The sheriff’s office called it a critical misstep on the record: the group had been advised to bring far less food and water than they needed, which mattered enormously once an eight-hour ascent became a multi-day ordeal. They summited well past the noon turnaround, descended in the dark, and wandered off-route. ABC News and the sheriff’s own account are the spine of this.
The office closed by telling people to call the ranger station for accurate information and to never rely solely on AI for trip planning.
Underneath, this is the same failure as the glue joke. Only the consequences changed.
Same failure, other authors
It doesn’t have to be a chatbot talking to a hiker.
In January 2026, CNN Travel reported that Tasmania Tours had published a “best hot springs” roundup, written by a third-party AI tool while the owner was travelling, which ranked a hallucinated Weldborough Hot Springs at #4. Hundreds of tourists turned up looking for it. The owner said the AI had messed up completely. A real business had published a page that other systems, and other people, could reasonably treat as a genuine guide, and nobody set out to game anything.
Two shorter cousins of the same story: ChatGPT sent tourists toward a nonexistent Sacred Canyon of Humantay in Peru (BBC, 2025), and an AI-generated video of a fake Kuak Skyride in Malaysia sent an elderly couple hundreds of miles (2025). One place was invented by an assistant, one by a video, one by a company’s own blog. In each case nobody weighed the source.
A citation means the system found a page. It does not mean anyone checked whether that page was a joke, a thin list, or a place that does not exist.
Read next if: Why AI won’t just say it doesn’t know · “Best things to do near me” is how you get the tourist list