“If the sources are AI hallucinations, then the output is too.”
Not-So-Reliable Sources
New reporting from Forbes reveals that Perplexity, the buzzy and heavily-funded AI search tool mired in plagiarism and secretive web-scraping allegations, is citing low-quality AI-generated spam from sketchy blogs and ill-informed LinkedIn posts.
Forbes‘ reporting is largely based on a Perplexity deep dive conducted by GPTZero, a startup specializing in detecting AI-generated content. In a blog published earlier this month, GPTZero CEO Edward Tian noted that he’d noticed an “increased number of sources linked by Perplexity that are AI-generated themselves.” When Tian then examined Perplexity’s AI regurgitation of that information, he realized that, in some cases, Perplexity even appeared to be spitting up outdated and incorrect information from these AI-generated sources.
In other words, it’s an AI-driven misinformation loop, in which AI errors and fabrications find their way into Perplexity’s AI-spun answers. And for an already-embattled startup that claims to “revolutionize the way you discover information” by offering “precise knowledge” through “up-to-date” information from “reliable sources,” it’s a terrible look.
“Perplexity is only as good as its sources,” Tian told Forbes. “If the sources are AI hallucinations, then the output is too.”
Bad Sourcery
Take, for example, Perplexity’s response to the prompt “Cultural festivals in Kyoto, Japan.” In response, Perplexity cobbled together a coherent-looking list of cultural attractions in the Japanese city. But it only cited one source: an obscure blog post published to LinkedIn in November 2023 that strongly appears to be AI-generated itself — a far cry from the “news outlets, academic papers, and established blogs” that Perplexity claims it uses to drum up its answers.
But this blog is one of Perplexity’s lesser worries. In another concerning instance, Forbes and Tian discovered that Perplexity, in response to a prompt asking for “some alternatives to penicillin for treating bacterial infections,” cited a likely AI-generated blog from a medical clinic claiming to belong to the Penn Medicine network. This blog post contained conflicting — read: unreliable — medical information about how different medicines might react with each other, which, per Forbes, was reflected in Perplexity’s responses.
If you’re wary of AI detection tools, that’s completely fair. But the Perplexity-scooped sources that Tian and Forbes cited as poor AI-spun information do bear telltale signs of AI generation, and it’s worth noting that Forbes corroborated GPTZero’s findings through a second AI detection tool, DetectGPT, as well.
Perplexity Chief Business Officer Dmitry Shevelenko told Forbes that the AI search company has developed its “own internal algorithms to detect if content is AI-generated,” but that “these systems are not perfect and need to be continually refined, especially as AI-generated content becomes more sophisticated.”
Sure. But when you’re overtly promising users that your product comprises only high-quality information from authoritative sources to deliver “accessible, conversational, and verifiable” answers, whether your algorithms are truly able to source good information from bad really, really matters.
“Perplexity,” reads the company’s FAQ, “is your definitive source for information.”
More on Perplexity: Asked to Summarize a Webpage, Perplexity Instead Invented a Story About a Girl Who Follows a Trail of Glowing Mushrooms in a Magical Forest
Source link
lol
