OpenAI just unveiled its own search engine called ChatGPT Search. This basically confirms that the age of the AI-powered search engine is officially here. Well, maybe not entirely. According to a new report, ChatGPT Search struggles with news content.
Just like Perplexity and Google AI Overviews, ChatGPT Search is a tool that creates an AI-generated overview of what you searched. What makes these tools search engines is the fact that they surface the links from the sources they got their information from. Right now, there are only a handful of major AI search engines on the market, but we expect the number to grow over time.
ChatGPT Search struggles with news content
Since ChatGPT Search is so new, people are looking into it to see how it stacks up to the competition. Well, based on a new study, things aren’t looking too good for the new tool.
Columbia’s Tow Center for Digital Journalism conducted a study on ChatGPT Search, and it found some worrying results. To conduct the test, they asked ChatGPT to identify the source of 200 quotes from 20 different news articles. With the internet at its fingertips, you’d expect that ChatGPT would do a good job with this task. Not only that, but OpenAI has access to dozens of news websites’ content through license deals.
However, according to the report. The mass majority of the responses that ChatGPT search gave were inaccurate or incomplete. Out of the 200 responses, it only gave accurate information on 47. 57 were partially correct and a whopping 89 were completely wrong. Lastly, it only admitted that it didn’t know the answer on seven occasions. Bear in mind that this was a pretty niche test.
Tow Center gave a few examples of some ChatGPT Search’s mess-ups. Firstly, when it was asked to source a quote from a New York Times article, the tool surfaced a link to a site that completely plagiarized the article. In another example, it misattributed a quote from an Orlando Sentinel story to Time.
OpenAI commented on this study, calling it an “atypical test of our product,” and saying that “Misattribution is hard to address without the data and methodology that the Tow Center withheld.” So, OpenAI is not happy about this test. Of course, it reassured the masses that it’s working to enhance the search results. We’re not sure if that’s the company basically admitting that it had messed up with ChatGPT Search. In any case, we’re sure that issues like these will be less common as time goes on.
