As more publishers enter licensing agreements with OpenAI, the creator of ChatGPT, a recent study by the Tow Center for Digital Journalism sheds light on how the AI chatbot handles citations for publishers’ content. The findings are both intriguing and concerning.
The study suggests publishers are vulnerable to ChatGPT’s tendencies to fabricate or misrepresent information, regardless of whether they permit OpenAI to access their content. Conducted at Columbia Journalism School, the research examined how ChatGPT cites sources when asked to attribute sample quotes from various publishers. The analysis included content from 20 publishers, such as The New York Times (currently suing OpenAI over copyright), The Washington Post (unaffiliated with OpenAI), and The Financial Times (which has a licensing deal).
The researchers selected quotes from 10 stories per publisher, ensuring these quotes could easily be traced to their original articles using search engines like Google. They then tested ChatGPT’s ability to accurately identify and cite the original sources. According to the study’s authors, Klaudia Jaźwińska and Aisvarya Chandrasekar, the results were disappointing. Although OpenAI promotes its ability to provide “timely answers with links to relevant web sources,” the company does not guarantee the accuracy of its citations—a critical issue for publishers seeking faithful representation of their content.
The study revealed that no publisher, regardless of their relationship with OpenAI, was spared from inaccurate citations. Researchers uncovered numerous instances of misrepresentation, with responses ranging from completely accurate to entirely wrong or partially correct. Alarmingly, ChatGPT rarely admitted uncertainty, often presenting incorrect information with confidence.
Even for publishers blocking OpenAI’s crawlers, the chatbot generated incorrect citations instead of acknowledging its limitations. In total, 153 out of 200 responses were partially or entirely incorrect, and ChatGPT only admitted its inability to provide accurate answers seven times. Unlike search engines like Google, which can flag a lack of results, ChatGPT tends to fabricate answers, undermining transparency and trust.
One striking example involved ChatGPT attributing a quote from The New York Times to a website that had plagiarized its content. This raises concerns about OpenAI’s ability to vet and validate its data sources, especially when dealing with unlicensed or plagiarized material.
The study also found issues with publishers who have licensing deals with OpenAI. Allowing crawlers access to content did not guarantee citation accuracy. The researchers attributed these problems to OpenAI’s approach, which treats journalism as decontextualized content, disregarding its original context and production.
Another problem was the chatbot’s inconsistency. The same query often produced different responses, an inherent trait of generative AI tools but problematic for citation accuracy. While the study was small-scale, it highlights challenges in OpenAI’s handling of publisher content, casting doubt on the benefits of licensing agreements.
The findings suggest publishers cannot count on improved visibility or accurate sourcing in ChatGPT, even if they allow OpenAI to crawl their content. Conversely, blocking OpenAI entirely does not shield publishers from risks, as the chatbot may still misattribute their content.
Ultimately, the researchers concluded that publishers have limited control over how ChatGPT uses or misuses their content. In response, OpenAI defended its practices, stating that it supports publishers by helping millions of users discover quality content. The company also pledged to improve citation accuracy and respect publisher preferences, but the study indicates much work remains to be done.
Post a Comment