In 2020, researchers at the Center for Terrorism, Extremism and Counter-Terrorism at the Middlebury Institute of International Studies found that GPT-3, the underlying technology for ChatGPT, had “an impressively deep knowledge of extremist communities” and could be prompted to engage in polemics to produce mass shooter style, fake forum threads discussing Nazism, a defense of QAnon, and even multilingual extremist texts.
The dissemination of misinformation and untruths
- Artificial intelligence: For the first time, AI generated personas were discovered in a state-sponsored disinformation campaign that opens a new chapter in online manipulation.
- deepfake Rules: In most parts of the world, there is not much the authorities can do about deepfakes as there are few laws to regulate the technology. China hopes to be the exception.
- Lessons for a new generation: Finland is testing new ways of teaching propaganda to schoolchildren. Here’s what other countries can learn from its success.
- Covid Myths: Experts say the spread of misinformation about the coronavirus – particularly on far-right platforms like Gab – is likely to be a lasting legacy of the pandemic. And there are no easy solutions
OpenAI uses machines and humans to monitor content fed into and produced by ChatGPT, a spokesman said. The company relies on both its human AI trainers and user feedback to identify and filter out toxic training data while teaching ChatGPT to provide more informed answers.
OpenAIs guidelines prohibit use of its technology to encourage dishonesty, deceive or manipulate users, or attempt to influence policy; The company offers a free moderation tool Dealing with content that promotes hate, self-harm, violence or sex. But at the moment, the tool offers limited support for languages other than English and does not identify political material, spam, deception or malware. ChatGPT warns users that it “may occasionally produce harmful instructions or biased content.”
Last week, OpenAI announced a separate tool to recognize when a text was written by a human, as opposed to artificial intelligence, in part to identify automated misinformation campaigns. The company warned that its tool isn’t entirely reliable — it only accurately identifies AI text 26 percent of the time (while human-written text is mislabeled 9 percent of the time) — and could be circumvented. The tool also struggled with texts that were under 1,000 characters or written in languages other than English.
Arvind Narayanan, computer science professor at Princeton, wrote on Twitter in December that he had asked ChatGPT some basic information security questions he had asked students in an exam. The chatbot responded with replies that sounded plausible but were actually nonsense, he wrote.
“The danger is that you can’t tell when something is wrong unless you already know the answer.” he wrote. “It was so disturbing that I had to look at my reference solutions to make sure I wasn’t going insane.”
Researchers fear the technology could be exploited by foreign agents hoping to spread disinformation in English. Companies like Hootsuite are already using multilingual chatbots like the Heyday platform to support customers without a translator.
