Meta AI Bot contributes to fake research online

Meta's AI bot, Galactica, was paused after just two days.

Photo: Andrea Verdelli (Getty Images)

meta it paused Artificial intelligence (AI)bot last week, just two days after it went live to the public. The bot, named Galactica, was trained “on 106 billion tokens of open-access scientific text and data. This includes papers, textbooks, scientific websites, encyclopedias, reference materials, knowledge bases and more. The everyday beast.

It was intended to help academics and researchers find papers and studies quickly and concisely, but was instead overwhelmed by massive amounts of misinformation that falsely cited reputable scientists.

Scientists’ reputations can be at stake if they are misquoted in the text, Carl Bergstrom, a professor of biology at the University of Washington, told CNET that Galactica’s problem is that it was promoted as a way to get facts and information. But instead he said it behaved like “a nice version of the game where you start with half a sentence and then let the rest of the story fill in automatically.”

Within hours of Galactica going live, users began reporting racist and inaccurate articles, with one person post an image of the response to a request about language bias. The bot’s response falsely claimed that black people “do not have their own language” and immigrants “do not speak a language different from the language of the country they are immigrating to”.

Other information generated included a bogus study on the benefits of eating broken glass and falsified information about Stanford University researchers who developed a “gaydar” software to find gays on facebook.

A Meta AI spokesperson told CNET: “Galactica is not a source of truth, it is a research experiment using [machine learning] systems for learning and summarizing information.” He added that Galactica is “exploratory research that is short-lived and has no product plans.” Meta AI Chief Scientist Yann LeCun told the outlet that the bot was removed because the team was “so distraught by the vitriol on Twitter.”

Two days after the launch of the Galactica, the Meta AI team stopped the bot and Meta’s main AI scientist Yann LeCun tweeted, “The Galactica demo is offline for now. It is no longer possible to have fun by casually abusing it. Happy?”

Meta AI (formerly called Facebook Artificial Intelligence Research) developed Galactica as a way to “organize” and condense science an overwhelming amount of scientific information found online. The idea was to have the AI doing things like solving math problems, writing scientific code, and summarizing research.

meta states on Galactica’s website that the AI ​​model has limitations that allow it to “hallucinate.” The site advises users to verify all information that pops up and goes on “There are no guarantees of truthful or reliable output from language models, even large ones on high-value data such as Galactica,” adding that the generated text may appear “very authentic and very confident” but still be wrong.

“I imagine that even with its many predictable flaws, there are desirable uses for such a system,” Vincent Conitzer, a computer science professor at Carnegie Mellon University in Pittsburgh, told The Daily Beast.

“My impression is that Meta could have done better by putting more effort into this public release, doing more serious user research first, drawing attention to desirable uses of the system, and being honest and upfront about unwanted uses.”

Leave a Reply

Your email address will not be published. Required fields are marked *