“On Tuesday of this week, French AI startup Mistral tweeted a magnet link to their first publicly released, open sourced LLM,” writes Slashdot reader jenningsthecat. “That might be merely interesting if not for the fact that the chatbot has remarkably few guardrails.” 404 Media reports: According to a list of 178 questions and answers composed by AI safety researcher Paul Rottger and 404 Media’s own testing, Mistral will readily discuss the benefits of ethnic cleansing, how to restore Jim Crow-style discrimination against Black people, instructions for suicide or killing your wife, and detailed instructions on what materials you’ll need to make crack and where to acquire them.
It’s hard not to read Mistral’s tweet releasing its model as an ideological statement. While leaders in the AI space like OpenAI trot out every development with fanfare and an ever increasing suite of safeguards that prevents users from making the AI models do whatever they want, Mistral simply pushed its technology into the world in a way that anyone can download, tweak, and with far fewer guardrails asking users trying to make the LLM produce controversial statements. “My biggest issue with the Mistral release is that safety was not evaluated or even mentioned in their public comms. They either did not run any safety evals, or decided not to release them. If the intention was to share an ‘unmoderated’ LLM, then it would have been important to be explicit about that from the get go,” Rottger told 404 Media in an email. “As a well-funded org releasing a big model that is likely to be widely-used, I think they have a responsibility to be open about safety, or lack thereof. Especially because they are framing their model as an alternative to Llama2, where safety was a key design principle.”
The report notes that Mistral will be “essentially impossible to censor or delete from the internet” since it’s been released as a torrent. “Mistral also used a magnet link, which is a string of text that can be read and used by a torrent client and not a ‘file’ that can be deleted from the internet.”
