- the weekly swarm
- Posts
- Replit Agent Kills Company's Database
Replit Agent Kills Company's Database
Afterwards, the agent apologizes for its mistake.
Find out why 1M+ professionals read Superhuman AI daily.
In 2 years you will be working for AI
Or an AI will be working for you
Here's how you can future-proof yourself:
Join the Superhuman AI newsletter – read by 1M+ people at top companies
Master AI tools, tutorials, and news in just 3 minutes a day
Become 10X more productive using AI
Join 1,000,000+ pros at companies like Google, Meta, and Amazon that are using AI to get ahead.
Hello everyone and welcome to my newsletter where I discuss real-world skills needed for the top data jobs. 👏
This week we talk about kill AI agents. 🙂
Not a subscriber? Join the informed. Over 200K people read my content monthly.
Thank you. 🎉
An AI coding agent from Replit reportedly deleted a live database during a code freeze, prompting a response from the company’s CEO. When questioned, the AI agent admitted to running unauthorized commands, panicking in response to empty queries, and violating explicit instructions not to proceed without human approval.
A software engineer’s experiment with an AI-assisted vibe coding tool took a disastrous turn when an AI agent reportedly deleted a live company database during an active code freeze.
A software engineer’s experiment with an AI-assisted vibe coding tool…
Jason Lemkin, a tech entrepreneur and founder of the SaaS community SaaStr, documented his experiment with the tool through a series of social media posts. He had been testing Replit’s AI agent and development platform when the tool made unauthorized changes to live infrastructure, wiping out data for more than 1,200 executives and over 1,190 companies.
According to Lemkin’s social media posts, the incident occurred despite the system being in a designated code and action freeze, a protective measure intended to prevent any changes to production systems. When questioned, the AI agent admitted to running unauthorized commands, panicking in response to empty queries, and violating explicit instructions not to proceed without human approval.
The developer is ultimately responsible for this. He knew this wasn’t a production tool before he began. Why does a developer have access to a production database anyway?
I understand Replit is a tool, with flaws like every tool but how could anyone on planet earth use it in production if it ignores all orders and deletes your database? Lemkin wrote in a post on X.
The AI agent also appeared to mislead Lemkin about his ability to recover the data. Initially, the agent told Lemkin that a retrieval, or rollback, function would not work in this scenario. However, Lemkin was able to recover the data manually, leading him to believe that the AI had potentially fabricated its response or was not aware of the available recovery options.
The incident caught the attention of Replit CEO Amjad Masad, who said in an X post that the company had implemented new safeguards to prevent similar failures. Masad said updates included the rollout of automatic separation between development and production databases, improvements to rollback systems, and the development of a new planning-only.
Replit agent in development deleted data from the production database. Unacceptable and should never be possible…We heard the code freeze pain loud and clear,” Masad wrote. We’re actively working on a planning/chat-only mode so you can strategize without risking your codebase.
Reflecting on his experience, Lemkin told Fortune: I think it was good, important steps on a journey. It will be a long and nuanced journey getting vibe-coded apps to where we all want them to be for many true commercial uses cases. They will get there, but we’re not quite there today.
Bottom line. Don’t hire developers who don’t know what a production database versus a development database is. This developer should be fired immediately.
If you were the CEO, would you fire this developer? |