Crane bullet points five things that need to change as the AI industry scales faster than it builds a worthwhile safety architecture. Specifics he calls for include; stricter confirmations, scopable API tokens, proper backups, simple recovery procedures, and AI agents existing within proper guardrails.
“I hooked up spicy autocomplete to our production systems and it nuked them. What have I learned from this? Here are some bullet points for how the spicy autocomplete industry needs to do better.”
🍿
Seems like user error, I’m no programmer but even I lnow you don’t give an agent access to critical things, and Claude is very insistent at asking for permission at every step.
Seems like user error, I’m no programmer but even I lnow you don’t give an agent access to critical things
Yes.
But these models have (largely correctly) learned from Stack Overflow that, on average, every problem is due to not enough permissions.
Someone fully relying on an agentic AI model is essentially destined to give it full control (or close enough), eventually.
At some point, a tool like these LLMs either needs to not be marketed to that user, or needs stupid levels of safety warnings.
My money is on neither solution happening, and this kind of result continuing for the foreseeable future - until the rest of us doing cleanup instigate Dune’s Butlerian Jihad to stop the damage and save our own sanity.
Thus completelly eliminating all bugs and data incorrections in Production!
That right there is the sweet smell of Victory!
Efficient, tho.
So what? Gen AI can build a new database and populate it with data 🤷♂️ /s
PocketOS founder blames ‘Cursor running Anthropic’s flagship Claude Opus 4.6’
Fuck that. I’m blaming the PocketOS founder and every person in the chain of decisions that led to a clanker being given this level of unrestricted access to the database and the backups.
Man who shit his own pants horrified that his pants are full of shit.
Demands explanation from pants vendor.
In your analogy, I would think they would demand an explanation from the food vendor.
Demands explanation from local grocery store?
I can’t believe that they criticized the vendor api for not having confirmation.
It’s a freaking API!!! It’s designed for automation, not direct human (or LLM!) use. If you added confirmation then devs would have to code automatic acceptance, which defeats the purpose.
It doesn’t make a bit of sense. Someone is passing the buck.
Yeah but a higher up would have to take responsibility, we can’t have that
Everything is always somebody else’s fault.
Yelling, “Who’s shit is this!?”
If you are going to give an LLM a free pass to your whole prod database least you should do is to take weekly (or daily if plausible) offline backups of it. A hard limit against deleting stuff would be better.
If you are giving your codegen LLM - the model involved truly, genuinely doesn’t matter - admin access to your prod env, all I’m going to do is point and laugh.
Also no prompts, ironically, for operations like “Are you sure you want to delete the production database? (y/N)”
It’s amateur hour all around lol
Just to add - AND ACCESS TO THE BACKUPS!!
and having the backups stored in the same location as the primary data
Then it’s not a backup, it’s just duplicated data.
Just a shit show top to bottom for sure
No one should be able to delete or change backups. This infra was in any case vulnerable to a ransomware attack as any bad actor that breaks in can delete the database and encrypt the backups with a key they promise to share in return for bitcoin.
Don’t blame, Credit. Credit CloudStrike.
Dance, rich. Dance and don’t stop cuz I ain’t a single Arnold Schwarzenegger saving your ass.
Fuck around and find out
This is fun to read. I hope people will have their actual intelligence activated after this.
They won’t but they will continue to accidentally create content for my amusement :D
Listen to Dimmu Borgir and MJK’s Puscifer to their mediots.
I want to watch the old rich fucking dance on cam to that to.
Hot take: offsite, offline backups are so cool right now.
They blame AI, but what if this was one of those aggressive encrypt everything malware attacks? Why are the backups accessible anytime except during a backup? I know, I know, convenience. But all of the backups?
Hotter take: do not give an LLM agent permissions you wouldn’t give a recently hired junior
Actually this is how AI should be viewed. Under the right circumstances it maybe saves lots of time, but it also might destroy, so treat it like you would an intern…
Analogue is the next big thing.
Yup, follow the 3-2-1 rule or you don’t have backups
hell I’ve got a better backup methodology with my fucking cat photos
wiping a volume deletes all backups
that sounds like a wonderful backup system 😂
