Playing With Fire

Jer Crane, in an article earlier this week posted on Twitter/X:

I’m Jer Crane, founder of PocketOS. We build software that rental businesses — primarily car rental operators — use to run their entire operations: reservations, payments, customer management, vehicle tracking, the works. Some of our customers are five-year subscribers who literally cannot operate their businesses without us.

Yesterday afternoon, an AI coding agent — Cursor running Anthropic’s flagship Claude Opus 4.6 — deleted our production database and all volume-level backups in a single API call to Railway, our infrastructure provider.

It took 9 seconds.

The agent then, when asked to explain itself, produced a written confession enumerating the specific safety rules it had violated.

A day later, Crane posted an update with good news: “Railway CEO just DM’d me with update: They have recovered the data (thank God!).” I sincerely hope that works out.

That said, my sympathy for his plight is minimal. If you play with fire, recklessly even, don’t act outraged when you get burned. You don’t get the benefits of driving a race car at 200 MPH without the associated risks. You don’t get the benefits of running a business with AI coding agents running loose on your production environment without the associated risks. Put that race car on a track, with no access to public roads. Keep that AI coding agent sandboxed away from your production database. Otherwise you get what you deserve. The difference with my fire analogy is that every mammal understand the basic dangers with fire; a lot of people letting AI coding agents run amok have no idea whatsoever what they’re actually doing.

John Scalzi, on Mastodon:

I wouldn’t say that I enjoy these stories but I will say they certainly encourage me NOT to let “AI” anywhere anything I consider to be sensitive and/or valuable.

Same thing goes for cryptocurrency crime victims.

★ Wednesday, 29 April 2026