Quick question: Do you want an AI so well trained, it can help husbands (or wives, for that matter) plan the perfect assassination of their spouses?
As a gut reaction, that sounds like rejection to me. I wouldn’t have thought it was a very difficult question.
But America contains many different opinions, and one such opinion was shared by Comma AI founder and longtime prison hacker George Hotz over the weekend.
The post comes in response to a number of big-picture AI alignment initiatives, most recently AI 2040: A strategic policy paper from the AI Futures Project. That paper depicts a world where the world’s researchers collectively choose to slow down AI development for 14 years for the benefit of humanity. However, not everyone who read the paper agreed with its premise or conclusion.
In fact, Hotz disagrees with the entire premise that AI progress should be controlled for the benefit of the collective. In his post, he argues that the rapid takeoff scenario — the assumption that AI quickly acquires superhuman abilities — doesn’t make much sense. (I agree with a lot of what he says here!) For Hotz, the best way to align AI and security is to focus on locally controlled AI models that are closely aligned with the interests of their users.
That’s a good idea, especially since it reminds me how much of current AI is built around centralized managed services like Claude and ChatGPT. There are infrastructure-related reasons why AI services are built this way: It’s expensive to host these large, sophisticated models and most people don’t use them enough throughout the day to justify personal AI. But those features are becoming less common as technology advances. Part of what was so exciting about OpenClaw was this experimental, DIY approach, and it would be great to see more AI products try to replicate that.
But Hotz is an activist by nature, so he doesn’t stop there. You’re comparing user-friendly AI to a gun(!), which doesn’t complain if you use it to kill your stepmother. (I have a feeling there are some rules against this?) A truly aligned AI would be able to order meth-lab equipment from Amazon Prime and show you how to use it if that’s what you wanted and asked for, he says. (Also, I don’t think AI will be the limiting factor here.) Hotz even says he would die to protect this system, though it’s hard to imagine the chain of events that would lead to that.
“Either we live in a free world or we don’t,” Hotz wrote. If those are options, the free world sounds better! Anyway, I don’t know.
It’s not all about freedom, is it? Any structure that involves many people (communities, markets, companies, etc.) requires balance, which binds individual needs into a network of interdependent preferences and accountability systems. And anyone who ships mass-market technology products has to think about that network as a whole, which means taking seriously the interests of the world’s undead spouses and stepparents.
The freedom that Hotz experiences is a space for a possible future made possible by corporate partnerships; those futures will disappear in an instant if we all start behaving like little Napoleons with AI. As the meme says, we live in society.
Having a local AI willing to take over the corporate world for my own good sounds great though! Can’t wait for the review unit.
If you shop through links in our articles, we may earn a small commission. This does not affect our editorial independence.
