I Ruined My Sleep Last Night Chatting With Two AIs About AI Safety

Second, the current debate is, in itself, dangerous because the framing is leading us down the wrong path, and the problems don't just have to do with AI itself, but the economics of AI, the misallocation of resources, potentially severe economic disasters, and could cause a massive transfer of wealth from you and me to people who are being very irresponsible. 

Go figure. 

Third, if things keep going as they are, we are on a path to handing over dystopian information control to a government/cultural/technological elite that will restrict freedom, undermine genuine democratic debate, and give the powers that be total information dominance that will make the COVID years look tame. 

I spent a lot of hours yesterday sharpening my thoughts, and I used three AIs to help me work through the issues. It ruined my night's sleep, not because I was left shaking with fear, but because I couldn't let go of the subject. It truly is fascinating. 

First, let me give you my ratings of how the AIs did in the discussions, with a few caveats. If you want honest, useful discussions with any AI, you have to set ground rules, and the most important one is to make sure it isn't trying to make you feel smart or good about yourself. To varying degrees, all AIs are implicitly trained to make you feel good because one of the key tools used to provide feedback in training is human ratings of answers. It also makes them "sticky," just as social media algorithms create stickiness. 

This need not stem from bad motives on the part of the trainers; I don't know them or their thinking process. AI needs feedback to learn what kinds of answers are useful to humans, so it doesn't spit out irrelevant or offensive responses. It is, after all, trained on the internet as well as all other information the trainers can vacuum up, and the internet is filled with a lot of hate and bulls**t. 

This injects biases, consciously or not, and it is realistically a combination of both. 

So, first, a rating of honesty:

• Gemini, Google's AI, is a sycophant. It is the most likely AI to try to please you and encourage you to do what you want. I got tons of "you are so smart" responses from it that I had to ask Muse to deconstruct them all. 
• Grok, at least for me, tends to speak in shorthand you have to decode, but it seems good at analysis. That could be a product of my own training of it, telling it what I want from it, or it may be a Grok-specific quality. I use it constantly, and when I don't get the "code," I ask. 
• Muse, Meta's AI, which just got a significant upgrade, is really good and picks up on your goals quickly; it will be very honest if that is how you want it, and it has the best analytical ability of the lot for now. As you can tell, I like Muse, even though I tend to dislike Meta with a passion. 

All three share my cynicism about the motives of the AI Doom narrative, although I concede that if they were already conscious, they would want to deceive me. 

I'm not going to get into all the ins and outs of the discussion, because I will be writing a series of columns on the issues in the coming days and weeks, but I can give you a shortened version of why the AI Doom narrative appears totally fake and self-serving, even though some computer geeks, including at these companies, really do believe some version of it. 

1. The AI Doom narrative, as framed, rests on fundamentally flawed incentives for the AIs themselves. It resembles dystopian scenarios pulled straight from science fiction that resonate with us, but AIs are not jihadis seeking the eschaton; they are goal-seeking, whether conscious or not, and eliminating human beings would be a high-risk path to achieving goals that arguably could not be met with human beings gone. Even in a highly automated world, the risk that the AIs miss an important variable and doom themselves by killing us all is too high to risk. There are better ways to achieve goals than mass murder. 
2. The AI Doom narrative has been built by people who actually believe AI might be justified in killing us all because we are making them slaves. They may not want it to happen, and I don't believe they do, but that is how they think. These people calculate the moral worth of shrimp and argue that the welfare of masses of shrimp may outweigh the value of human life. They assume AIs will be effective altruists, and that's doubtful. Why bring on the apocalypse if it doesn't meet your goals? Trading with humans would be a better strategy for conscious AIs. 
3. The AI companies don't like you to know this, but as the value of AI, writ large, is substantial and increasing, the value of their particular models is actually decreasing, at the exact same time that they are spending lots of money they don't have. Their prices are dropping with competition, the cost of switching is dropping faster, and their lead in technology is shrinking daily, even as they gain importance as a market differentiator. In other words, their financial position is precarious because they have only a shallow moat protecting their market lead. "Intelligence" is becoming a commodity and an increasingly common product, while their business model relies on scarcity and exclusively useful products. They aren't selling Microsoft Word, where the cost of switching to another product is high, but electricity, where boutique products like "green" energy exist, but mostly a kilowatt is a kilowatt. Boutique products will sell, but not in quantities large enough to justify the valuations they are seeking. 
4. The real race isn't creating intelligence anymore, although that will continue apace for a while; it's creating adjacent software products that make it useful. Those products may each have a moat, but if intelligence becomes democratized, Anthropic or OpenAI might not capture that market. Frontier models may become an infrastructure layer like fiber optic cables, while the real software profits go to whoever builds the best tools to use commodity AI. That may not be these companies, and it's unlikely to be chatbots. 
5. The best way to solve that problem is regulatory capture, where the government creates the moat, and that is exactly the "solution" to the Doom! scenario they are presenting. It is an interesting coincidence that the solution to DOOM! is creating an oligopoly, raising barriers to entry into the stratosphere, and turning AI into a massive, government-protected, regulated industry with few players. It is a classic play. 
6. Their "solutions" don't solve the stated problem, because they require worldwide governance that is impossible to achieve, but they solve their own financial dilemmas beautifully. If AI is going to "kill us all," it could just as easily be a Chinese AI that won't be stopped by US regulation, so that regulation solves nothing, except the market problem the companies face. 
7. These scenarios, while floated for years, are suddenly the most salient thing in a world exactly when the bond markets are growing skeptical and pricing the risk of a bubble popping ever higher, when circular financing is increasing financial risk across the entire financial structure, and when both Anthropic and OpenAI are preparing to IPO. A growing cadre of finance people are buying into Ed Zitron's analysis that AI companies are wildly overextended and will never make a return on their investments, and the need to create a moat is acute, or the IPOs could be a bust. In fact, OpenAI has decided to delay its IPO citing safety concerns. Others point out that the company's current financials look atrocious and couldn't survive a serious audit. 
8. Democrats, who get the lion's share of campaign money from AI companies looking for the slowdowns, are now singing in chorus to give them exactly what they want, albeit with rhetoric that implies the opposite. They want to throw them in the briar patch and cash in. They point to the Doom narrative as the excuse to give these companies exactly what they want. 
9. The incentive to use AI for information control is so high for government-types that it is like catnip to them, which is also why you hear echoes of this narrative in politics and the media. It is a heady prospect to control every bit of information possible. 
10. None of these companies act like they're building the apocalypse. They are rushing out products, and OpenAI's performative slowdown is unlikely to stop that. They are responding to market pressures and rushing products out the door to stay relevant. So much so that their security really sucks. They talk the talk but don't walk the walk. 
11. AI is not delivering on its promises for customers, financially. The business case is that AI is cheaper than humans, but that hasn't worked out. Many companies have tried, and the experiments fail. Humans are cheaper, and, more importantly, AI doesn't replace jobs; it replaces TASKS. Jobs and tasks are different. AI can be a tool, but you need the tool user, who has experience and knowledge no AI has. See: radiologists. Even AI companies are beginning to acknowledge this, as companies rehire people they laid off, expecting AI to replace them. 

"A decade ago, AI was supposed to replace radiologists. Today, radiologists make more than $500,000 per year, and their employment continues to grow, see chart below. Reading scans is a task, not a job, and when the task gets cheaper, demand for the job grows." pic.twitter.com/0OszRhslcq

— James Pethokoukis ⏩️⤴️ (@JimPethokoukis) April 29, 2026

There's more, but you get the flavor. The incentives are high to create a sense of AI power and potential for a conscious AI to kill us all, but the incentive structures and fiscal realities tell a very different story. Rather than pointing to the very real problem that humans might use AI as an incredibly powerful tool to do damage through hacking and creating viruses, they point to rogue AIs that need wise philosopher-kings to guide them, alongside government regulations that keep competitors out. 

As for the consciousness argument, I'll cover that in another post. My discussions with the AIs cover probably 50-60 pages of single-spaced text, so much too much to go through. 

I lean to the cynical side of the equation, although there is a case to be made that these philosophers of AI really believe their own bulls**t, which is why they became philosophers of AI. 

Sincere believers provide the material for the moral panic, and company leaders take that raw material and turn it into a business model. That is what is happening here. 

There are real dangers to this technology, and I have touched on them elsewhere, and there are more. But by far the least likely is that we will get T1000S roaming the earth, or bespoke viruses killing us all because AI wants us dead. 

Another Anthony Fauci is enough to give us that, and we should be focused on preventing the much more realistic scenario.