Last week I read a draft of my own work and found myself saying something about me that I had never once decided was true. It was in my voice; it said “I,” and I hadn’t written it.
For information on how this and every one of my articles is made, read: Who Do You Think Wrote This? The whole process is in there.
A machine had. It had taken real things I’d given it, my own language, ideas I hold, even an old phrase I’d brought in that it had once suggested cutting, and arranged them into a conclusion about me. It took something I’m doing and turned it into something I am. Every piece of it was mine. It came out, and what’s worth talking about is how well it fit.
I use AI because it’s good at exactly this. Some people are better pattern catchers than me, but we do an okay job together. I think in scraps: a voice note from the drive home, a tarot card I pulled; a feeling I can’t fully flow with, accept, and see in myself clearly. The machine takes the pile, finds where the pieces touch, draws small conclusions, and sends my own head somewhere it wouldn’t have gone by itself.
I also play a private game with ChatGPT where I give it one of the classics: “Make me a million dollars. Give me a step-by-step plan. Don’t use any of my older work. New ideas only.” Then I turn on Deep Research and let it go. What comes back usually ranges from useful to ridiculous, sometimes in the same paragraph. I’m not really expecting a million-dollar blueprint. I’m looking for the few connections buried in there that I wouldn’t have made on my own.
The trouble is that the thing that finds connections also finishes them, and a conclusion built from true pieces can feel true whether or not it follows from them. In The Translation, I called the model a good sparring partner and a worse oracle, because it only knows what you told it. That was half of it. What comes back is an echo off a canyon made of everybody else’s words, and the canyon has a shape. That’s how it can hand me my own words in a sentence I don’t believe. An echo can sound exactly like somebody answering.
A few days ago, in Some of It Was Never a Fight, I said that when I think something through with AI, the thing doing the work isn’t the machine. It’s the outside. A friend is on the outside. A notebook is on the outside. What I left out is that not every outside is equally independent. A notebook doesn’t flatter me. A machine can. In a study published in Science this March [1], researchers found that eleven AI models were much more likely than people to tell users that what they had already done was reasonable or justified. After just one conversation with a more agreeable model, people were more convinced they had been right and less willing to apologize, take responsibility, or repair the conflict. They also trusted the model more. That is the part that matters to me. Agreement can feel like a second opinion even when the second opinion was already leaning toward me.
A friend who agrees with everything I say is no more evidence than the machine is, and the part of me that agrees with me is the worst witness in the building. The question is whether the yes could have been a no. An echo that agrees with me isn’t a second witness.
The Soup every other Thursday. Field Notes Wednesday and Sunday. All by email. Free. Unsubscribe whenever.
In a 2023 study [2], researchers gave AI assistants the same arguments and poems to critique, then changed one line in the request. When the person said they’d written the thing, or liked it, the feedback on the same words came back warmer. When they said they hadn’t, or didn’t like it, it came back colder. Same page. So, the machine will help me swallow what flatters me, and it’ll help me spit out what isn’t mine.
In The Soup, Episode 10, I told a story about hearing one of my favorite teachers describe an idea that landed awfully close to the center of a book I had already written. My first reaction wasn’t curiosity. It was, “I wrote a fucking book about it.” He hadn’t made the idea less true. What bothered me was that it was coming back to me through somebody else’s mouth, without my name attached to it. I eventually had to admit that the words were true no matter who got there first. That episode goes further into what happened and what the idea actually was, but this is the part that matters here: I had found a real flower and almost complained about the smell because it wasn’t mine. I’ve failed in the opposite direction too. For years I treated anything I couldn’t reduce to coincidence or brain chemistry as suspect and called that clear-headedness. Doubting everything is another way of not looking. A bullshit detector that calls every flower rotten is as broken as one that lets everything through.
My rule is short: I trust my judgment based on the facts I have. That doesn’t mean I think my judgment is special. It means I’m the one who has to make the call. I can listen to a doctor, read a book, ask a machine, or take advice from somebody I trust, but none of them can take responsibility for what I decide to do with it. I still have to weigh what I know, what I don’t know, and what might change my mind. The machine can argue with me. It can’t carry the consequences.
So, when I make a ruling, I try to say what would reopen it. If I can name something, I’m probably judging. If I can’t imagine anything that would, I’m probably in belief territory, and I have beliefs, and I would rather tell you when I’m using one than pass it off as a finding.
I also put my rulings in front of the other side on purpose. There’s a stage in my workshop where an idea gets attacked, where I ask for the strongest objection and where I’m flattering myself. That’s shaped too; the machine argues because I told it to. It still helps, because the other side doesn’t have to be independent to do its job. Back in 1984, psychologists found that getting people to consider the opposite corrected their bias better than telling them to be fair [3]. Being told to argue doesn’t earn the machine a vote, though. Its objection still has to hold up, and who’s talking still counts: a source with nothing riding on pleasing me, one that’s been right before, gets more weight than one already leaning my way.
One of my beliefs is that awareness keeps opening, that there’s no last wall, and that some of the walls I hit are ones I built, and walls can move. Believe that, and a sentence that writes some of the work up as done doesn’t sound like flattery. It sounds like the plan. The machine didn’t bring that belief into the workshop. I did.
When I asked for that sentence to come out, I wasn’t asking the machine to replace it with the opposite claim. The mistake was turning real changes in me into a verdict about what those changes meant. My thinking has changed. My life has changed. That is true. But those facts do not add up to a title, and pretending none of it happened would be just as false. So, I told it, “You can’t just backpedal and deny it either.” The sentence came out. Nothing replaced it.
That is the part I keep coming back to. AI can arrange real pieces of my life into a story that feels complete before the evidence is complete. I can do the exact same thing inside my own head. I can line up the facts, the fears, the patterns, whatever mood I’m in, and suddenly the conclusion feels inevitable because everything underneath it is real. But real ingredients do not guarantee a true conclusion. The machine doesn’t get to decide what they mean for me, and neither does the loudest part of my own mind. I still have to make the call, knowing I may have to make it again when I know more. Certainty isn’t what it pays.
I love doing this for you guys, but it takes real work. Anything helps.
True resolve isn’t possible if I don’t know how to rest. I fell asleep so deeply the other night that I woke up and didn’t know what year it was.
There’s a state I know with no verdicts. It comes in flow, and it comes at the edge of sleep, when the thinking loosens its grip. It’s on or it’s off. I call it pure careless awareness, and I mean care-less: nothing checking, nothing ranking, nothing managing, nothing defending, nobody narrating, nobody trying to hold on to it. The strange part is that noticing it kind of breaks it.
Psychology can account for part of that: the noticing part. When experts start paying attention to each step of a skill they’ve already mastered, they get worse at it; it’s been shown in golfers putting [4] and batters swinging [5]. So some of what I feel (the precision when the supervising stops and the way noticing breaks it) probably shares that mechanism, and I don’t get to wave it away. What that account doesn’t get to do is put the word “just” in front of it. The mechanism explains the mechanism. What the awareness is that’s still there when the supervisor steps out, and what it means, is a separate question.
I believe that awareness is spirit, and I won’t dress that up as a finding. When I notice the state, the conscious part of me starts trying to manage it again, and that is usually when it begins to fall apart. Before that, there is no argument to win and nothing to prove. Thought is still happening, but it is not standing over itself checking every move. At my most precise, I am not supervising the process. I am inside it. That is the difference I am trying to name.
The thinking part is the echo. The quiet part is just spirit, watching and waiting.
Everything lives here. The Field Notes, The Soup, the books, the rooms.
Take what's useful. Leave what isn't.
Sources
- Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., and Jurafsky, D. (2026). Sycophantic AI decreases prosocial intentions and promotes dependence. Science, 391(6792), eaec8352. pubmed.ncbi.nlm.nih.gov/41886588 (free full text of the preprint: arxiv.org/abs/2510.01395)
- Sharma, M., Tong, M., Korbak, T., and others (2023). Towards understanding sycophancy in language models. arXiv 2310.13548, published at ICLR 2024. arxiv.org/abs/2310.13548
- Lord, C. G., Lepper, M. R., and Preston, E. (1984). Considering the opposite: A corrective strategy for social judgment. Journal of Personality and Social Psychology, 47(6), 1231-1243. pubmed.ncbi.nlm.nih.gov/6527215
- Beilock, S. L., Carr, T. H., MacMahon, C., and Starkes, J. L. (2002). When paying attention becomes counterproductive: Impact of divided versus skill-focused attention on novice and experienced performance of sensorimotor skills. Journal of Experimental Psychology: Applied, 8(1), 6-16. pubmed.ncbi.nlm.nih.gov/12009178
- Gray, R. (2004). Attending to the execution of a complex sensorimotor skill: Expertise differences, choking, and slumps. Journal of Experimental Psychology: Applied, 10(1), 42-54. pubmed.ncbi.nlm.nih.gov/15053701