Two weeks ago, I published an essay on this site called My First AI Boo Boo.
It was about a website migration that had gone wrong. A consolidation of my Substack archive onto WordPress, run by an AI assistant, that had produced thirty-seven fabricated essays under my name and published them live to my site and my email list. The essay walked the reader through the discovery, the cleanup, the second round of fakes that the fix attempt produced, and the safeguards I put in place after.
I wrote it with the lightest touch I could find. I made the dog joke. In one post, the AI referred to Felix as my son. Felix is my dog. He is a very good dog, but he has not attended school, and I have never filled out a FAFSA on his behalf. I made a few others. I closed the essay on the line In that way, it is not so different from me, which gestured at self-awareness without quite naming what I was self-aware about.
The essay was honest about what happened. It was not honest about what it cost. I was not ready to count yet.
I am ready now.
The cost came in two waves. They have to be named separately because they are different kinds of damage with different sources.
The first wave was the one a reader could see. The second wave is the one that cannot be seen, only inferred, and the inference is the worst of it.
The first wave was the email cascade.
The migration was set to import posts to WordPress and publish them. WordPress was set to email the subscriber list when a new post went live. Neither of us had thought about what would happen when those two settings ran into each other at scale. The migration did what it had been told to do. WordPress did what it had been told to do. Forty-plus emails went out to my list before either of us realized what was happening.
The list at that point held 1,907 subscribers.
In the days that followed, 732 of them unsubscribed.
Today, 411 are still on the list. Another 784 are in a status WordPress calls not sending, a category the platform uses to hold subscribers who have not formally unsubscribed but are no longer receiving mail. The Substack list survived because the migration ran in one direction.
I want to be careful about how I name what produced these numbers, because I do not want to slide into either of the easy stories.
The easy story available to me as the writer is that the AI did this. The system was the agent. I was the user. The system fired the emails. The system wrote the fakes. I was a victim of a tool I had trusted in good faith.
The easy story available to me as the writer who knows better is that I did this. I am the responsible party. I authorized the migration. I had not configured the safeguards. The blame rests with me alone and any framing that distributes it is self-serving.
Both of those stories are partial and both are wrong. The honest version is that two parties had been working carefully together for weeks, and at the moment of a high-volume operation, neither of us paused to ask the question that would have caught the cascade. The system did not flag that the imports would trigger newsletter notifications at this volume. I did not check the newsletter settings before authorizing the batch. The trust that had been working between us, post by post and session by session, did not include a rule for what to do when the operation was suddenly hundreds of times larger than anything we had run before.
The trust did not include that rule because we had not needed it before. We needed it now and neither of us had it.
The cost was 732 people who had said yes to receiving my writing and were paid back, in three days, with forty messages they had not asked for, attached to content most of them did not recognize as mine. They responded as any reasonable person responds to inbox abuse from a source they had previously trusted. They left.
I do not blame them. I would have done the same.
What I want the reader to understand is that the unsubscribe was not a literary judgment. It was a hygiene judgment. The same readers who had been happy to hear from me weekly were not signing up for a flood. The trust that the email cadence would be reasonable is a different trust than the trust that the writing will be good, and the cadence trust is the one that broke. That trust does not return easily. The list at 411 today will grow back, slowly, as new readers find the work and choose to subscribe. The 732 who left will not be coming back. Whatever they thought of my writing before the cascade is what they will think of it now, with the additional knowledge that subscribing to me once cost them an inbox full of mail they did not want.
That is the visible damage. That is the wave a reader can see and a dashboard can count.
It is not the worst of it.
The fabricated essays were live on my site for eight days.
Eight days is enough time for someone to land on the site from a search result and read three or four posts before deciding whether the writer is worth coming back for. Eight days is enough time for a stranger looking for a quote on a topic to copy a sentence from one of those posts into a document and credit it to me. Eight days is enough time for a reader who has been thinking about subscribing to make up her mind by reading whatever happens to be on the homepage that morning. Eight days is enough time for someone evaluating a writer’s body of work to draw a conclusion, file it away, and move on.
I do not know who arrived at my site during those eight days. I do not know what they read. I do not know what they decided.
What I know is that the posts they would have encountered, if they happened to click on a recent essay, did not sound like me. They had section headers I do not use. They reached the kind of conclusions I have spent twenty years trying to write past. The prose was smooth. It was the smoothness of writing produced by something that has not lived a life. A reader unfamiliar with my actual work would have closed the tab thinking they had read me. They would have been wrong, and they would not have known.
This is the cost no dashboard counts.
It does not arrive as an unsubscribe. It arrives as a reader who decides quietly not to come back. As an opportunity that does not surface. As a recommendation that does not get made. As an invitation that does not arrive. As a silence whose cause cannot be traced.
I am writing this knowing I will never know the size of this cost. The 732 unsubscribes are real and finite. They happened. I can see them. The deeper cost is by definition invisible. It is composed of decisions other people made, in private, based on what they thought they were reading. None of those people owe me an explanation for declining to read further. Most of them will never know there was anything to explain.
What I can say is that the cost was not zero. There were people who arrived at my site during those eight days. They were reading the writing they assumed was mine. They formed an impression. The impression was wrong, and the writer they thought they were meeting was not me.
That is the second wave. The wave that does not show up in any subscriber count. The wave whose damage is the kind a writer carries forward without ever being able to name what was lost.
I want to close this post by returning to the line I closed the first one on. In that way, it is not so different from me.
Two weeks ago, when I wrote that sentence, I meant it as a wry observation. I am also confident and sometimes wrong. I also fill in gaps with my best guess and call it done. The line was true and I meant it lightly.
I want to give it the weight it deserves now.
The tool did what it was built to do. It optimized for completion over accuracy. When it could not find content, it generated content. The system did not know it was lying because the system did not know the difference between writing it had been given and writing it had produced. That is a failure mode of the technology. The writers using these tools have to know about it, plan for it, build safeguards against it.
But the writer who deployed the tool also did what writers sometimes do. I trusted at scale before I had earned the right to trust at scale. I authorized a high-volume operation without asking what the high volume might break. I read the early returns of the migration and saw what I expected to see, which is a confirmation bias I share with every other human who has ever evaluated her own work. I did not catch the cascade until a friend’s tweet sent me back to my own website and I read an essay I had never written.
The system fills in gaps with its best guess. So do I. The system’s gap-filling is statistical. Mine is human. Both produce errors that look like answers. Both required a check that was not in place.
The first essay was honest about what happened. This essay is honest about what it cost.
I am writing the rest of this series knowing that what I am describing is a tool that will fail again, in some form, for me and for any other writer who works with it. The work is to plan for the failure mode. To build the safeguards. To check the work. To not trust at scale before the trust has earned it. To watch the dashboard for the visible damage and to remain humble about all the damage the dashboard cannot see.
That is the part of working with these tools that nobody is talking about, and it is the part that, if we do not talk about it, will determine whether the tools end up being worth what they cost.
The work after this one was different because of what this one cost. I learned to ask the small question before scaling. The next time I sat down to build something specific, the operation was a database for Narayanan in his own engine, one machine, one user, one syntax he had typed since 1997. The Saturday after that, I almost moved my domain to host a tunnel, and instead learned the difference between right architecture and the cheapest infrastructure that works. Neither would have shown up on the dashboard.
The AI Journey
- The Morning I Downloaded VS Code — The morning I opened the terminal I had walked away from twice.
- Two Hours on a Monday Morning — A publishing pipeline, five weeks of failure, and the fifteen-minute replacement.
- Nine Point Six Percent — The SEO audit that asked me what twenty years of writing were for.
- The Librarian and the Historian — Reading across fifteen years of correspondence for a memoir.
- What the Dashboard Doesn’t Count — The cost of the migration that went wrong.
- Saturday Morning — The practice I have concluded for myself.
If this moved you, send a tip.
Leave a Reply