I wrote a book about AI in business, about possibilities in practice. I used AI throughout: for research, drafts and cross-checking. That was the honest thing to do. A book about the technology should be written with the technology, at least as a test of what it could write.
The drafts came back and they sounded like me. The word choices were mine, the structure made sense, the paragraphs read the way I would have wanted them to read. I kept them open for a while and reread them. Then I understood what was wrong. The voice was mine, yet AI had only assembled the sentences and had never worked them through into anything I had thought.
Correct and true are not the same thing. A machine can produce a correct sentence, one that is formally sound, plausibly built, free of obvious error. It cannot produce a true one. A true sentence needs someone who means it, someone who has been through the thing the sentence is about and has found the right words for it rather than the optimised ones. The machine produces the output but skips the thinking that would stand behind it.
This is easiest to see in writing, because writing is thinking. When I write a sentence I find out where my understanding stops. The gap shows up on the page as a passage I cannot finish, an argument that will not close, a claim I cannot back. Hand the writing to a machine and the gap gets filled in smooth prose. The prose is smoothest where no thinking occurred.
So I wrote the book myself. I used AI as a tool and kept the writing to myself. It searched, drafted alternatives and argued against me when I asked it to, and every one of those uses was real. None of them was the writing. That turned out to be the most important thing the project taught me, and it shows in how I made the book.
The point generalises beyond writing a book: it applies to any application of AI. The output arrives without the thinking that would make it true, and the smoother the output, the harder that absence is to notice. A book about AI that refutes itself through its own making still does its job: in the one place the reader can check, the prose, it marks the limit of the technology.