Trying to get across this idea that AI-generated writing can be syntactically sophisticated without being semantically sophisticated. A plainer language version, at least to my mind, is that a lot of AI writing feels hollow.
The sentences often look right. The paragraph has a shape (short sentences like this will likely become a “tell”). The language may even sound more academic than what a student would have written on their own. But when you press (is that more human or more AI for a verb?) on it, there is not always much there. The writing seems to know what academic writing is supposed to sound like, but not (oh look, a “not”!) necessarily what this particular piece of writing is supposed to do.
I wonder if computer scientists have a similar experience when they see vibe-coded code, if something just feels “off” to them. Maybe the code runs, and maybe it looks impressive to a non-specialist, but someone with more experience can see that the choices are not quite right. Not obviously wrong, necessarily. Just not fully thought through.
Now, of course AI is going to get better at all of this over time, so my minor version of the uncanny valley might go away. The obvious tells will probably change or disappear. But I still think something will be missing for particular genres, especially those genres where audience awareness is really key.
I suppose I could just be focusing on the wrong thing. Maybe the real issue is the cognitive offloading piece. If someone vibe-writes a paragraph, for example, like “Please turn this idea into an academic-sounding paragraph,” and gets that immediate hit of “Yeah, that makes sense” when they see the output, they may move on without going through the thinking process required to produce something genuinely effective that aligns with both their purpose and their audience.
That may be the hollow feeling. The paragraph exists, and it sounds enough like academic writing to pass a first glance, but the writer may not have made the decisions that give the paragraph its real force. Why this claim? Why this evidence? Why this level of detail? What does this reader need next?
A colleague thinks the brick wall will be the reader. Readers will learn what feels off about AI-generated writing. In other words, their pattern-seeking will adapt, like it has with em dashes or the overuse of semicolons. But I am not so sure that is true, especially given reader attention span, and apparently a drop in literacy.
The neighbouring claim in my public notes is simpler: writing is the thinking.