The not-X-but-Y machine: Differentiating non-human articles from human ones
This will be short since a lot of similar articles exist.
This takes forward the inference drawn from an article by Sean Goedecke that concludes that em-dashes are not a modern phenomenon and were used by authors way too much between late 1800s and early 1900s. Since the SOTA models around October last year included high quality literature from that time period that had ~30% more em-dashes than all the other literature, they used to produce a lot of em-dashes and even now continue to do so. It made any piece of writing look so visibly AI generated. Even multiple lines of prompt wasn't sufficient for them to be able to stop doing that.
Since then models have evolved, and can stop producing em-dashes by just a single line of prompt. Consequentially there have been other patterns that the modern SOTA language models produce. I'll list down a few common ones:
- 'it's not X, but Y': This is the most commonly used trope that is visible across the entire spectrum of AI writing.
- 'it is worth mentioning...': Announcing the thing rather than just doubling down on the point. More similar patterns include 'it's important to note that...', 'it should be remembered...'
- 'while X is correct, Y also may be correct': Refusing to pick a side. Very damage controlled rhetoric.
The not-X-but-Y syndrome occurs from the modern training data that heavily includes posts from reddit, linkedin, quora where there's a correction culture and marketing nuances.
It is also a mathematical outcome of RLHF and DPO (optimization processes) being similar to contrastive learning in their equivalency. They directly involve a Human rater who prefers a response that validates and then empathizes. During optimization, it is spiritually baked into the process. This also directly leads to model staying more empathetic than human counterparts. Coupled with the modern training data, it is just a natural outcome.
The above patterns appear more than often in an average piece of writing generated by AI. The tone always feels balanced and there are no sharp twists or opinions. Human writing is very opinionated, and right from the first sentence you can feel so. Do I think AI should be used less? No. If anything, the training process should be made more human-centric when using the creative writing mode.
Fin