How to spot AI slop, discovered the hard way, over four hundred entries, in a dressing gown, at two in the morning, and ending in a place I did not care for.
I agreed to judge a technical writing contest while holding a drink. This is how most of the poorly considered decisions of my life have entered the historical record.
Four hundred entries. Nobody mentioned the figure four hundred until after I had said yes, which I now understand was not an oversight but a technique.
The first entry was fine. The second entry was fine. The third entry was fine.
I need you to understand that this is the problem.
In four hundred pieces of writing produced by human beings, you expect roughly forty to be genuinely terrible. You expect two to be incomprehensible. You expect one to be about the author’s divorce for no reason anybody can establish. Terrible is a sign of life. Terrible means somebody was in the room.
Nothing was terrible. Everything was fine. By entry thirty I had a feeling in my stomach that I associate with walking into a house where all the furniture is new.
A quick note. The contest, the entries and the people are blended and reshaped from several places and nobody here is identifiable, including, I am beginning to suspect, me.
The Spreadsheet Began on Day Two
By entry sixty I had started a spreadsheet. Not of scores. Of phrases.
The phrase in today’s fast-paced digital landscape appeared one hundred and forty seven times. On one occasion it appeared twice in the same paragraph, which suggests that the landscape had accelerated during the writing of it.
The phrase let’s dive in appeared eighty nine times, always immediately before a section in which nobody dove into anything, and frequently before a bulleted list.
Then I noticed the threes.
Everything arrived in threes. Three benefits. Three challenges. Three best practices. Three considerations for the modern enterprise.
Nothing in the real world comes in threes. Databases fail in ones, and in sevens, and one memorable Thursday in eleven. In twenty years I have never attended an incident where the causes turned out to be exactly three, arranged in ascending order of importance, each with its own subheading and a smooth transitional sentence in between.
The Rules, Which I Wrote Down at Night, in a Dressing Gown
There are twenty two rules governing honest technical writing.
I had never written them down, on the grounds that nobody needs to be told not to eat the furniture. I wrote them down that week. I wrote them in a notebook, at night, in a dressing gown, which I have since decided is the correct posture for the discovery of a law.
Here are the nine that matter.
1. It requires a grudge.
All genuinely good technical writing is powered by the author having been wronged by a vendor in approximately 2013 and never fully recovering. You can feel the heat coming off it. It is in the choice of adjective.
Four hundred entries about databases and not one of them was angry about anything. Four hundred people wrote at length about a technology that has personally ruined more of my weekends than any other force in my life, and every single one of them was in a lovely mood.
2. It requires at least one number that could get the author into trouble.
Not “significant improvements.” Not “up to forty percent.” A real number, of the specific and unhelpful kind that somebody in legal would have preferred you rounded.
The entries were full of numbers. Every one of them was safe. Every one of them was a number that had already been to a meeting.
3. It must smell of somewhere.
Real writing happens at a time of day. There is weather in it. There is a room, or a colleague eating something aggressive at the next desk, or the specific and terrible carpet of an office in a business park.
Slop occurs nowhere, at no hour, in perfect climate control, in a building with no windows and, apparently, no cutlery.
4. It must admit to not knowing something.
In four hundred submissions, nobody was confused. Not once. Four hundred authors, complete mastery of the subject, no gaps, no hedging, nobody who had tried something and could not work out why it went wrong.
I have never met a person like that. I have been to a great many conferences, which is where such a person would live if they existed.
5. It must not arrive in threes.
See above. See also the shape of this list, and hold that thought, because we will be returning to it in a way I am not looking forward to.
6. It must contain one sentence the author’s employer would rather they had not published.
Just one. It does not have to be reckless. It has to be a sentence where you can feel the author decide to leave it in.
7. It must name somebody who might read it and be annoyed.
Nothing produces prose quite like the knowledge that a specific individual is going to see this. It sharpens every sentence. Slop is written for nobody in particular and it reads exactly like that, in the way a form letter reads like a form letter even when it has your name at the top.
8. It must not end where it began.
The conclusion that carefully restates the introduction is the machine tidying up after itself, because a closed loop is neat and neatness is the whole ambition.
A human being who has spent three thousand words on a subject has moved. They end up somewhere they did not plan to be, and quite often somewhere they are not entirely happy about, which we will also be returning to shortly.
9. It must have cost the author something.
This is the master rule. The other eight are only its symptoms.
Day Nine, and Some Loss of Perspective
By day nine I was applying the rules to email.
By day eleven I applied them to a birthday message from my aunt. It passed comfortably, largely on the strength of offense one, since my aunt has a grudge against nearly everything and expresses it with real economy.
Then I did something I regret.
I ran the rules against a vendor whitepaper from 2011. Written entirely by human beings. Before any of this existed. Twelve pages about a storage appliance.
It failed twenty of the twenty two.
I sat with that for a while, in the dressing gown, with the lamp on.
We have been producing this by hand for decades. At enormous expense. Using people with degrees. Committees have been convened. Budgets have been signed off. Entire teams of expensive human beings have gathered in rooms with a whiteboard and produced, over several weeks, something functionally indistinguishable from what now costs nothing at all.
The machine did not invent slop.
The machine removed the price floor.
Then, Because I Am a Certain Kind of Person, I Ran It on Myself
You knew this was coming. I did not, which is the difference between us.
Offense five went almost immediately. I like three. Three is a good number. Three is comfortable and balanced and I have been doing it for years and I am not proud of any of this.
Then something worse.
Every post I have written for several years now ends with a sentence of the form, this is not X, it is Y.
I went and checked the archive, which I recommend to nobody. It is in a number of them I am not going to state here, because offense two applies to me as well and it turns out I am a coward.
My defence is that I have always written that way. That I liked the shape of it long before any machine had an opinion about anything. That it is genuinely how I think, and that a construction does not stop being yours simply because something else learned to do it.
My defence is pathetic.
My defence is also correct.
Those two things being true at the same time is the least comfortable room I have stood in for a while, and I would like to point out that offense eight predicted this exact outcome, which is the sort of thing that happens when you write your rules down properly.
The Entry That Won
The entry that won was, by every conventional measure, the worst written of the four hundred.
It was about a migration that had gone badly. It named a vendor. It contained a real number, and the number was awkward, and you could tell nobody had approved it.
There was a paragraph in the middle that was clearly a mistake. Some earlier draft had lived there and part of it had been left behind, the way a person leaves things behind, and reading it you could feel the exact Tuesday afternoon at which the author had stopped caring about paragraph nine.
It was also, in the closing lines, quite unfair to a colleague. I noticed this. I did not deduct for it. I am not going to defend that decision, I am simply reporting it.
It smelled of a room. It had weather in it. Somebody had paid for every line.
What I Would Tell You, Having Been Through It
Do not build your detector out of style. Style is a losing test. It improves every six months and yours does not, and anyone who tells you they can spot it by the em dashes is describing a skill with a very short warranty.
Build it out of cost.
Ask what this sentence cost the person who wrote it. Ask whether anything here was risky, or embarrassing, or specific enough to be checked, or unkind enough to be remembered. Ask whether the author knew who was going to read it and wrote differently because of that.
Nine times out of ten you will have your answer before the second paragraph, and you will not have thought about a single em dash.
The tenth time you will be wrong, and it will be a real person having a bad week, writing carefully because they are frightened, and you will feel terrible about it. That is the cost of owning a detector. I do not have a solution and I am declining to invent one, on the grounds of offense four.
This is roughly the territory of all thirty essays in my book AI: Nobody’s in There. But we’re still in here. All thirty are free to read at pinaldave.com.
I have kept the notebook. It is in a drawer, and it has twenty two rules in it, and every so often I take it out and read rule five and put it straight back.
This is not a writing problem, it is a cost problem, and yes, I have seen the shape of that sentence, and yes, I am leaving it in.
Reference: Pinal Dave (https://blog.sqlauthority.com/), AI Slop, X
