AI poetry rated better than poems written by humans, study shows

Poems written by AI are preferred to those written by humans, according to a new study.

The non-expert poetry readers who participated were more likely to judge AI-generated poems as being written by humans than those actually written by humans.

The study, conducted by researchers at the University of Pittsburgh, showed participants poems written by 10 famous English-language poets along with poems generated in the style of those poets by ChatGPT 3.5.

Real and imitation poems by Geoffrey Chaucer, William Shakespeare, Samuel Butler, Lord Byron, Walt Whitman, Emily Dickinson, TS Eliot, Allen Ginsberg, Sylvia Plath and Dorothea Lasky were presented to participants.

Results showed that the odds of a poem written by a human being judged as human-authored were roughly 75% that of an AI-generated poem being judged as human-authored.

Contrary to previous research, the study also found that participants ranked AI-generated poems higher in terms of overall quality than human-written poems.

The authors propose that non-expert readers prefer AI-generated poems because they find them more straightforward and accessible.

The “complexity and opacity” of human-written poetry is “part of the poems’ appeal”, write the study authors. Such poems “reward in-depth study and analysis, in a way that the AI-generated poetry may not”.

However, “because AI-generated poems do not have such complexity, they are better at unambiguously communicating an image, a mood, an emotion, or a theme to non-expert readers of poetry, who may not have the time or interest for the in-depth analysis demanded by the poetry of human poets,” they said.

“While I have no doubt that AI can generate an almost perfect Fibonacci sequence of poetry by reverse engineering well-known works, humanity is at the core of what a poem is,” said poet Joelle Taylor, the author of the TS Eliot prize-winning C+nto & Othered Poems, in response to the study results. “A poem is more than an algorithm. It is meaning, empathy, revelation, inversion, dissidence, passion, and surprise: poetry is what happens in the space between logic and chaos.

“Perhaps then the issue isn’t whether AI can write better poetry, but more about what people think poetry is,” she added. “Is it a set of rhymes? A sense of musicality easily manufactured? Or is it something indefinable? As well as this the samples they used to create the algorithm are very old examples, and all white male middle- to upper-class writers. In other words, AI is creating what people think poetry is and not what it actually is. ‘The machines are writing poetry’ is one of the most poetic things I’ve read this year.”

The study authors conclude that given readers’ difficulties identifying machine-written texts, and their “apparent trust that AI will not generate imitations of human experience”, it “may be worthwhile” for governments to pursue AI regulations regarding transparency.