For centuries, people have dreamed of undoing Babel.
Sci-fi novelists envisioned universal translators, and linguists devised international languages, all in pursuit of a world where one person could speak and another could understand, regardless of where either was born.
Artificial intelligence appears to be taking humanity one step closer toward that goal.
AI-powered tools are already being widely used by lawyers to translate legal documents from one language to the next. The mass market romance publisher Harlequin has turned to AI to translate its novels for international audiences. And hospitals are deploying AI translation to communicate directly with patients in multiple languages.
The speed and skill with which these AI-powered translation tools operate are certainly impressive.
But there is an important frontier for translation technology, one that it might never be able to breach: the poem.
That’s because translating poetry, thus far, has been a uniquely human experience. It demands intimate knowledge of two languages, which large language models certainly possess. But it also requires a mastery of different cultures and perspectives, what literary critic Gayatri Chakravorty Spivak calls a “worlding” of language and culture.
Pushing the limits of language
When scholars have studied the creativity of chatbots by prompting them to produce poetry, they’ve noticed that the poems tend to be more homogeneous and standardized than those written by humans.
Chatbots’ poetry translations have similar issues.
AI seems to struggle in three main areas: rendering metaphor, decoding complex sentence structure and creatively conveying mood or emotion.
To demonstrate these flaws, I worked closely with Adeeba Shahid Talukder, an award-winning poet and translator, to write this piece and to translate the 1953 poem “Mulāqāt,” or “Meeting,” composed by one of the most famous Urdu poets of the 20th century, Faiz Ahmed Faiz.
Wikimedia Commons, CC BY-SA
Faiz was born in 1911 in Sialkot, in pre-Partition India. In 1947, when that area in the Punjab became part of present-day Pakistan, Faiz became a citizen of the newly founded country, though his relationship with it was marked by both hope and disillusionment. He lived in different parts of the world for long stretches of time, but he returned to Pakistan toward the end of his life. He died there in 1984.
His poetry is celebrated for its marriage of classical tropes like the ache of unfulfilled longing for the beloved — and the lover’s willingness to be consumed or destroyed in his pursuit of her — with the political struggles of his time, like the plight of working people and the dream of a Marxist revolution. His poetry is notoriously difficult to translate because of its highly ornate, extended metaphors, which have few easy equivalents in English. This forces the translator – be it human or machine – to push the limits of language.
From 1951 to 1955, Faiz was imprisoned on allegations of sedition against the Pakistani state. His first three months were spent in solitary confinement.
From Faiz’s letters, scholars learned that despite the isolation of his imprisonment, he felt as if the experience made him more attuned to his senses, as if he were “falling in love again for the first time.” His time in jail was also among his most prolific as a poet, with “Meeting” composed during his sentence.
Here’s our translation of the opening verses of “Meeting”:
This night is the dark,
lush tree of a grief greater
than you and I—
greater, for caravans
of torch-bearing stars
have lost themselves
in the siege of its branches;
in its shadow, a thousand moons
have wept away all their light.
This night is the dark,
lush tree of a grief greater
than you and I—
but from the tree of this very night
the amber leaves
of these moments
have fallen,
grown entangled
in your locks and flowered
as the fires of pomegranate
blossoms; its dew
has rained drops of silence
upon your brow,
bedecked it with a silver
string of pearls.
When prompted to provide a “lyrical translation into English,” ChatGPT produced the following:
This night is a tree of pain,
Vaster than you, vaster than me.
Its branches bear
Thousands of torches in place of stars,
Caravans lost
Within a thousand moons’ shadows,
And all my light has wept itself away.
Yet from this very tree,
A few golden leaves of fleeting moments
Fall into your hair,
Entwined, they bloom like roses.
From its dew in quiet,
A few drops rest upon your brow,
And glitter like tiny diamonds.
Chatbots fail at translation’s most basic task
When Faiz composed “Meeting,” he had been separated from his wife and two daughters for two years. He was keenly aware of missing parts of their childhoods that would never return.
This biographical information can help translators understand the poem’s exploration of “vaṣl,” or the meeting of lover and beloved. A common trope in classical Urdu poetry, it usually captures the joy and exultation of the lovers’ union.
But in Faiz’s poem, the union with the beloved also contains an acute awareness of mortality and the transience of beauty – a recognition of what has been lost, and the suffering still to come.
That’s why we rendered the opening lines of “Meeting” as “This night is the dark,/lush tree of a grief greater/than you and I.”
ChatGPT’s translation is more literal: “This night is the tree of pain.”
While there is nothing technically wrong with this translation, the chatbot’s version doesn’t capture the nuances of the tree metaphor and the way its dense, expansive branches can encompass the complexity and beauty of the emotions evoked by the night of the lovers’ union.
AI also fails to grasp the poem’s intricate sentence structure. ChatGPT has translated “in its shadow a thousand moons / have wept away all their light” as a nonsensical construction: “Within a thousand moons’ shadows, / And all my light has wept itself away.”
This error appears to have happened because the chatbot translated “apnā” – a reflexive possessive pronoun in Urdu – as “my,” inaccurately parsing it as referring to the speaker instead of the moons.
Finally, and most importantly, AI models lack the ability to express emotion the way a human can. A machine with no bodily experience of being human cannot meaningfully perceive a poem so enmeshed in human experience. Its engagement is merely superficial.
In its attempt to convey the mood of the original piece, ChatGPT offers: “From its dew in quiet, / A few drops rest upon your brow, / And glitter like tiny diamonds.”
It’s clear that ChatGPT is struggling to decode the grammatical structure of the poem and is trying to make the text lyrical enough to convey the awe and wonder of the original. But the model’s contrivances toward the lyrical – for example, describing diamonds as “tiny” or “glittering” – have no relation to the original poem.
“From its dew in quiet” is an incoherent clause. The phrase that seems to have thrown the model off is “isī kī shabnam se khāmoshī ke yeh cand qaṭre,” or “its dew / has rained drops of silence.”
Urdu literary critic Shamsur Rahman Faruqi has explained how Urdu poets “treat metaphor as fact and then go on to create further metaphors from that fact.” In “Meeting,” the metaphor of the night of the reunion as a tree becomes a fact, which allows for the flowering of a new metaphor – that of the dew on its leaves as drops of silence that fall on the lover and their beloved. These silences, heavy with sorrow, then adorn the beloved like precious jewels, conveying the idea that only a profound grief can beget such beauty.
The model has failed at conjuring this sense of wonder because it cannot parse the poem in accordance with the literary conventions of Urdu poetry.
ChatGPT prefaced its translated text by assuring us that it had “crafted a lyrical, poetic English version of Faiz’s ‘Mulāqāt,’ keeping the imagery, rhythm, and emotional flow intact so it reads like a poem rather than a literal translation.”
Yet as we have shown, its translation fails to accomplish the most basic task of literary translation: to convey the heart of the original.
Chatbots, in other words, are a poor substitute for the literary translator, and they bolster the assertion of the late Indian poet, scholar and translator A.K. Ramanujan that “only poems can translate poems.”
Adeeba Shahid Talukder helped with the research and writing of this article, in addition to the translation of Faiz’s poem.
