Analyzing the Mistakes in Grokipedia's Entry for Alliterative Verse
Apparently, Elon Musk has just launched Grokipedia, based off GenAI, as a way to compete with wikipedia (which it's probably plagiarizing).
But given that the Modern Alliterative Revival is a hobby of mine, I decided to check out Grokky's entry on Alliterative Verse, particularly its sections on the Modern Revival. Everyone knows GenAI is prone to hallucations, so I wanted to see how well it did with something I know well.
MY VERDICT: Grokipedia is ... surprising accurate ... but also dangerously almost accurate.
Solely as a predictive algorithm, Grokipedia said many true things about the Modern Alliterative Revival. (Unsurprisingly, it cites my work frequently, not to mention Paul Deane's website Forgotten Ground Regained.) In fact, it says enough true things that if Wikipedia were to disappear tomorrow, any lay person might even find Grokky helpful.
Nor did I find any outright GenAI hallucinations. I checked every passage of poetry from Tolkien, Lewis, and Auden that Grokipedia cited, and they're all 100% fully accurate ... a major problem for other GenAI programs like ChatGPT.
But Grokipedia also subtly wrong, too -- a lot. It's terrible with poetical analysis, for example; the principles it hopes to demonstrate are rarely within the passages of poetry it cites. It cannot distinguish parodic information from real information. Citations are vague; sometimes, they're even irrelevant.
Here's my evidence, in no particular order, on a partial analysis of Grokipedia's alliterative page,
Oopsie #1: the parody article.
Brooke-Rose, Christine (1963). "Notes on the Metre of Auden's 'The Age of Anxiety'". Essays in Criticism. XIII (3): 253–264.
Notably, Brooke-Rose claims this article was written in the year "2185" after the "first Atomic War," and she's clearly making fun of scholarly assumptions often made about the 14th-century Alliterative Revival. Yet it's cited seriously by Grokipedia. Grokky also implies Brooke-Rose is talking about, among others, Richard Wilbur and Ted Hughes .... but she's not.
Poetics fail. Grokky then suggests that poems like Auden's The Age of Anxiety place "stress and alliteration on grammatical function words". Which .... okay, maybe sometimes. But then Grokky quotes -- accurately -- a passage from The Age of Anxiety. Although these lines aren't in proper OE meter, the stresses, notably, all look fine to me. Grok's claim about putting stress on "grammatical function words" is nonsense, then, but it seems to be borrowing random true information (without knowing what it means) that Auden substantially modified the OE meter.
What's especially interesting, though, is how Grokky decided to select this exact passage from The Age of Anxiety. Neither citation surrounding this excerpt actually cites that excerpt. I have no idea where Grokky's getting its (bad) analysis, but it's clearly not based on the sources it purports to be using.
Oopsie #2: Tolkien & Old Norse.
Here's a doozy of a mistake:
GROK: "He [Tolkien] also experimented with alliterative verse based on the Poetic Edda (e.g., the Völsungasaga and Atlakviða) in The Legend of Sigurd and Gudrun (2009)."
- Sorry, Grokky, but Völsunga saga is famously a PROSE text. It's not a poem. The material on which Völsunga saga is based, however, does come from various alliterative poems in the Poetic Edda. So Grokipedia is making a mistake, but the kind of mistake a "B"-level college undergraduate might make.
Poetics Fail. "Like Tolkien, [C. S. Lewis's] poems follow the rules of Old English alliterative verse, while maintaining modern English diction and syntax, as can be seen from lines 562-67 of The Nameless Isle."
- Again, Grok's accurately cites a passage from The Nameless Isle, and Grokky's scansion even gets the alliterations right. But I'm not quite sure what "maintaining modern English diction and syntax" means, or how the quoted passage demonstrates that.
Oopsie #3: Middle English, ftw?
GROKKY: "a Middle English alliterative poem could refer to men by such a variety of terms as were, churl, shalk, gome, here, rink, segge, freke, man, carman, mother's son, heme, hind, piece, buck, bourne, groom, sire, harlot, guest, tailard, tulk, sergeant, fellow."
- Right off the bat, Grok seems to have included some MODERN English words in its list of Middle English "synonyms".
- The other thing, though, is that the source Grok cites doesn't contain any of that information. Although MdE does have plenty of poetic synonyms for "man," I have no idea where Grokky's getting its information. The citation is just a random but totally irrelevant source.
Oopsie #4: Poetics Fail (massive).
- This is just nonsense. Although MdE does have three alliterations, oftentimes, that has no logical connections with the Parson's claim. Grok was mislead by Chaucer using three alliterating words in a row, but of course the Parson is being parodic. Notably, his actual prologue appears in iambic pentameter with heroic couplets.
Oopsie #5: Odd but Interesting Citation Fail.
- This is interesting. Although Grok's statement is objectively true, it's citation doesn't actually confirm that. Instead, the citation directs us to a single modern alliterative poem by Theresa Werba. One example obviously doesn't prove or confirm anything, so that's a problem of logic, but Werba's poem was only published a month ago. So Grokky was quick, at least, to find it.
Comments
Post a Comment