Large Language Models (LLMs) have become fluent in dozens of languages — & they weren’t even trying to pass an ELT placement test. So, can they learn any language? Even a made-up one?
That’s the question behind Mission: Impossible Language Models, a recent award-winning paper by Julie Kallini & colleagues at Stanford that could shake up how we think about both artificial intelligence & how humans acquire language.
While Noam Chomsky & others (Chomsky et al., 2023) have argued that LLMs are irrelevant to linguistic theory — precisely because they can (allegedly) learn impossible, unhumanlike grammars — this team wasn’t so sure. Couldn’t LLMs, if carefully studied, offer a window into why some patterns are learnable & others aren’t?
To investigate, they trained multiple versions of GPT-2 (a well-known LLM) on artificial languages derived from English, but with a twist: these grammars were designed to break the rules of natural human language. Some examples?
- Shuffled word order (random or systematically jumbled)
- Backwards clauses
- Verb markers that appear four words after the verb
Each “impossible” language came with its own logic — but no connection to what humans actually do. The researchers tested whether GPT-2 could learn them as easily as it learns English.
What they found was surprising — but not in the way Chomsky predicted.
Despite being trained on the same data size, models learned natural or near-natural grammars much more effectively. They showed that the most “natural” of the impossible grammars (the NOHOP (Non-Human Order Protocol) condition, where verb agreement markers directly follow the verb, mimicking English) were learned most successfully — suggesting that even LLMs are biased towards information locality: the idea that words that depend on each other tend to be nearby. This aligns with psycholinguistic research on human language processing (Futrell, 2019; Hahn et al., 2021).
Grammars that defied information locality — the principle that related words tend to appear close together — were hardest to learn. GPT-2 was consistently worse at learning impossible languages. It achieved lower predictive accuracy, took longer to “settle” on grammatical patterns, & relied on awkward internal workarounds. In fact, models trained on grammars with random or arbitrarily shuffled word orders never learned them effectively. Those with more systematic but still unnatural rules (e.g. placing verb markers at a distance) were learnable — but only with difficulty. Even AI, it seems, prefers its grammar human-shaped.
So while LLMs don’t learn like babies — no one’s claiming that — they also don’t learn just anything. Their inductive biases matter. That alone casts serious doubt on the claim that “LLMs can’t distinguish possible from impossible languages.”
This matters. It challenges the claim that LLMs are indiscriminate pattern recognisers. It also suggests they might share some biases with human learners — not because they’re “like us,” but because language has constraints rooted in cognition, not just code.
As the researchers put it: “Models & humans exhibit fundamental differences… but the extent to which models favour natural languages is influenced by design.” In other words, AI isn’t just useful for automation — it might help us understand acquisition, too.
Teacher Takeaways?
While this research won’t help you sequence a grammar lesson, it hints that learners—like AI—might struggle more with rules that feel ‘unnatural’ (e.g., distant verb markers). Finally, we’re moving beyond theory into evidence.
Have you ever shown learners a list of grammar rules to see if they can spot which ones are nonsense?
Leave a Reply