Is it possible to measure how close two words are to each other? Cat & kitten, teacher & classroom, hope & optimism, for example. Surprisingly, yes. This is what researchers call semantic similarity‘ – a way of quantifying how related words are in meaning.
Because it feels intuitive, many people assume that if the words in someone’s speech are semantically related, then the speech itself must be coherent. After all, coherent discourse tends to stay within a topic area, so you’d expect the words to cluster meaningfully. This is exactly why semantic similarity became such a popular proxy in computational linguistics & clinical research: if the words stay “close”, perhaps the ideas do too.
But coherence is something much broader: it’s the way ideas link across sentences, how topics develop, how a listener follows the thread of meaning over time. Semantic similarity captures lexical closeness; coherence reflects conceptual connectedness.
A new study in Scientific Reports puts this long‑standing assumption to the test using multilingual datasets & clinical speech samples from people with psychosis. By comparing semantic similarity to human coherence ratings – alongside 131 other linguistic features – the researchers show just how limited similarity is as a window into real-world communication.
The study
The team analysed speech across three neurotypical corpora (English, Chinese & Danish) & a clinical dataset of 94 speakers, including individuals with first-episode psychosis, chronic schizophrenia & high-risk groups. They extracted 131 linguistic features – such as semantic similarity, graph-theoretic measures (network-style measures that map how ideas connect across a piece of speech) temporal structure, surprisal (a measure of how unexpected a word is in a given context), perplexity (a measure of how predictable a whole sequence of words is) – then compared each one to human coherence ratings.
They then compared all of these features to human ratings of coherence to see which ones actually matched what listeners perceive as “making sense”.
In a second step, they looked at how these linguistic patterns relate to the brain. Using high-resolution fMRI scans, they examined how coherence connects to large-scale brain organisation – essentially, how different regions communicate when speech is more or less coherent.
The findings
1. Semantic similarity doesn’t tell us whether speech is coherent
Across English, Chinese & Danish, the “closeness” of words didn’t match how coherent listeners judged the speech to be. In fact, some of the least coherent speech had more semantically related words – the exact opposite of what many researchers expected.
2. Out of 131 language measures, only a tiny number linked to coherence – & only weakly
The team tested over a hundred ways of describing speech. Only six showed any reliable connection to coherence, & even those were small. These included measures of how ideas link across sentences & how predictable the unfolding speech is.
3. In psychosis, incoherence is real – but similarity doesn’t capture it
People with first-episode psychosis produced speech that listeners rated as less coherent. But semantic similarity didn’t pick this up. Instead, the best indicator was how unpredictable each next word was – speech with high unpredictability tended to be harder to follow.
4. Coherence depends on whole‑brain organisation, not just words
Brain scans showed that more coherent speech was linked to broader patterns of brain organisation – how different regions communicate across a large‑scale hierarchy. This suggests coherence is a global cognitive process, not something you can detect by looking at word meanings alone.
Why this matters
This is abstract research, far removed from the classroom, but it offers a valuable reminder: coherence isn’t about choosing related words, it’s about connecting ideas. Learners may produce semantically related vocabulary yet still struggle to build a coherent narrative or argument. That’s a discourse-level skill, not a lexical one.
Teacher Takeaways?
- Teach coherence explicitly: Focus on topic development, logical sequencing & discourse markers that guide listeners through ideas.
- Use prediction-based tasks: Activities that encourage learners to anticipate the next idea mirror the probabilistic processes linked to coherence.
- Remember that similarity isn’t coherence: Tools that measure how related words are won’t show whether a learner’s message is easy to follow.
Do you ever work on coherence as a skill in its own right?
Leave a Reply