The idea is appealing and old: build a task that depends on neither language, nor schooling, nor background. Abstract figures, no words, and you would measure reasoning in its pure state.
That has been the ambition of Raven’s Matrices since 1938, and of every “culture-fair” test since. It is partly met. Only partly, and it is honest to say why.
What non-verbal items get right
They remove the harshest factor: language. A verbal item taken in a second language systematically understates, and by a lot. On a figure matrix, that handicap disappears.
They also reduce dependence on school knowledge. You can solve a matrix without being able to read. That is real progress, and it is why these items remain the reference tool in international settings.
The problem the Flynn effect exposes
Here is the awkward fact. Across the twentieth century raw performance rose by about three points a decade in industrialised countries — and it is the abstract items that rose most, far more than vocabulary or knowledge items.
If a task measured an invariant biological capacity, independent of environment, it would have no reason to move thirty points in three generations. That it moves — and more than the others — demonstrates that it captures something dependent on era and setting.
In other words: the item designed to be the most culturally neutral turns out to be the most sensitive to cultural change. That is the central paradox of the field.
What remains cultural in an abstract figure
Several things, none visible at first glance.
The abstraction reflex itself. Sorting the world by formal category rather than by use is not universal: it is a reflex that schooling installs. Ask what a dog and a rabbit have in common — “you hunt rabbits with dogs” is a perfectly intelligent answer, and the abstract one is what the test rewards.
Graphic conventions. Reading left to right then top to bottom, understanding that a blank cell invites completion, knowing an arrow indicates direction: all learned.
Familiarity with the exercise. Answering multiple-choice items against a clock is a skill in itself, taught from primary school in some systems and never encountered in others.
The relationship to assessment. Accepting being timed by a stranger, understanding that guessing beats leaving blanks, daring to answer when unsure — these attitudes vary enormously with educational context.
The stereotype threat case
Another mechanism has been studied heavily: simply knowing you belong to a group reputed to do worse at a task degrades performance at that task, as cognitive resource goes into managing anxiety rather than the problem.
This field has seen serious methodological debate in recent years: some effects reported initially proved smaller or less reproducible than announced. The basic phenomenon remains broadly accepted; its real size and the conditions under which it appears are contested. It is a good example of what can be said honestly: something established, whose exact magnitude is unknown.
The reasonable conclusion
A scale rather than a yes or no. Non-verbal items are less culturally loaded than verbal ones, considerably. They are not neutral, and “culture-fair” promises more than the tool delivers — which the literature itself acknowledges, often preferring “culture-reduced”.
In practice, for you, this means two things. A score obtained outside the context the test was standardised on should be read with extra caution. And comparing two people who share neither a language, nor a schooling, nor a familiarity with this kind of exercise, on the basis of a score, does not mean what it is taken to mean.
It is one of the reasons our test writes its verbal items separately in English and French rather than translating them, and why it always shows a range. Our method page covers the remaining limits, including the one we cannot yet fix: standardisation.