For Global Learning Metrics, Ask Cognitive Scientists

By Helen Abadzi, College of Education, University of Texas at Arlington.

learningThe December of 2016 brought worrisome messages to many Ministries of Education.  The 2015 PISA and TIMSS[1] scores were publicized.  Despite efforts, Latin American countries have low scores, as do the wealthy Gulf States.  Policymakers are unsure how to improve instruction for better outcomes.

The results’ release coincided with several regional workshops to contextualize the Sustainable Development Goal 4 (SDG 4).  The targets were developed after consulting with thousands of people who were related to education in some ways.  Agreement could only be reached in the broadest terms: e.g. “Percentage of children and young people in grades 2/3 achieving at least a minimum proficiency level in reading and mathematics, by sex.” [Global indicator 4.1.1]

Now comes the hard part. How to set valid benchmarks of “minimum proficiency” in reading and math for Yoruba, Arab or Lao children in grades 2/3 or justify comparisons with the Finns and Chinese?

Apparently no one knows. Thus far, the only emerging benchmarks are for more monitoring and consultations. One proposal is for subject experts to meet regionally, discuss how learning progress in each subject is understood in different contexts, and which assessment questions best capture learning progression.  Organization and funding will be needed, and any answers may be years and millions of dollars away.

In the meantime, policymakers despair over test scores. Some wish that this enthusiasm for testing spread into curricula, teacher training, textbooks, and instructional time. But the SDG consultations focus on lofty, big-picture strategies. The “translation” job is left for local actors to figure out.

Is it possible to shortcut the looming years of uncertainty and teach students efficiently?  Most certainly, if someone asks cognitive scientists. But readers must be warned that some answers run counter to popular beliefs.

The unifying principle for international comparisons is the DNA of homo sapiens.  Our brains think in similar ways, say the writings of Dr. William Huitt, Professor Emeritus of Valdosta State University in Georgia, USA.  Despite individual differences, humans process information similarly across countries and cultures. This bedrock of human commonalities could be used to build SDG benchmarks.

An obvious aid for benchmarking is “working memory,” would say Dr. William Klemm, senior professor of neuroscience at Texas A & M in the US and a prolific author on memory topics. Working memory contains what is in your mind right now and accurately maintains information that is necessary to work through problems.  But its capacity is very limited; it holds 4-7 items for maybe 12 seconds. This gives us just seconds in which to comprehend, calculate, make decisions.  We must be able to act rapidly and effortlessly.

Thus the holy grail for comparing performance is execution speed and automaticity. Global metrics could be developed for fluency in reading, math, science, various other skills.

Dr. Pierluigi Zoccolotti, professor at University of Rome psychology department, finds this a promising approach.  Words per minute are a rough but easy way to gauge whether students retain enough text in working memory in order to understand it. For that, students must know the language and recall meanings instantly. If they must exert mental effort, they use up extra milliseconds, lose their trail of thought and may answer incorrectly. Students taking TIMSS essentially engage in the battle of the milliseconds.

But how to speed up processing to the point of instant, effortless, automatic performance?  The nervous system demands practice. Classroom time and homework must offer sufficient practice opportunities, according to Dr. Craig Speelman, of Edith Cowan University in Australia. Improvement is rapid at first, then slows down. Learning curves have mathematical trends.  They could help predict the amounts of time children would attain various objectives and the number of instructional hours.

How else to speed up reading?  The size and spacing of letters read are critical, says Dr. Marialuisa Martelli, University of Rome.  Our visual system slows down if the letters are too small or too dense.  Optimal spacing and size will shave seconds off the reading time giving our working memory more time to consider a message before it disappears. Younger students are affected. This matters a lot in the younger ages, but adults often write texts for their own expert reading processes.  In principle, 4th grade TIMSS scores in visually challenging scripts like Arabic or Thai could improve a bit by optimizing size and spacing.

Which mental functions would improve math performance in TIMSS and other tests?  Accurate estimations are important, says Dr. Daniel Ansari of U. Western Ontario.  Our brains have an exact and an approximate number system.  Again, practice is needed to automatize the components of the exact system, and also to fine-tune the approximate system.  This way, a student may get a fuzzy estimate and see which response alternatives fit.

But practice, memorization, automaticity are not on the menu of modern schools.  They sound like narrow, sterile, traditional, 19th century education! Governments nowadays aim for creativity, discovery learning, and critical thinking.  Some fun projects are certainly important, but many activities gobble up teaching time to the detriment of practice. And neurocognitive research suggests that is putting the cart before the horse. Complex thinking is possible only when lots of prerequisite knowledge arrives instantly into working memory.  Students who were merely “exposed” to certain concepts may forget them entirely or retrieve them after a delay. Life events require instant retrieval.  To spend our money wisely, for example, we must calculate unit prices quickly, not three hours after leaving a store.

Why is speed a hard concept to digest? Perhaps because of incongruity. Memory functions are largely unconscious, so we are unaware of what we do. Please pay some attention and see how many seconds you took to read and understand this paragraph.  If you used more than about 10, you would get tired and give up.

The primacy of processing speed has big testing implications.  Most tests do not exactly measure knowledge; they only measure the knowledge that can be retrieved and applied within a few seconds. To perform, students should spend significant time memorizing and practicing sequences, which then pass as a single item into working memory. Top-scoring countries in TIMSS or PISA have historically emphasized practice.  Their students may not necessarily know more; instead, they may be able to retrieve more in fewer milliseconds.  Emphasis on fun activities worldwide may reduce practice time and eventually test scores. And governments that invested millions in modernizing methods may be at a loss to explain why.

Below are some of the ways to specify terms like “minimum proficiency.”  If cognitive scientists had been consulted about the indicators for SDG 4, they might have recommended some items like the ones below:

  • Reading speed for the primary school grades to monitor the attainment of automaticity
  • Calculating speed, magnitude processing tests, speed of single-digit operations
  • Learning rates for students in key tasks

Neuroscientists may have much more to add.  For example, research to model structural changes in developing brains may help estimating learning rates for various subjects beyond basic skills. By 2030, neuroimaging devices may assess knowledge through learners’ brain states and thus bypass some problems of international comparability. This is already feasible for some tasks. Experimentally, reading automaticity is often assessed using neuroimaging methods in ways that bypass the complexities of different languages and scripts. Thus, comparative education may morph into comparative neuroscience.

To achieve the 2030 targets, governments and international organizations must engage cognitive scientists and neuroscientists. Right now neither side knows that the other exists. Education colleges keep their distance from psychology, and their degrees rarely include courses on memory. During the 2015 consultations about the SDGs, the input of the few learning specialists was averaged with that of generalists. The consequences are now obvious.

If we continue to disregard the DNA guidance, test scores will continue to be modest. By 2030 surely more policymakers will demand help from memory functions. But students, particularly low-income students, should not have to wait until then.


[1] Programme for International Student Assessment; Trends in Mathematics and Science Study.

Helen Abadzi is a researcher at the University of Texas at Arlington.  To improve the outcomes of education investments she regularly monitors research in cognitive psychology and neuroscience.  Her publications can be found at:

>> View all NORRAG NEWSBite Blogs on Learning Outcomes

Follow this blog by email, Facebook or via Twitter @NORRAG_NEWS

NORRAG (Network for International Policies and Cooperation in Education and Training) is an internationally recognised, multi-stakeholder network which has been seeking to inform, challenge and influence international education and training policies and cooperation for almost 30 years. NORRAG has more than 4,700 registered members worldwide and is free to join. Not a member? Join free here.

This entry was posted in Uncategorized and tagged , , . Bookmark the permalink.

3 Responses to For Global Learning Metrics, Ask Cognitive Scientists

  1. PISA is a leaning tower, something of an educational tourist distraction but liable to collapse at any moment. This is a convincing analysis but the word that stands out for me is ‘fun’. If we must internationally measure anything (who was it who said ‘comparisons are odious’?) let’s start by comparing the levels of ‘Enjoyment of Education’ country by country – please read (and enjoy) my book of the same name. If schooling is intended to lead to better jobs and increased output and hence economic growth, and if that in turn is believed to lead to more contented people, why not take a short cut and ensure that education is pleasurable in the here and now?

    • H. Abadzi says:

      Is fun an efficient method to encode a lot of information? Humorous but also negative episodes help consolidate learning, but students also become habituated. So please show published research that systematic fun is closely linked to subsequent achievement. Since you wrote a book, you must have solid evidence from cognitive science about your beliefs.
      Or is your argument as valid as the Pisa tower statement? For the last 400 years the latter has refused to fail.

  2. W. R. Klemm says:

    Helen is making good points. Knowledge defines what is known and thereby sets the stage for pursuing what is not yet known. The emphasis of various governments on critical thinking, creativity, and problem solving put the cart before the horse. Knowledge must come first. For example, didn’t several of Edison’s inventions pave the way for inventing modern audio recording systems? Or in modern times, didn’t Xerox’s use of computer windows help Microsoft invent its operating system? In short, to think outside the box, you have to know what is in the box. This aspect of creativity has been amply reviewed by Mumford et al. (2012).

    Acquisition of knowledge most definitely requires memory skills, which are taught poorly if at all. A comprehensive approach to learning encompasses what I call the “Learning Skills Cycle” (motivation, attentiveness, organization, understanding, memorization, creativity and problem solving. This cycle is explained in detail in my book, “Teach Your Kids How to Learn,” coming out next month from Rowman and Littlefield.

    W. R. Klemm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s