Schools and governments have spent a substantial amount of money on classroom technology over the past two decades, computers, tablets, interactive whiteboards, and more recently AI-powered platforms, on the reasonable assumption that more technology should mean more learning. The actual research on this question is more uncomfortable, and more interesting, than that assumption suggests.

Some of the strongest available evidence shows that heavy use of general classroom technology is associated with worse outcomes than moderate use, and that broad access to computers does little to close the gap between advantaged and disadvantaged students on its own. Other, equally strong evidence shows that a specific, well-designed category of technology can approach the effectiveness of one-to-one human tutoring, historically one of the most powerful interventions in all of education research. Both of these findings are true at once, and the difference between them is the entire story.

The uncomfortable finding: the ‘ICT paradox’

The most influential evidence here comes from the OECD's landmark analysis of PISA data across 31 countries, Students, Computers and Learning: Making the Connection. Its conclusion was blunt: despite considerable investment in computers, internet connections and educational software, there was little solid evidence that greater computer use led to better reading, maths or science scores. Countries that invested most heavily in classroom technology saw no appreciable improvement in their PISA results at all.

More specific analysis of the same data revealed what researchers now call the ICT paradox: an inverted-U relationship between technology use and achievement. Students who used computers moderately at school outperformed those who barely used them at all, but students who used computers very frequently performed worse than moderate users, in most cases even worse than students who used them rarely. Perhaps most sobering for anyone hoping technology might be a shortcut to educational equity, the OECD found that expanding access to computers and digital services did little to close the gap between advantaged and disadvantaged students. As the report's authors put it plainly, ensuring every child reaches a baseline level of literacy and numeracy does more to create equal opportunity in a digital world than expanding access to high-tech devices ever will on its own.

Andreas Schleicher, the OECD's director of education, offered the line that has stuck with this finding ever since: technology can amplify great teaching, but great technology cannot replace poor teaching.

So why does some technology work extremely well?

Set against this sobering national-level picture, a completely different body of research tells a strikingly positive story, but about a much more specific category of technology. Intelligent tutoring systems, software specifically designed to give step-by-step, adaptive feedback as a student works through a problem, have been the subject of extensive meta-analysis, and the results look nothing like the OECD's general computer-use findings.

A comprehensive 2011 meta-analysis by Kurt VanLehn, reviewing 54 comparisons across 28 evaluation studies, found intelligent tutoring systems raised test scores by an average of 0.58 standard deviations compared to no tutoring. Critically, VanLehn found this effect varied by design: systems offering only a final answer check produced a comparatively modest effect (0.31), while systems offering step-by-step feedback throughout a problem, correcting a specific misstep rather than just marking a final answer right or wrong, reached an effect size of 0.76, approaching the 0.79 effect size VanLehn found for one-to-one human tutoring in the same analysis. A well-designed piece of software, in other words, can close much of the gap to a human tutor, a benchmark long treated as close to the ceiling of what educational intervention can achieve.

Duration and implementation matter as much as design

A further meta-analysis, by Steenbergen-Hu and Cooper in 2013, examined intelligent tutoring systems used specifically for K-12 mathematics and found a pattern worth knowing before expecting quick results: these systems showed no significant effect on learning when used for only a short period, but a meaningfully positive effect when used consistently over a full school year or longer. Technology used briefly, as a novelty or a one-off intervention, behaves differently from the same technology embedded consistently into how a subject is actually taught and practised over time.

This connects directly to a finding closer to home. A 2025 systematic review commissioned by the Education Endowment Foundation found that the effect of EdTech on disadvantaged pupils specifically was smaller and less certain than the average effect across all pupils, and warned that poorly implemented EdTech risks widening the attainment gap rather than closing it. Read alongside the research above, this isn't a contradiction, it's the same underlying pattern: technology's impact depends heavily on design and sustained, well-supported implementation, and disadvantaged pupils are statistically more likely to encounter technology without either.

Where even good EdTech falls short

None of the positive evidence above is a case for unlimited confidence in technology, and it's worth being direct about the genuine limits. The intelligent tutoring system evidence is strongest in structured, well-defined domains, mathematics above all, where a step can be objectively right or wrong and feedback can be precise. It is considerably less established in open-ended domains like essay writing, historical analysis or creative work, where good practice is harder to specify in a way software can reliably assess.

Technology also cannot substitute for the relational and pastoral dimensions of teaching, noticing that a student seems anxious, adjusting tone in response to a bad day, the kind of judgement human teachers make constantly and largely invisibly. And no piece of software addresses the structural drivers of the wider attainment gap, attendance, funding, housing instability, that sit upstream of anything happening on a screen. Technology can be a genuinely powerful lever inside a well-functioning system. It is not a substitute for the system itself.

What this means in practice

Look for structured feedback, not just digital delivery. The research gap between a 0.31 and a 0.76 effect size sits almost entirely in whether a system gives step-by-step, in-the-moment feedback or just marks a final answer right or wrong. A digital worksheet is not the same category of tool as an adaptive tutoring system, even though both are technically ‘EdTech.’

Expect real effects to take a term, not a week. Given that meta-analyses found no significant benefit from short-term use but real gains over a full school year, a platform judged after two weeks of use is being judged on the wrong timescale entirely.

Don't expect technology alone to close a disadvantage gap. The evidence is consistent on this point from two independent directions: broad computer access doesn't close it, and EdTech's own effect is smaller and less certain for disadvantaged pupils specifically, unless implementation actively accounts for the access and support barriers this group is more likely to face.

This is precisely the category of technology BrainStrata is built to be: not general-purpose screen time, but structured, step-by-step adaptive practice with feedback at the point a student gets stuck, sustained over a full course rather than a novelty, and designed to sit alongside a teacher's judgement rather than in place of it. The OECD's own framing captures the ambition plainly: technology that amplifies great teaching, not a replacement for it.

Sources and further reading

[1] OECD (2015). Students, Computers and Learning: Making the Connection. Analysis of PISA data across 31 countries; the source of the ‘ICT paradox’ finding and the finding that technology access did not close the advantaged/disadvantaged achievement gap.

[2] VanLehn, K. (2011). The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. Educational Psychologist, 46(4), 197–221. Meta-analysis of 54 comparisons across 28 studies; effect sizes for step-based (0.76), sub-step-based (0.40) and answer-based (0.31) tutoring systems, and human tutoring (0.79).

[3] Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on K-12 students' mathematical learning. Journal of Educational Psychology, 105(4), 970–987. Findings on duration of use and effect size.

[4] Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014). Intelligent tutoring systems and learning outcomes: A meta-analysis. Journal of Educational Psychology, 106(4), 901–918.

[5] Education Endowment Foundation (2025). Understanding Quality Characteristics of EdTech Interventions and Implementation for Disadvantaged Pupils: Systematic Review with Meta-Analysis.