What Counts as Evidence in EdTech?

Over the past weeks, we’ve received a large number of applications to the Learning Cabinet, and the call is still open for EdTech providers globally.

One topic comes up repeatedly in conversations with applicants, especially the ones whose application has got rejected: What do we actually mean by “evidence”? 

It’s a fair question, and the answer is not always straightforward. 

There is no single evidence standard that fits every type of EdTech solution. The type of evidence we consider relevant depends largely on the nature of the solution and the type of learning outcomes it aims to influence. 

 
Matching Evidence to the Type of EdTech Solution 

Some EdTech tools have clearly defined, curriculum-aligned learning goals and include structured learning content. For these types of solutions – for example in mathematics, literacy, or language learning – it is often feasible to measure direct learning outcomes through pre- and post-tests or other quantitative methods. 

However, many EdTech solutions do not operate in this way. Some tools focus on supporting teachers and improving classroom pedagogy, rather than delivering specific content directly to learners. These tools may help teachers plan lessons, structure classroom activities, collaborate with colleagues, or streamline assessment practices. 

For such solutions, it is often much more difficult to isolate and measure their direct effect on students’ final learning outcomes. Teaching and learning are influenced by many factors, and the tool may play an indirect but still meaningful role in improving the learning environment. 

In these cases, it can be more appropriate to gather evidence on other levels of impact instead. This might include measurable improvements in teachers’ workload, efficiency in assessment, adoption of new pedagogical practices, or increased collaboration among educators. Qualitative feedback from teachers and classroom observations can also provide valuable insights into how the tool supports teaching practice. 

In other words, the type of evidence should match the type of impact the solution is designed to create.  

Why We Ask for Evidence? 

When we match EdTech solutions with governments and education systems, we need to be confident that the solution actually creates impact, and that the impact has a reasonable chance of being replicated in different countries and contexts. 

Evidence is what gives us that confidence by providing indicators and a history of the scope, design and outcomes or impact of a tool. And when we present solutions to countries, it is also the key element that helps them trust the tool. 

Popularity or fast growth in the number of users can certainly be a positive signal. It may indicate that a solution is easy to adopt and responds to a real need, but user growth alone does not necessarily indicate meaningful learning improvements.  

In large education systems, governments need to understand not only whether a product is popular, but also what outcomes it produces in classrooms and whether those outcomes can be replicated at scale. 

What We Mean by Impact Evidence 

Impact evidence refers to evidence that helps demonstrate whether and how a product contributes to meaningful educational outcomes. Depending on the product, these outcomes may include improvements in learning, teaching practices, educator effectiveness, learner engagement, inclusion, or other intended educational objectives. At the same time, educational outcomes are always influenced by many factors: teachers, classroom context, curriculum, and others. Technology alone rarely determines the results. 

This is why it is important to understand the role a digital solution plays in the learning environment and in the learner’s experience. As the field of EdTech evaluation continues to evolve, we are learning from research, practitioners, governments, and solution providers. These insights are helping shape the upcoming EdTech for Good Framework V2.0, which aims to further strengthen how evidence of learning impact is assessed. 

The Learning Cabinet distinguishes between adoption metrics and impact evidence. Metrics such as the number of users, schools reached, downloads, subscriptions, or revenue growth may demonstrate scale and adoption, but on their own they do not constitute evidence of educational impact. 

Impact evidence should be based on the systematic collection and analysis of relevant data using transparent and credible methods. The purpose of the evidence levels is to assess the strength of the evidence supporting a product's intended educational outcomes, rather than the size of its user base or commercial success. 

Learning Cabinet – Impact Evidence Levels 

In order for a solution to be listed in the Learning Cabinet, it must achieve at least Level 3 in impact evidence. While Level 3 serves as the minimum eligibility requirement for listing, providers are encouraged to develop evidence across both lower and higher levels where relevant. Evidence gathered across multiple levels strengthens confidence in the solution and can provide valuable complementary perspectives for decision-makers and solution buyers. 

Level 1: Logic Model / Theory of Change (ToC) 

A clear and credible explanation of how the product is expected to contribute to its intended educational outcomes, informed by relevant research, theory, or established practice. 

Level 2: Independent Validation 

Independent expert reviews, quality certifications, procurement evaluations, or other third-party assessments that provide external validation of the product's quality, design, or suitability. 

Level 3: Credible Empirical Evidence of Outcomes 

Empirical evidence demonstrating outcomes associated with the use of the product. This may include case studies, pilot studies, or other outcome-focused research. Studies may be conducted by the provider or an independent party, provided the methodology and reporting are transparent and credible. 

Credibility is demonstrated through clear documentation of the study design, methodology, implementation process, data collection and analysis methods, results, and limitations, allowing others to understand, assess, and interpret the findings. 

Level 4: Strong Empirical Evidence of Outcomes 

Large-scale empirical studies demonstrating positive outcomes across substantial numbers of users, schools, or learning environments. Studies should use transparent and credible methods and may include benchmarking against relevant standards, norms, or population averages. 

Level 5: Rigorous Evidence of Impact 

Rigorous research demonstrating that the product has contributed to the observed outcomes. Depending on the nature of the product and its intended use, this may include randomized controlled trials (RCTs), quasi-experimental studies, longitudinal studies, or other robust research designs capable of providing strong evidence of impact. 

What does this mean for EdTech providers? 

Every scale of evidence counts. What we want to see is a credible and transparent evidence story. 

• Does the solution clearly explain why it should work? 
• Is it aligned with existing research? 
• Has impact been observed in real-world use? 
• Is the data collection transparent and methodologically sound? 

If you do not yet have strong empirical evidence, that is not the end of the road. It simply means the next step is to start building your evidence portfolio: 

• Clarify your theory of change. 
• Align your solution with existing research. 
• Collect structured data from real users. 
• Document findings transparently. 
• Strengthen your evidence gradually over time. 

Evidence building is a journey, and we are happy to see solutions at different stages of that journey, particularly those at minimum Level 3 as described above. The Learning Cabinet application call is still open, so if you are developing an EdTech solution with real potential for impact, there is still time to apply.