Approximately 15% of New Zealand children live in households without adequate and safe food. This level of food-insecure children is concerning as food insecurity and hunger have been linked to lower levels of concentration, cognitive functioning, engagement and school achievement, as well as worse developmental outcomes later in life.
In 2020, a government-sponsored healthy school lunch programme – Ka Ora, Ka Ako – was introduced by the New Zealand Ministry of Education. The programme provides lunches to all learners, free of cost and on a daily basis, within schools with the highest concentration of learners that face socio-economic barriers. The programme was expected to reach around 222,000 learners daily, in 967 schools and kura across New Zealand.
Standard of Proof was commissioned to evaluate the impact of the programme. The key requirement for this evaluation was to help the Government understand the effects of the programme – those effects above and beyond what would have been achieved otherwise (without the programme). Precise estimates of the effect were essential given that the resultant decisions would affect, most directly, a significant proportion of learners in New Zealand.
The Evaluation
We applied the principles and techniques from the Real World Evaluation approach to support the objectives, strengthening the evaluation design and validity of conclusions while working within the given context. Notably, the evaluation was commissioned after the programme was fully implemented, and therefore capturing pre-programme data was no longer possible. It was also implemented, at this point, in nearly every school that was eligible for the programme.
The evaluation team adopted several evaluation techniques to address these challenging gaps in evidence. First, the team made use of the learner’s home food environment to understand what life looked like without the programme (their ‘contextual anchor’). The team also engaged learners in schools just above and just below the Ministry’s criterion for the programme eligibility, making use of a regression discontinuity design. Both approaches worked exceptionally well, and provided robust evidence for this national programme.
The evaluation report
The evaluation and technical reports can be downloaded here:
New Zealand healthy school lunches programme: impact evaluation (apo.org.au)
