Evaluating the school pilot program for economic education

The Austrian foundation for economic education (Stiftung für Wirtschaftsbildung) was established in late 2020. It aims to create a neutral, evidence-based platform for the sustainable enhancement of comprehensive economic education across Austria that caters to the real-life needs of different target audiences. The founding entities of the Foundation include the Chamber of Labour, DIE ERSTE österreichische Spar-Casse Privatstiftung (ERSTE Foundation), Federation of Austrian Industries, MEGA Bildungsstiftung, Oesterreichische Nationalbank (OeNB) and the Austrian Economic Chamber.

To foster economic and financial education, the Austrian foundation for economic education has launched a school pilot program. Through a comprehensive support package, this initiative supports approximately 60 secondary schools, divided into two cohorts, over four years (with cohort 1 starting in the school year of 2022/23 and cohort 2 in the school year of 2023/24) in implementing a focus on economic and financial education in their curriculum. The support package includes newly developed teaching and learning materials, professional development for teachers, networking opportunities connecting schools and businesses and financial support.

In collaboration with the Institute for Advanced Studies (IHS) and the University of Duisburg-Essen, the OeNB's Financial Literacy Unit is conducting a comprehensive scientific evaluation of the school pilot to generate evidence-based insights.
 

Evaluation design

The school pilot program aims to initiate changes among three distinct groups: school principals, teachers and students. School principals (group 1) can promote a stronger integration of economic education at their schools within the scope of school autonomy. Consequently, this influences teachers (group 2) in their ability to convey economic competencies to students (group 3).
The impacts on all three groups are being explored through a rigorous, holistic mixed-methods approach, employing a quasi-experimental design that combines process and impact evaluation. This involves a combination of qualitative methods (interviews and focus groups) and quantitative methods (competency tests and implementation surveys), conducted with both the intervention group and a randomized control group.
The process evaluation addresses the question HOW and WHY the school pilot works or does not work, while the impact evaluation examines the development of students’ competencies in economic education. The control group consists of comparison schools not actively participating in the pilot, which plays a crucial role in providing evidence about the pilot’s effectiveness.
The evaluation spans the entire four-year duration of the project. Most data are collected on an annual basis, which facilitates the identification of changes over time.
Throughout the entire research process, it is of paramount importance to us to adhere to principles of data protection and research ethics, including confidentiality, voluntary consent, participant anonymity and secure data storage. We consistently ensure that all information and data are treated with the highest level of confidentiality. Researchers are obligated to maintain confidentiality even after the project’s conclusion. All data are securely stored on internal severs and are used solely in an anonymized format for publications, research reports and presentations at scientific conferences to ensure that it is not possible to trace data back to any of the participants.