Consultant for Learning Site (impact) Evaluation Study

Palladium International

Consultant for Learning Site (impact) Evaluation Study

Posted

10 September 2024 10:36am

Apply by

30/09/2024

Regions

Latin America and the Caribbean

Middle East and Africa

Pacific

South and West Asia

South-East and East Asia

Sectors

Education

Summary

Education Pathways for Peace in Mindanao Program will conduct a three-year research study to determine the impact of the program’s interventions on targeted schools (kinder to grade 3 only) in the Bangsamoro Autonomous Region in Muslim Mindanao (BARMM). The research will be designed as a small n study testing the theory of change for each of four types of school: community schools (known as Learning Centres), catchment schools connected to the Learning Centres (known as Associated Catchment Schools), targeted madrasah schools, and the model inclusive schools (known as Inclusive and Supportive Centres of Learning).

This theory-based evaluation approach will be used to identify what worked by looking into the changes brought about by Pathways-supported intervention and, importantly, to understand why and how the interventions worked. The research will start in 2024 and be repeated in 2025 and 2026. The study will provide baseline data in 2024 and, where possible, estimate retrospective baselines as of September 2022 when schools reopened after the COVID-19 Pandemic. Data collection will estimate where Pathways is contributing to better learning outcomes, classroom teaching and learning practices, learning and development for teachers, support to schools by divisional offices, school-based management practices, and inclusion practices including equity outcomes for key variables.

The study aims to:

Identify to what extent Pathways’ interventions are reaching and creating change at the school and classroom level, and impacting children’s literacy and numeracy
Identify adaptations and modifications that are both positively and negatively influencing the outcomes
Understand results for different variables across the type of schools
How contextual factors are influencing the outcomes expected at the school-level
Identify the enablers, barriers, and challenges influencing school-level outcomes
Identify good practices and effective strategies within each school type
Provide learning and recommendations as to how policy and implementation can be improved during the remainder of the program, as MBHTE and the Philippines Department of Education take policy interventions forward, and to inform Australia’s ongoing education sector work.
The program’s overall target beneficiaries for each school type are 210 Learning Centres, 123 Associate Catchment Schools, 25 Madrasah, and one Inclusive and Supportive Centre for Learning (ISCL). Pathways support to each school type has included^[1]:

Learning Centres

Pathways' assistance to the MBHTE Abot Kaalaman sa Pamilyang Pilipino (AKAP) Program aims to offer quality early years' education to learners in barangays without schools. This involves establishing functional learning centres (LCs) or providing flexible learning options in validated school-less barangays. By the end of the 2023-2024 school year, AKAP provided kindergarten to grade 3 education to 123 geographically isolated, disadvantaged, and conflict-affected communities in BARMM by introducing grade 3 classes.

Pathways' support to AKAP focuses on enrolling out-of-school children, enhancing social inclusion and strengthening stakeholder engagement at various governance levels and with the school. This involves enhancing stakeholder engagement in learning centres, community structures, school division offices, local government units, and the regional AKAP Technical Working Group.

Associated Catchment Schools

An Associated Catchment School is a catchment school assigned to manage an AKAP Learning Centre established under the Pathways Program. Pathways has progressively lifted the Catchment Schools linked to AKAP Learning Centres to ensure they provide support and management to the Centres for them to succeed. The focus is for the Associated Catchment Schools to develop specific support and management skills. The program’s support to the Associated Catchment Schools includes providing trainings to K-3 teachers and Catchment School Heads, designing and planning K-3 teaching and learning materials, developing K-3 learning assessment tools and manual, and production of teaching learning readers in Filipino and mother-tongue languages, among others.

Madrasah

Pathways support to madrasah is at both the system level to help establish system arrangements that will improve both formal and non-formal madrasah and improve delivery by the targeted madrasah. At the system level, Pathways support has included the development of the Bangsamoro Education Code, Madrasah Education Strategic Plan, Madrasah Comprehensive Plan, and the Madrasah Establishment System. The 25 targeted madrasah is a group of `formal’ madrasah that were established before 2017 that are still operating and are considered `stable’ and development ready. All 25 madrasah are private entities and `formal madrasah’ are recognised under the Bangsamoro Education Code as delivering the governments’ curriculum and are subject to government quality assurance processes. Pathways’ support ranges from development of curriculum and learning materials, teacher and school leader training, and introducing school-base management. Madrasah are recognised as historically under resourced and therefore less developed institutions whereby responding to Pathways’ supported interventions will take longer.

Inclusive and Supportive Centres of Learning

The ISCL refers to a primary school that is being transformed into a model inclusion institution of learning with an ecology that adapts to the needs of children. The model school supports a network of other public schools, known as Satellite Public Schools, which are coached by the ISCL and MBHTE (regional and divisions) to progressively implement the Inclusive Schools Approach. Pathways started supporting the development of the model inclusive school in 2022. To date, Pathways has supported ISCL with the development of the Inclusive Schools Approach Omnibus Policy, ISA-ISCL Five Essentials, and training of teachers and school leaders in inclusion and Play-based Approach.

_________________________________

[1] Pathways will provide a table to the successful candidate identifying the scope, timing, and nature of interventions rolled out to-date.

Description

About the position

Pathways requires a consultant to develop the Learning site impact Evaluation Study (LS(i)S) using a small-N study design and to conduct quality assurance of data analysis and reporting for each of the three implementation years. Pathways will contract a research consulting firm to implement the study, including associated fieldwork,

The key responsibilities of the consultant include, but are not limited to:

Develop the research design and methodology for the LS(i)S. The body of the design should not be more than 30 pages and include but not be limited to:
1. Purpose and key research questions
2. Identification of the theory of change for school type
3. Identification of the methods and variables that will provide a valid assessment of each component
4. Identification and development of the tools for each assessment [2]
5. Identification of the case selection criteria and the in-school sampling framework for students and teachers.
Develop the tools that will cover but not be limited to assessing learning outcomes including for disadvantaged learners (excludes ISCL), teacher practice, teacher capacity development, inclusion practices, school improvement processes. Each tool should be contextualised for each learning site.
Outline (include rationale) of key fieldwork processes and procedures required to support the fidelity of implementation.
Determine the key approaches to data management and cleaning.
Develop the processes and approaches required to analyse the results, prepare summary findings, and to present the data (templated tables to be annexed).
Identify the expected and potential limitation and challenges for the study.
Develop a high-level implementation schedule that a lines with the school year, 2025 national elections, and limitations associated with Ramadan.
Develop (annexed to the design) the quality assurance plan for the study.
Prepare a brief (2-4 pages) outlining how the consultant will quality assure the analysis and reporting each year.
Prepare a 4-5 knowledge dissemination plan covering the three-years (separate to the design document).
Provide a briefing presentation (document only) on the research design suitable for Pathways’ key stakeholders including country counterparts (separate to the design document).
Provide a one-hour verbal briefing to the successful research consulting firm who is contracted to implement research (separate to the design document).
Provide a peer-review and comments on the fieldwork plan and procedures to be used by the research consulting firm (separate to the design document).
Provide a documented peer-review and recommendations on the i) validity of the data analysis, and ii) annual summary report of the study’s findings.

Timeline and Deliverables

Activities	Deliverable	Level of Effort¹
Development of the learning site evaluation using Small-N study design	1. Pathways Program Learning Site (impact) Evaluation Design (document) 2. Pathways Program Learning Site (impact) Evaluation Knowledge Dissemination Plan (document) 3. Pathways Program Learning Site (impact) Evaluation Design Briefing (presentation)	29 days
Briefing, and Evaluation of Fieldwork Plan and Procedure	4. Pathways Program Learning Site (impact) Evaluation Design Implementation Briefing (1 hour presentation for implementing researchers) 5. Peer-review of the Pathways Program Learning Site (impact) Evaluation Fieldwork Plan and Procedures (document)	1 day
Quality Assurance of Data Analysis and submission of Summary of Findings Report (2024)	6. Review and Recommendations Regarding the Data Validity and Findings of the Pathways Program Learning Site (impact) Evaluation 2024 Summary of Findings Report (document)	5 days
Quality Assurance of Data Analysis and submission of Summary of Findings Report (2025)	7. Review and Recommendations Regarding the Data Validity and Findings of the Pathways Program Learning Site (impact) Evaluation 2025 Summary of Findings Report (document)	5 days
Quality Assurance of Data Analysis and submission of Final Report of the Learning Site Evaluation Study (2026)	8. Review and Recommendations Regarding the Data Validity and Findings of the Pathways Program Learning Site (impact) Evaluation 2025 Summary of Findings Report (document)	5 days
Total		45 days

_________________________________

[2] Development of the draft tools will be collaborative between the consultant and Pathways, with Pathways providing draft (and tested) tools for assessing literacy and socio-emotional learning and teaching practices. Pathways will also provide 3^rd party expert technical peer-review and input into the design of these tools. As such, the successful consultant is expected to understand education delivery but not necessarily be an expert in learning assessment design.