SSA_2023_South Africa_ Primary English Language Reading Test design and evaluation

Tender description

This section details the scope for this assignment including the range of activities the supplier is expected to deliver, the qualifications and expertise required of the supplier, and the expected timeline for the task.<br /><br />OBJECTIVES<br /><br />Phase 1 - Develop a theoretical approach to testing reading comprehension for a comprehensive English Language Reading Test for primary school pupils in development projects contexts.<br />Phase 2 – Using this theoretical approach, design a set of test specifications (overall test spec. identifying target reading construct, number of tasks and items, formats, etc), with detailed task/item specifications.<br />Phase 3 – Use the test/task/item specs. to design and validate specific materials that can be used to evaluate the impact of reading projects in South Africa.<br /> (NB. We require proposals to cover all activities required to deliver all three phases, but Phases 2 and 3 will only commence upon successful completion of Phase 1 outputs.)<br /><br />PROJECT DURATION Mar 2024 – Dec 2024<br /><br />PROPOSED TEAM<br /><br />This project can be carried out by a small team of academics and/or practitioners with a strong track record in assessment from an academic institution that will be selected through a competitive selective call for proposals. <br /> Suitable applicants will be able to demonstrate expertise and experience in following areas<br />- Teaching and learning of reading, research into reading comprehension for young learners, particularly for learners in the development contexts in which the test is likely to be used<br />- Clear understanding of the context for Phase 2 outputs (South Africa) as well as the wider context <br />- Demonstrated experience and expertise in language test development and research<br />If the applicant has suitable testing experience in other subjects, but language test experience is a gap, there are several UK language testing research centres who could potentially partner on this project. If this is of interest, please contact us before the clarification question deadline.<br />In addition to expertise in assessment, the team must also have a comprehensive understanding of the complexities, issues, and risks involved in conducting assessments in a low-resource development context where English is taught as an Additional Language, and a sound knowledge of the South African context and relevant features of the situation in relation to the education system and development partners working to support it.<br /><br />EXPECTED OUTPUTS<br /><br />Phase 1 (a report):<br />a. Literature review of current theory and practice of primary aged pupil exams relevant to low resource contexts such as those in SSA, including considerations such as cognitive load, multilingual approaches, and different aspects of validity. Suppliers should consider the Socio-Cognitive Model as described in Chalhoub-Deville and O’Sullivan (2020) as this is widely used by the British Council and other test developers. The Khalifa and Weir (2009) reading model should also be addressed in the review.<br />b. In light of this literature review, analyse and evaluate strengths and weaknesses of tests used in the SSA development sector (including EGRA and the BLF pupil test). <br />c. Adapt and/or design a theoretical approach for a comprehensive English Language Reading Test for primary school pupils in development projects contexts. This must be based on a clear validity model emerging from the literature review in (a) above.<br /><br /> Phase 2 (a test specification):<br />a. Based on the theoretical approach, adapt/design a set of test specifications (overall test spec. identifying target reading construct, number of tasks and items, formats, etc), as well separate detailed task/item specifications which provide the detail for item writers to develop specific content that is predictable in terms of difficulty and fits the overall approach and reading skills that different task formats may need to focus on.<br /><br />Phase 3 (a set of validated test materials):<br />a. Use this test specification to develop test materials and scoring rubrics for P1-7 pupils in South African state schools that align with the national curriculum and learning outcomes that are age-appropriate, culturally sensitive, and suitable for delivery at scale. (If feasible, include EGRA ORF questions in the test package to evaluate concurrent validity.) <br />b. Throughout the test development process, collect appropriate evidence to assess specifications, content, and items for validity, reliability, and effectiveness in evaluating pupils' reading skills in line with the national curriculum and stated learning outcomes. We would expect at least three levels of evaluation rather than one summative validation: small scale trialling with small groups of students, collecting detailed qualitative information on how they have approached the test; larger pilot(s) in which the test is tried with a larger number of students after test content is reasonably settled; and a final larger scale field trial with the final test design and content that allows for precise scoring validation.<br />c. In-depth analysis of trial results and refinement of the specification, content and items based on the trial outcomes throughout the development cycle.<br />d. Produce a final testing toolkit with the validated materials, scoring rubrics, and assessor training materials.<br /><br />