Our Assessment Innovation Fund pilots: Bolton College

Shaping the future of learning and assessment

We’re on a mission to break boundaries in assessment with an investment fund to support and pilot new ideas on the future of assessment.

Bolton College

Bolton College is one of the North West's largest providers of vocational training and further education. The college enables individuals, businesses and communities within Bolton and surrounding areas to reach their potential. They provide courses for a wide range of people of all different ages and are proud to be able to offer hundreds of courses to suit everyone.

About the pilot

The Assessment Innovation Fund (AIF) has provided two grants to Bolton College to develop and test the FirstPass platform. 

The initial 15-month pilot tested the efficacy and impact of their FirstPass platform. FirstPass is an online platform that has been developed by Bolton College to support learners and teachers with the formative assessment of open-ended questions. Using FirstPass, learners responded to open-ended questions using free-form text. FirstPass offers real-time feedback to learners as they compose their answers; allowing them to reflect and improve upon their answers before submitting them to their teachers for final review and commentary. Teachers across six institutions used FirstPass to set and mark formative assessments and check understanding and knowledge retention. 

The second pilot picked up on learnings from the first and had three objectives. Firstly, to further develop FirstPass capabilities based on user feedback. Secondly the pilot wanted to measure the resource cost of training FirstPass to give feedback on two of NCFE’s flagship qualifications in the L2 Diploma in Early Years Practitioner (apprenticeship cohort) and the L1/2 Technical Award in Child Development & Care (school cohort). Thirdly the pilot measured the impact on learning of the new capabilities and training with teachers and learners at NCFE providers. 

Prior to winning these two AIF grants, FirstPass was granted funds by Ufi VocTech Trust which has been supporting the development of the platform with grant-funding and enrichment since 2019. 

Key findings

Key findings from Pilot 1 

• 82% of teachers stated that FirstPass will be an effective AI service for supporting them as they undertake formative assessment. 

• 92% of learners stated that FirstPass correctly detected what they entered on the screen as they composed their responses to open-ended questions. 

• 92% of learners said that the feedback from FirstPass helped them to compose better answers. All learners welcomed the immediacy of the feedback from the platform. 

The FirstPass pilot study offers a compelling case for the adoption of AI-powered formative assessment tools. The platform's real-time feedback and crowdsourcing model show promise in addressing longstanding challenges in open-ended question assessment. While the study is not without limitations, its findings contribute to the ongoing conversation about the role of technology in education and the potential for AI to support teachers and learners. Overall, the FirstPass pilot study represents a significant step toward more efficient and effective formative assessment practices, with implications for educators and students alike. It paves the way for further exploration and development of AI-driven solutions in the field of education. 

Key findings from Pilot 2 

• Technical developments including additional classifier functions and single sign-on for Microsoft and Google were successfully added to FirstPass. 

• Training FirstPass using subject experts was successful but resource intensive. 

• When deploying FirstPass with the school cohort, students receiving First pass improved significantly more than those who did not, even after accounting for differences between the groups at baseline. 

• A large, positive effect was observed in the school which reported the highest levels of usage of FirstPass. 

• The apprenticeship cohort used FirstPass less and did not improve their knowledge compared to their peers in the control group. 

FirstPass shows promise as an effective feedback system, however the process evaluation highlights a few areas of consideration when implementing the platform around task variability, the user interface, and the type of learner who would most benefit from using it with those in schools gaining benefit whilst apprentices did not.