Ofqual

Experts discuss research into assessment reliability

Tuesday, 22 March 2011 12:53

Ofqual marked the end of its two-year study of assessment reliability with a seminar event today (Tuesday, March 22) to discuss the research and the next steps.

The Reliability Programme was set up to gather evidence on a range of issues around assessment reliability to help awarding organisations develop better methods of testing and assessment.

Dennis Opposs, Ofqual’s Director of Standards, said: “This is a complex technical area, and until we started the Reliability Programme, was something of a taboo subject.

“Over the past two years we have looked at different aspects of reliability, including how to generate evidence, how to interpret and communicate the findings and what people understand and think about these issues.

“One of the findings of note concerns Key Stage 2 National Curriculum Tests. Research from 2001 suggested 30 per cent of students were awarded a level higher or lower than they should have been due to unreliability. Our research on more recent tests indicates that reliability is much higher. The research indicates classification accuracy figures ranging from about 85 per cent for English to about 90 per cent for mathematics.

“The Reliability Programme has made substantial progress in our understanding of this subject and we will consider the range of findings as we develop policies on reliability. We also know there is still much more work to be done in this area.”

Ofqual has identified three priorities for its future work on reliability:

To continue to research reliability as part of supporting the quality assurance process for qualifications
To encourage awarding organisations to generate and publish reliability data
To continue to improve professional and public understanding of reliability issues

The event today, titled ‘The Reliability Programme: Leading the way to better tests and assessments’, was attended by about 100 people from the assessment and education communities.

Ofqual has also today published a final summary report of the Reliability Programme research and a number of research papers completed under the programme. These can now be found on the Ofqual website.

Ends

Notes to editors

1. For more information, the media can contact the Ofqual press office on 0300 303 3342 or 07798 924552 (out-of-hours)

2. Reliability, in education assessment terms, can also be thought of as consistency. A high level of reliability would mean that broadly the same outcomes would arise were an assessment to take place again – would the candidate get exactly the same result if they took the examination on a different day, had different questions on the paper, or if it was marked by a different person.

3. The reports published today are: