Reliability Programme: Technical Advisory Group

Friday, 03 September 2010 11:48

Members of the technical advisory group, which will help guide the Reliability Programme, have now been appointed. The group will advise on the methodology for generating evidence of the reliability of results and interpreting the evidence.

Dr Jo-Anne Baird

Dr Jo-Anne Baird is a Reader in Educational Assessment and the Director of the Doctorate in Education Programme at the University of Bristol. Before this, she was Head of Research at the Assessment and Qualifications Alliance (AQA), where she managed the research programme and for the standard-setting process for AQA’s examinations. Dr Baird has also worked as a Lecturer at London University’s Institute of Education and has taught for the Open University and at A level. She is currently a member of the Department for Children, Schools and Families' 14-19 Expert Advisory Group, the Independent Reviewer for the National Assessment Agency’s Key Stage test standard setting and is an Executive Editor for the journal Assessment in Education. In 2008 Dr Baird became a Fellow of the Association of Educational Assessment-Europe. In 2007, Dr Baird co-edited the book Techniques for comparing examination standards, which was commissioned by the Qualifications and Curriculum Authority.

She said: "Life is not ruled by exam results but they have a big hand in opening – or closing – doors to people. This project is about how exam results can be monitored to make sure students get the right results. Exams need to be reliable to be fair. I am interested in working with Ofqual on the best ways to monitor the reliability of exams."

Professor Paul Black

Professor Paul Black is currently Emeritus Professor of Education at King's College London. He took his first degree in physics at the University of Manchester, followed by a PhD at Cambridge University. He spent 20 years a faculty member of the Department of Physics at the University of Birmingham and in 1976 he became Professor of Science Education and Director of the Centre for Science and Mathematics Education at Chelsea College in London. When Chelsea College merged with King's in 1985, he became the head of the King's Centre for Education Studies, King's College London.

Professor Black was also Chair of the government's task group on assessment and testing in 1987-88 and Deputy Chairman of the National Curriculum Council from 1989 to 1991. He is an honorary life member and former president of the Association for Science Education (UK), and in 2004 he received the lifetime service award for Distinguished Contribution to Research in Science Education (US).

He said: "Assessment results have a strong influence on young people's career opportunities and on their own career decisions. It is a public duty to do as much as we can, both to make them as trustworthy as possible, but also to ensure that all those who use them understand the inescapable limitations to their reliability."

Alastair Pollitt

Alastair Pollitt set up the Cambridge Exam Research consultancy, which offers research and training on formal assessment, last year. Before this, he was made the Director of Research and Evaluation at the University of Cambridge Local Examinations Syndicate (UCLES) in 1994, where he stayed until 2004. He began his career at the University of Aberdeen, where he graduated in chemistry with mathematics. He spent two years teaching science in the West Indies which sparked his interested in learning, and spurred him to take an MA in Education and Psychology at the University of Aberdeen. He then trained in psychometrics and statistics in Edinburgh.

From 1980 to 1983 Mr Pollitt embarked on a programme of government-sponsored research to investigate the psychology of examining, which resulted in his 1985 book What Makes Exam Questions Difficult? He was director of the national project to monitor standards in English language amongst Scotland's schoolchildren in 1989.

Mr Pollitt is currently working on a project with the University of London to explore awarding grades using holistic comparative judgement.

He said: "I am very interested in unreliability, since everything that makes results less reliable is a threat to validity. Just as it's hard to recognise justice, but easy to recognise injustice, it's more profitable to look for sources of unreliability and how to measure them."

Professor Gordon Stanley

Professor Gordon Stanley is the Director of the Oxford University Centre for Educational Assessment, as well as a research fellow of St Anne's College and Pearson Professor of Educational  Assessment at the University of Oxford.

Prior to taking up this position at Oxford, he spent 10 years heading a curriculum and public examinations authority in New South Wales, Australia. His career has involved teaching and research in assessment as well as holding statutory offices in education. In 1997 her was made Chair of the Australian National Board of Employment Education and Training and of the Higher Education Council. During the 1990s he became involved in quality assurance issues in education and was a member of the Committee for Quality Assurance in Higher Education. In his NBEET role he provided advice on quality issues to the Commonwealth Minister for education. Professor Stanley is an Emeritus Professor of Psychology at the University of Melbourne and Honorary Professor of Education at the University of Sydney.

Professor Stanley said his career had involved measuring what many people would call the immeasurable. "Reliability of public examinations and testing regimes has been of particular interest from both a technical and public interest point of view."

Dr Anton Béguin

Dr Anton Béguin is the director of the Measurement and Research Department at Cito, an international measurement and assessment organisation, where he has worked since 2001. He has been responsible for statistical and psychometric procedures used in the central examinations in the Netherlands. Dr Béguin is one of the project directors of a large scale longitudinal study in the Netherlands and has been involved in many research projects related to standard setting and equating. He has worked as consultant advising on measurement, testing and accountability with a range of national and international organisations.

He began his career as a research assistant at University of Twente in 1995, where he also wrote his doctoral thesis. Dr Béguin has written many publications on examinations and measurement and is the Chair of the Methodology and Evaluation division of the Netherlands Educational Research Association (NERA), and Fellow of the Association of Educational Assessment-Europe.

He said: "Reliability is one of the essential quality indicators of tests and assessments. The interpretation of results of any measurement should depend on the accuracy of the assessment."

Last Updated on Friday, 03 September 2010 11:57 Tags: reliability