Given the curriculum or syllabus, the purposes of an assessment and the constructs to be assessed must be set before the assessment is developed. It is important for major stakeholders to reach consensus on the following aspects of the assessment development and validation processes:

• The curriculum: Who should be responsible for designing and developing the curriculum, and identifying or defining the learning outcomes?
• Assessment purposes and performance standards: What are they and who should define them?
• Assessment construct, design, criteria and methods: Who should be responsible for specifying them?
• Assessment development, administration, analysis, reporting and interpretation, and evaluation: The responsibility of assessment providers.
• Conceptualisation of validity and approaches to validation:
  • How should assessment validity be conceptualised?
  • What is the best approach to validation and what kinds of evidence should be included in the validation process?
  • Who should be responsible for carrying out the validation process?
  • Who should be responsible for evaluating the strength of the validity argument and drawing conclusions about validity?
• How should Ofqual regulate assessment validity?
• The roles and responsibilities of the major stakeholders: How should they be involved in the assessment development and validation processes?

Validation is the process of developing a sound argument based on theory and empirical evidence collected from all stages of the assessment process to support the proposed interpretation and intended use of the assessment results. Each proposed interpretation and intended use of results from an assessment will need to be validated because different interpretations or uses may involve different inferences and assumptions.

Interpretations and uses of regulated assessments

For National Curriculum assessments and general qualifications, final results for individuals are reported using performance categories and should therefore be interpreted in terms of the level of attainment in the curriculum as required by the performance standards which are pre-defined. This interpretation of results for individuals applies to the primary purposes (uses) of regulated assessments and will be retained in others uses.

Currently, assessments regulated by Ofqual are primarily used for:

• Certification/reporting (at individual level: NCAs, GCSEs, GCEs and VQs)
• Employment/entry to university (at individual level: GCSEs, GCEs and VQs)
• Accountability (at classes, schools, LAs levels: NCAs, and GCSEs)
• National performance monitoring (at national level: NCAs and GCSEs)

In the present study, focus of the validation of the selected assessments will be on the use of results at individual student level (reporting or certification / university entry / employment).

An argument-based validation framework

Argument-based approaches have been widely used for validating educational assessments. Given the purpose and intended use/s of the assessment and the proposed interpretation of the results, an argument-based approach to validation general involves:

• The development of a set of clear and coherent propositions that support the proposed interpretation and intended use of the results.
• The evaluation of the strength of the propositions using appropriate evidence collected from various stages of the assessment process (ie to develop a sound validity argument).

In view of the nature of regulated assessments and the regulatory role of Ofqual, an argument-based framework involving the following propositions has been suggested for validating Ofqual regulated assessments for the purpose of individual certification:

1. Alignment between the assessment and the curriculum: The assessment tasks and criteria are well aligned with the learning outcomes of the curriculum or syllabus.
2. Accuracy and reliability of scores: The assessment scores (results) are accurate and reliable.
3. Alignment between performance on the assessment and the performance standards defined for the curriculum or syllabus: The assessment performance category boundary marks are well aligned with the pre-defined performance standards.
4. Indicator of future performance: Assessment results are a good indicator of future performance (when the results are used for selection to enter university/employment or prediction of future attainment).

These propositions form a theory or interpretive argument that supports the proposed interpretation and intended use of the results and need to be evaluated using appropriate evidence collected from the assessment process (detailed guidance on how to evaluate these propositions are provided in a document that can be downloaded from the link provided at the end of this page).

Validity studies and validation case studies

A series of empirical studies will be carried out to:

• Investigate specific aspects of validity for a selection of regulated assessments.
• Validate a range of regulated assessments at the overall assessment level.

All assessments regulated by Ofqual are of high stakes. Investigations will be conducted to explore the impact of the use of regulated assessments:

• Impact on teaching
• Impact on learning
• Resource implications and practical difficulties
• Impact on the curriculum in general

Results from the validity studies, validation case studies, and impact investigations will be presented to the major stakeholders, including educational researchers, assessment experts, policy makers and practitioners, using seminars or workshops for further debate on:

• The strength of evidence for the specific aspects of validity of the assessments investigated.
• The strength of the validity argument for the assessments investigated in the validation case studies in terms of:
  • meeting the defined performance standards
  • meeting the requirements of the users
• Impact on learning, teaching and the curriculum or syllabus (and therefore on future assessment).
• Aspects of the assessment system that need improving.
• Communication of validity evidence to the public.

This will involve:

• Review of existing regulatory criteria and requirements.
• Debate on how Ofqual should regulate assessment validity.
• Evaluation of findings from this programme and other validity studies and practices used in other countries.
• The development of Ofqual validity auditing procedures.