Annex I: Consistency in examination conventions

1. Introduction

Across the disciplines within each division in the University, there are many similarities in the way in which we teach and assess undergraduate students and teach and assess graduate students. Some common features in examining, marking and classification would therefore be expected within divisions. The QAA expects that ‘Higher Education providers operate equitable, valid and reliable processes of assessment’.1  This may involve a degree of variation from discipline to discipline.

Whilst academics in subject disciplines are best placed to determine the criteria used in marking and classification, Education Committee also has a duty to ensure that the processes used to apply these criteria are fair, explicit, and transparent.2  Where the criteria used in marking and classification differ from the norm and from cognate disciplines, there should be a rationale for the divergence.

 

1  From the Expectation of the QAA UK Quality Code for Higher Education, Chapter B6: Assessment of students and the recognition of prior learning.

2  From Indicator 2 of the QAA UK Quality Code for Higher Education, Chapter B6: Assessment of students and the recognition of prior learning.

2. Purpose of examination conventions

Examination conventions are the University’s formal record of the specific assessment standards for the course or courses to which students apply. They are a student-facing document and should be written in a clear and comprehensible manner. The same version of the examination conventions should be used by examiners, with more detailed local operational guidance appended if necessary.

Education Committee’s Policy and guidance on course information states that there are three key sources of information for on-course students about their course of study. These are the Examination Regulations, the relevant course handbook and the relevant examination conventions. Information about the structure of the course and the way it is assessed should be contained in those three documents. Key information on those matters on which students are entitled to rely should not be solely located elsewhere (for example, in a ‘Notice to candidates’ focusing on administrative arrangements).

3. Publication

Examination conventions must be circulated to all students and also published, either as part of the course handbook or separately, in a place easily accessible to students. Ideally, examination conventions should be publicly available so that prospective students may have access to them. If this is not possible, they should be accessible via Single Sign On (SSO) to anyone in the University so that the Proctors and colleges have access to them.

4. Content

The template below provides the headings of the information that should be supplied in examination conventions with a description of what is expected. In square brackets are references to further information in the Policy and guidance for examiners and others involved in University examinations (P&G for examiners) and/or the Examination Regulations where available or relevant. Please ensure that information is provided in clear and comprehensible language.

Suggested or sample text is provided in [square brackets].

Template for Examination Conventions

1. Introduction

Include:

• The full title of the course(s) to which the conventions apply;

• The year to which the conventions apply;

• Details of the supervisory body (divisional or faculty board) responsible for approving the conventions;

• The purpose of the examination conventions. You may wish to include the text below:

[Examination conventions are the formal record of the specific assessment standards for the course or courses to which they apply. They set out how examined work will be marked and how the resulting marks will be used to arrive at a final result and classification of an award.]

2. Rubrics for individual papers

Information on the structure of individual examination papers, for example, requirements relating to questions, number of questions, compulsory questions etc. Also include any paper specific regulations on, for example, the use of calculators, permitted reference material etc.

3. Marking conventions

3.1 University scale for standardised expression of agreed final marks
[P&G for examiners, 11.2]

Include one of the following as appropriate:

For FHS or Honour Mods

 Agreed final marks for individual papers will be expressed using the following scale:

70-100

First Class

60-69

Upper second

50-59

Lower second

40-49

Third

30-39

Pass

0-29

Fail

 

For Moderations or Preliminary Examinations

Agreed final marks for individual papers will be expressed using the following scale:

70-100

Distinction

40-69

Pass

0-39

Fail

 

PGT Model 1

Agreed final marks for individual papers will be expressed using the following scale:

70-100

Distinction

50-69

Pass

0-49

Fail

 

PGT Model 2

Agreed final marks for individual papers will be expressed using the following scale:

70-100

Distinction

60-69

Pass

0-59

Fail

 

3.2 Qualitative criteria for different types of assessment

Qualitative criteria should be given for different bands of marks for different types of assessment item. For example, a set of qualitative criteria might be given for answers to unseen timed (exam) essay questions, another for translation questions, another for extended essays, and another for extended research projects. Each would give a qualitative description of the type of answer expected to obtain a mark in the range ≤39, 40-49, 50-59, 60-69 etc. (mark ranges should be chosen as judged most appropriate, but should include bands of marks within a first).

3.3 Verification and reconciliation of marks

[P&G for examiners, 11.3]

For FPE
There should be a clear statement on how each script/item is marked and the moderation process which is to be followed.

For FHS, Honour Moderations, and PGT courses
For papers without a model solution, there should be a statement that each script/item of work is marked independently by two examiners or assessors (sometimes referred to as ‘double-blind marking’). There should be a clear statement on reconciliation procedures demonstrating that any relevant University and divisional guidance is being followed. This statement should encompass an explanation of instances in which the marks from two examiners will be averaged, rather than reconciled using alternative means. If an alternative method of marking has been approved by Education Committee, details of this should be provided.

For papers for which there is a model solution and marking scheme approved by the examiners, there should be a statement that each script is marked by an examiner or assessor and is checked independently to ensure that all parts have been marked and the marks and part-marks have been correctly totalled and recorded.

For papers which are made up of a number of elements, give an explanation of how marks are awarded for the individual elements of assessment and how these marks are translated into paper level marks on the scale set out above (see section 3.1). Information should be provided about the decimal precision of the calculations and the conventions used for rounding marks.

3.4 Scaling

[P&G for examiners, 11.4]

Where scaling is used a clear description should be given of the circumstances in which it will be used and the methodology which will be used (detailed algorithms should be included as an appendix rather than in the main part of the examination conventions, and further detail should be given in examiners’ reports). It should be made clear that scaling is not a mechanistic process, but one in which the examiners will use their academic judgement to ensure that appropriate classifications are awarded.

 The following text is provided as an example:

[The Examiners may choose to scale marks where in their academic judgement:

a) a paper was more difficult or easy than in previous years, and/or

b) an optional paper was more or less difficult than other optional papers taken by students in a particular year, and/or

c) a paper has generated a spread of marks which are not a fair reflection of student performance on the University’s standard scale for the expression of agreed final marks, i.e. the marks do not reflect the qualitative marks descriptors.

Such scaling is used to ensure that candidates’ marks are not advantaged or disadvantaged by any of these situations. In each case, examiners will establish if they have sufficient evidence for scaling. Scaling will only be considered and undertaken after moderation of a paper has been completed, and a complete run of marks for all papers is available.

If it is decided that it is appropriate to use scaling, the examiners will review a sample of papers either side of the classification borderlines to ensure that the outcome of scaling is consistent with academic views of what constitutes an appropriate performance within each class.

Detailed information about why scaling was necessary and how it was applied will be included in the Examiners’ report and the algorithms used will be published for the information of all examiners and students.]

3.5 Short-weight convention and departure from rubric

There should be a statement on the short-weight convention that will be applied. If there are alternative arrangements (for ‘compensation’) these should be described.

The following texts are provided as examples:

[A mark of zero shall be awarded for any part or parts of questions that have not been answered by a candidate, but which should have been answered.

OR

The maximum deduction that can be made for short weight should be equivalent to the proportion of the answer that is missing.]

This section could also describe the treatment of instances where a candidate fails to comply with the paper rubric (for example by not answering a compulsory question).

The following text is provided as an example:

[Where a candidate has failed to answer a compulsory question, or failed to answer the required number of questions in different sections, the complete script will be marked and the issue flagged. The board of examiners will consider all such cases so that consistent penalties are applied.]

3.6 Penalties for late or non-submission
[P&G for examiners, 9.4; Examination Regulations, Regulations for the Conduct of University Examinations, Part 14]

There should be a clear statement of penalties for late or non-submission of items, or non-completion of practical work. It should be made clear that non-submission of a required element will result in failure of the whole Examination (FPE, FHS, MSc, MSt, etc.), or in the case of an FHS assessed in Parts, the whole Part of the Examination.

The following text is provided as an example:

[The scale of penalties agreed by the board of examiners in relation to late submission of assessed items is set out below. Details of the circumstances in which such penalties might apply can be found in the Examination Regulations (Regulations for the Conduct of University Examinations, Part 14.)

Lateness

Cumulative mark penalty

After the deadline but submitted on the same day

[insert mark deduction]

[insert time period]

[insert mark deduction]

[insert time period]

[insert mark deduction]

[insert time period]

[insert mark deduction]

 

Failure to submit a required element of assessment will result in the failure of the whole First Public Examination/Second Public Examination/Part.]

3.7 Penalties for over-length work and departure from approved titles or subject-matter

[Examination Regulations, Regulations for the Conduct of University Examinations, Part 16, Regulation 16.6]

 There should be a clear statement of the penalties for over-length work and departure from approved titles or subject-matter if these are in place.

The following texts are provided as examples in relation to over-length work:

[Where a candidate submits a dissertation (or other piece of written coursework) which exceeds the word limit prescribed by the relevant regulation, the examiners, if they agree to proceed with the examination of the work, may reduce the mark by up to one class (i.e. from a 1st to a 2:1, or its equivalent).

OR

The Board has agreed the following tariff of marks to be deducted for over-length work:

Percentage by which the maximum word count is exceeded

Cumulative mark penalty (up to a maximum of [insert mark deduction])

Up to [insert value] %

[insert mark deduction]

Over [insert value] % and up to [insert value] %

[insert mark deduction]

Over [insert value] % and up to [insert value] %

[insert mark deduction]

For each further [insert value] %

[insert mark deduction]

 

]

3.8 Penalties for poor academic practice

[P&G for examiners, Annex G]

Assessors should mark work on its academic merit with the board responsible for deducting marks for derivative or poor referencing. There should be a clear statement of the penalties for poor academic practice.

The following text is provided as an example:

[The scale of penalties agreed by the board of examiners in relation to poor academic practice is set out below.

Band into which each case falls

Mark penalty (must be between 1% and 10% of the marks available)

Band A: [insert example case]

[insert mark deduction]

Band B: [insert example case]

[insert mark deduction]

Band C: [insert example case]

[insert mark deduction]

 

 

 

 

 

 

 

]

Where the deduction of marks results in failure of the assessment and of the programme the case must be referred to the Proctors.

4. Progression rules and classification conventions

4.1 Qualitative descriptors of classes (FHS) / Qualitative descriptors of Distinction, Pass, Fail (FPE/PGT)

Qualitative descriptors should be given for classes for FHS examinations and for Distinction, Pass and Fail for FPE and postgraduate taught courses. Qualitative descriptors for bands of marks may be given as an alternative.

4.2 Classification rules (FHS) / Final outcome rules (FPE/PGT)

There should be a clear explanation of the classification rules/rules for obtaining the final outcome. This should include the weight accorded to each element of assessment and how the marks aggregate to produce the classification or final outcome. For example, papers may have equal weight and an average taken, papers may be weighted and an average taken, and/or there may be preponderance rules. There may also be rules that specify that no paper may be below a certain threshold.

In the light of the rules followed, a statement about the way in which the board of examiners undertakes consideration of borderline outcomes might also be included.

When provided for in the relevant Examination Regulations (i.e. MSt, MPhil and MSc) you should include a statement on the restrictions on the award of distinction for candidates who have resat an element of assessment. You may wish to include the text below:

[Candidates who have initially failed any element of the examination will not be eligible for the award of a Distinction.]

4.3 Progression rules

[to be taken from the special Examination Regulations for the course]

The subject-specific Examination Regulations should state the rules for progression, for example, from one ‘Part’ to another within the FHS or from year one to year two of a two year Master’s course. This information should also be provided or referenced in the examination conventions and may include more detailed information on the rules for progression. It should also be clear what happens if the student fails to meet the progression requirement.

4.4 Use of vivas

[P&G for examiners, 11.9]

There should be a statement on the use and purpose of vivas where these are permitted by regulation. This should indicate whether vivas are to be used for all candidates, for candidates with outcomes on the borderline between particular classifications, or for failing candidates. Such vivas should be distinguished from any requirement for an oral element of a standard examination which is marked or part of a marked component. 

5. Resits

[P&G for examiners, 13.3 and 13.4]

The Examination Regulations state the circumstances when a resit is permitted either in the general regulations or the subject-specific regulations. In the examination conventions there should be a clear explanation of the circumstances in which students are entitled to resit an element of assessment and when resits would take place, with cross-references to the relevant Examination Regulations. Where resit marks will be capped, this should be clearly stated.

For PGT courses where an element, or elements, of an examination have been failed at the first attempt, students are entitled to one further attempt unless otherwise specified by the special regulations for a course. Marks for any element that has been successfully completed at the first attempt may be carried forward, and therefore it will only be necessary for students to re-sit the failed element(s).

6. Factors affecting performance

[P&G for examiners, Annex C: Consideration of medical and other special circumstances in examination and assessment; Examination Regulations, Regulations for the Conduct of University Examinations, Part 13]

There should be a statement explaining the procedure that will be adopted for the consideration of applications for factors affecting performance made under Part 13 of the Regulations for the Conduct of University Examinations.

The following text is provided as an example:

[Where a candidate or candidates have made a submission, under Part 13 of the Regulations for Conduct of University Examinations, that unforeseen factors may have had an impact on their performance in an examination, a subset of the board will meet to discuss the individual applications and band the seriousness of each application on a scale of 1-3 with 1 indicating minor impact, 2 indicating moderate impact, and 3 indicating very serious impact. When reaching this decision, examiners will take into consideration the severity and relevance of the circumstances, and the strength of the evidence. Examiners will also note whether all or a subset of papers were affected, being aware that it is possible for circumstances to have different levels of impact on different papers. The banding information will be used at the final board of examiners meeting to adjudicate on the merits of candidates. Further information on the procedure is provided in the Policy and Guidance for examiners, Annex C and information for students is provided at www.ox.ac.uk/students/academic/exams/guidance.]

7. Details of examiners and rules on communicating with examiners

List the name, position, and institution of the external examiner(s) as well as the names of all internal examiners (but not assessors). In conjunction with this, however, the conventions should underline the fact that candidates must not under any circumstances contact examiners directly. The following text is provided as an example:

[Candidates should not under any circumstances seek to make contact with individual internal or external examiners or assessors.]

Appendix [optional]

Provide details of any operational information for examiners if required. This would not normally be provided to students.