7. Preparing the examination

7.1. Assessment

Divisions should keep the nature and methods of assessment for each course under review. In reviewing the assessment procedures used throughout the University or any unit or sub-unit of the University, divisions should pay attention to:

  • the extent to which the assessment methods used

-     remain a valid, fair and reliable means of assessing student achievement;

-     provide appropriate evidence of the academic standards of the course being met by the majority of candidates;

-     are appropriate to the teaching methods employed and the intended learning outcomes of the course;

  • any evidence that the amount of assessment or its timing is having a detrimental impact on the work of candidates;
  • the evidence of the teaching quality and methods as revealed through the examination process and the reports of examiners.

Examiners may require candidates in taught course examinations to submit material solely by electronic means only when special subject regulations prescribe this.

This does not prohibit submission in electronic form or by electronic means in addition to submission by other modes, for example for the purposes of plagiarism detection, again when allowed by special subject regulations.

7.2. Examination conventions – responsibility of supervisory bodies

In approving conventions, supervisory bodies should ensure that consideration is given to:

  • clear criteria for marking;
  • internal moderation;
  • the agreed tariff of academic penalties for late submission;
  • aggregation of marks (any changes to aggregation procedures should be checked against a set of marks for a previous year to ensure that there are no unanticipated effects);
  • procedures for the reconciliation of marks in cases of disagreement between markers;
  • rules and regulations for progression (including qualifying tests in graduate courses);
  • final awards and classification (including criteria for the award of ‘Distinction’ where relevant) and;
  • compensation.

7.3. Marking scheme and conventions for classification

At their first meeting, the examiners should satisfy themselves, e.g. in the light of comments from their predecessors, that their conventions (including marking scheme and method of aggregating marks) are comprehensive and unambiguous. If this is not the case, they may suggest amendments and formalise interpretations: any such modifications must be approved by the supervisory body responsible for the course and the examination.  Modifications must be published to prospective candidates not less than one whole term before the examination takes place or, where assessment takes place in the first term of a course, at the beginning of that term.

Marking conventions should be as clear and comprehensive as possible.  They should include details of the decimal precision of the calculations (and thus conventions for rounding), details of the nature of any scaling used and criteria for the use of such scaling.  They should also include details of academic penalties for late submission.

Final marks must be expressed as whole numbers not decimal points, using the full 100-point scale. A copy of the finalised conventions must be supplied to each candidate and to the Proctors’ Office.

If the examiners find it necessary to make major and immediate changes to established examination conventions after approval by the divisional or faculty board, the chair should seek the approval of the board and the Proctors, since the latter will need to be satisfied that such changes will not have an adverse or discriminatory effect on candidates whose preparation for the examination is far advanced.

No changes should be made to conventions after marks are known (except when a particular run of marks reveals unsuspected ambiguities or omissions that have to be resolved).  If, at the end of the examination process, the examiners wish to propose major changes to the conventions they will pass on to their successors, they should include the proposals in their examiners’ report for consideration by the divisional or faculty boards or other responsible bodies.

Examiners should also satisfy themselves that if a computer algorithm is used in the classification process, its rules are fully consistent with the current conventions, especially if changes are being made to the conventions (see Section 12 below, particularly 12.4).

7.4. Setting of question papers

Principles relating to assessment are set out in Section 2 above.  Examiners are obliged to set papers in accordance with the prevailing regulations for the course, and in line with any current course handbook or web information.  Any discrepancy between the formal regulations and other information given to candidates must be reported to the Proctors (who will advise what action to take).  Precedent represented by past papers should also be taken into account.  Any substantial changes in the rubric or format of a question paper should be notified to candidates and tutors at an early date. Material for examination papers should not be taken from the Internet. In the case of foreign language sources this is particularly important because of the transcription problems that can arise. The external examiner should have the opportunity to comment on draft examination papers.

The importance of clear and unambiguous expression cannot be overstated. Examiners should use straightforward English that is not likely to present a barrier to comprehension by non-native speakers. Question papers should be subject to careful scrutiny by the Board. In the case of small subjects, a draft paper must be scrutinised by at least one established member of staff who is not the paper setter.

The setting of all papers for an examination should be completed according to a schedule overseen by the chair.  Mishaps are far more likely to befall a paper that does not progress through the stages of setting, scrutiny, and proof-reading along with the others. Errors in the setting of papers are very difficult to rectify once the examination has begun.

Where a paper is to be shared with another school, or where a paper has common content with that belonging to another school, this information should be notified to the Examinations and Assessments Section at an early stage in order to ensure that the timetable takes this into account, and to avoid the extended supervision of candidates. 

The Proctors are concerned about the number of errors appearing in question-papers and so ask chairs to ensure that the rubric and the content of questions are double-checked before each paper is submitted to the Schools. Examples of problems that have occurred in recent years include:

  • Papers not set in accordance with current syllabus;
  • Papers containing incorrect passages for analysis;
  • Papers containing images that are not clear;
  • Papers that reproduced questions from recent previous years;
  • Questions containing typographical errors or errors in mathematical formulae.

When the content of all examination papers has been agreed, final versions should be prepared as camera-ready copy under secure conditions in faculty or departmental offices.  7.5 below gives full details of how camera-ready copy should be prepared.

7.5. Production of question papers

The instructions which follow are particularly important. Examiners who hand in copies of question papers late or whose papers contain inaccuracies create considerable additional pressure on the staff of the Schools, and the potential for error is increased. The staff of the Schools are instructed to notify the Proctors of examiners failing to comply.

Examiners are responsible for producing camera-ready copy (CRC) of examination papers; that is, a final master which is completely ready for reproduction. Chairs of examiners should ensure that this is done to a timetable that allows sufficient time to prepare accurate CRC which has been approved by all examiners and can be delivered to Examination Schools on time (Examination Regulations, 2014, Regulations for the Conduct of University Examinations, Part 8, cl. 8.2-8.4, p. 20, ll. 28-43) The Head of Examinations and Assessments is responsible for printing the bulk copies required for the examination, and (once the examination is complete) for the publication of papers on the OXAM website.

The following notes provide advice on the detail of production.

(a)          Two originals (not photocopies) of the CRC should be delivered by hand to the Examination Schools (Monday – Friday, 8.30 am – 4 pm). Electronic copies can be submitted via USB Memory Stick or CD.  If copies are being submitted electronically, then the file should be in Portable Document Format (pdf) and saved as the paper reference number (e.g. 2305).

(b)         Details of any special printing or reproduction requirements (e.g. punched hole in question paper; photographs to be reproduced) should be included with the CRC.  Examiners should note that reproduced photographs may not be as clear as the originals, and should check legibility on the version to be given to the candidates (not CRC) in good time.

(c)          CRC must reach Schools no later than five weeks before the first day of the examination.  Where enlarged papers (e.g. A3 instead of A4) or tables or extracts are required, these should be submitted seven weeks in advance.  When Braille papers are required, papers should be submitted nine weeks in advance.

(d)         All printed papers will be held until the date of the examination by the Head of Examinations and Assessments.  If papers are to be sent elsewhere (e.g. a practical paper taken in a laboratory), this should be discussed with the Head of Examinations and Assessments when the CRC is submitted.

(e)         In the interests of security, copies of the reproduced papers will be retained at all times within the Schools and cannot be sent to chairs for checking.  Chairs who wish to inspect papers at the Schools should arrange this in advance by telephone (2-76917) or by e-mail (exam.arrangements@admin.ox.ac.uk). It is important that papers containing complex or colour images be checked well before the examination is scheduled to take place.

(f)          CRC must be prepared on white A4 paper and on one side of the paper only.  Care should be taken to make sure that the margins are no smaller than 26 mm at the top and bottom of the page and 21 mm at the left and right sides of the page.

(g)         The front page should not contain any questions, only headers and rubric, including information on any materials or special stationery provided, and, where appropriate, any items which the candidate is permitted to bring, e.g. calculator.  It should have the paper reference number (as shown on the entry form, e.g. 0225) at the top left-hand corner.  The title(s) of the examination(s) and the paper title(s) should be stated clearly (as shown in the annotated example on the next page, which should be used as a model). Example exam paper page 1 (78kb)

(h)         The reference number should be included at the bottom left-hand corner of the second and all subsequent pages.

(i)           The sequence of page numbers starts with the front (cover) page, which is page 1. Page numbers should appear in the centre of the bottom line.

(j)           The instruction ‘turn over’ should appear on page 3 and if necessary on every subsequent odd-numbered page, placed at the bottom right-hand corner, within the typing area described in(f) above.  Blank pages and the last page should be identified as such.

(k)         Papers will be produced directly from CRC without enlargement or reduction.  Copy should be as clear as possible, as there may be a slight loss of quality on reproduction. Best results are obtained by using a word-processor with a laser-quality printer. Whenever possible, use a proportionally spaced font (e.g. Times New Roman).

(l)           If appropriate software is not available, drawings, symbols, accents, etc. may be added in black ink, using a fine pen.  It may be best to produce diagrams (or passages of foreign-language text, etc.) separately and then paste them into position in the text (avoiding any surplus glue that might show on final copies).

(m)        Sheets of CRC should not be stapled or pinned together, but placed in a folder showing clearly the examination title, paper title and reference number.

(n)          The production of CRC (typing, layout, etc.) should be done either by examiners themselves or by departmental and faculty staff in normal working time.  In exceptional circumstances the Proctors may authorise payments for overtime worked by departmental and faculty staff in the production of CRC, provided approval is sought well in advance from the Proctors.

(o)         Where prior approval has been given by the Proctors, claims to cover the cost of production of papers must be made on a form supplied by the Head of Examinations and Assessments.   The claim must include details of the number of hours worked and the number of pages prepared.  All such claims must be submitted via the chair to the Head of Examinations and Assessments as early as possible.

7.6. Proof-reading of CRC

The accuracy of papers is the responsibility of the chair and the examiners, and CRC must be carefully checked before submission to the Examinations and Assessments Section, as there will be no proofs and limited opportunity to check it after the bulk copies are printed. Departments will be required by the Proctors to provide a statement on how they intend to reduce the numbers of errors in cases where there are serious or large numbers of errors on papers.

7.7. Security

Persons preparing examination papers should ensure that no-one can enter their room and observe examination material on a screen or in paper form on a desk or printer.  If possible, they should work in a separate, locked room not frequented by visitors.  Staff should take particular care when laptop computers or personal machines are used to process examination material at home or in other locations (e.g. in public places, or on public transport) outside the institution. Laptops containing examination material should have properly implemented security measures that are proportionate to the anticipated risks. These may include passwords, password-protected screensavers, biometric security mechanisms and encryption. Printed copies of draft examination papers should be securely destroyed (shredded) immediately after use. If they must be kept in paper form, say between examiners’ meetings, they should be stored in locked filing cabinets.

The Proctors should be notified at once if any breach of security is suspected.  It is essential that back-ups are maintained and that there is good control over versions of draft papers.  Any media storing backed-up papers must be kept secure.

Paper drafts should be transported between examiners by hand, or sent by Recorded Delivery.  Questions, and question papers, should not be sent electronically unless encrypted.  Advice on the encryption and decryption of examination papers may be obtained from IT Services.  No electronic transmissions should be made without previously informing the Proctors. In 2011-12, some boards of examiners were authorised by the Proctors to use WebLearn to aid them in sharing draft examination papers during paper preparation. This pilot scheme is now available to other boards of examiners, and the chair should contact the Examinations Manager to discuss the practicalities of implementation.

Any exception to these rules must be agreed in advance by the Proctors, who will need to be convinced (taking technical advice if necessary) that it will cause no breach in security.

The attention of all examiners is drawn to their obligation to keep all question papers strictly confidential.  In no circumstances should details of the questions or the discussion of papers at examiners’ meetings be disclosed to anyone other than examiners, properly appointed assessors, and, if required, the Pro-VC (Education) and Proctors.

Under no circumstance should examination questions be substantially the same as sample questions made available to candidates before the examination.   If tutors have been asked to suggest questions for an examination, it should be made clear that such questions must not be used in tutorials or classes, nor should sample solutions be provided to candidates in advance of the exam.  Special care should be taken where an examiner or assessor runs revision classes. Evidence of such similarities will be brought to the Proctors’ attention and may result in the striking out of the compromised paper from the examination.

7.8. Special requirements

Question papers which have special requirements, whether in timetabling (e.g. reading time; duration other than 3 hours) or the provision of materials (e.g. tables, reference books, tracing paper) should be agreed by the examiners (including those in joint schools with which papers may be shared), at their initial meeting.  A schedule of materials previously used by the examination will be provided by the Head of Examinations and Assessments; this schedule, amended if necessary, and any other information should then be returned to the Head of Examinations and Assessments as soon as possible.  As indicated above, each question paper should specify in the rubric on page 1 details of any special provisions or materials that relate to it.  Question papers that contain pages printed in colour may also need special consideration by the Head of Examinations and Assessments.

7.9. Printing of examination papers

Examiners are responsible for setting the question papers, the accuracy of the contents, and for the production of the camera-ready copy (CRC) in paper or pdf format (details in 7.5 above). Staff within the Examinations and Assessments Section are responsible for the printing of the bulk quantities required in the examination room.