Grading scanned answer sheets (Background)

At the University of Twente, paper multiple-choice exams are graded automatically by ConTest. After the exam, the Examinations Office scans the stack of answer forms, and the software grades every student and delivers the results as an Excel workbook. Figure 1 shows the two stages of that software. The questions that follow design one stage each. The stages share only two files, so each question can be answered without the other.

graph LR
    accTitle: Two stages of the grading software
    accDescr: The stack of scanned sheets goes into the stage Process sheets, which writes a results file and a key file. The stage Build workbook reads the two files and produces the Excel workbook.
    S[/"Stack of<br>scanned sheets"/] --> P[["Process sheets"]]
    P --> R[("Results file")]
    P --> K[("Key file")]
    R --> B[["Build workbook"]]
    K --> B
    B --> W[("Excel workbook")]
Figure 1: The two stages of the grading software. The first processes the scanned sheets into two files; the second builds the workbook from those files only.

The answer forms. ConTest offers two answer forms, which can be downloaded from the page above: form 38431 with up to 100 questions and options A to D, and form 25223 with up to 50 questions and options A to G. The positions of the bubbles differ between the two forms. The software reads three parts of a sheet:

  • the form code, printed in the top-left corner;
  • the student-number grid, with 7 columns, one per digit, and one bubble for each digit 1 through 9 and 0 in every column; and
  • one question row per question.

Name, course, version, and exam date are handwritten and are not read, so all students are graded with the same key. The teacher block and the true/false block of form 25223 are not used.

The examiner fills in the correct answers on an ordinary answer sheet and marks the student number 0000000. Such a sheet is called the key sheet. Together with the stack, the examiner supplies N, the number of questions, and the names of the files to write.

The two files. Questions and student-number columns are numbered from 1, as printed on the form. An answer is a letter, Blank when no bubble was marked, or Multiple when several were.

  • The results file has one row per student sheet in stack order: position in the stack, student number, the answer to every question, points, score (%), grade, blank rows, multiple rows, and reason. A sheet that is not graded is flagged: its row holds only the position and the reason.
  • The key file has the correct letter for every question.

How to answer. Answer each question with modular pseudocode or flowcharts. Give every procedure inputs, outputs, and one responsibility, and let a main procedure call the others. Each question lists operations that already exist and that you may use without designing them. Naming the procedures, including those operations, is part of the task: a good name says what the procedure does.

Build the results workbook

Design the stage Build workbook. The number of questions is the number of letters in the key file. You may use the operations in Table 1.

Table 1: Operations that may be used without designing them.
Operation Needs Gives
Read a file a file name the list of rows in the file
Create a workbook nothing an empty workbook
Add a worksheet a workbook, a name, column headings the new worksheet
Append a row to a worksheet a worksheet, the values nothing
Save a workbook a workbook, a file name nothing; the file is saved

Tasks. The tasks in Table 2 go from easy to complex.

Table 2: Tasks with their weight in the score of this question.
Task Your design must Weight
1. Design the workbook Name five worksheets: the grades, each student’s answers, the key, the flagged sheets, and a question analysis. For each, state what one row represents and list the column headings. Scores are percentages. 25%
2. Copy the files into worksheets Read both files, separate the graded rows from the flagged rows, write the grades, answers, key, and flagged worksheets, and save the workbook. 35%
3. Analyze the questions For every question, compute the percentage of graded students who answered it correctly, left it blank, or marked several options. The design must also work when no student was graded. 25%

Naming (5%) and test cases (10%). Name your procedures. Create test cases for your design: give a key file and a results file as input, and the worksheet rows you expect.

Write one procedure per worksheet. Most of them only copy fields: they add the worksheet with its headings and then add one row per item. Only the question analysis needs counting.