What do these test results establish?

A teacher writes a set of instructions for labelling whole-number scores from 0 to 100, including both limits. The required rule, called the specification, is Pass for scores of 50 or higher and Fail for lower scores.

A tester follows the written instructions exactly for each score below. The expected result is what the required rule says; the actual result is what following the instructions produces. Assume there are no copying or calculation mistakes by the tester.

Table 3: Observed results of three independent classification tests.
Score being checked Expected result Actual result
20 Fail Fail
60 Pass Pass
75 Pass Fail

You have the results but cannot see the teacher’s instructions. Which conclusion is justified by these results alone?

  1. The instructions give the correct result for every score below 50 and the wrong result for every score above 50.
  2. The instructions give the wrong result for 75; the faulty step has not yet been identified.
  3. The expected result for 75 should change to Fail so that it agrees with the result obtained.
  4. The instructions must say “above 50” where they should say “50 or higher”.