Lesson 6.6

Design test cases: normal, boundary, invalid

Part 6: Errors, testing and filesReading, about 5 min2 exercises in this lesson

Design test cases: normal, boundary, invalid

By the end of this you will be able to choose test data of three kinds, normal, boundary and invalid, say what each kind checks, and write down the result each test should give before you run it.

A test is a value and an expected result

In lesson 6.1 you saw that a logic error gives no message. You only find it by working out the right answer yourself and comparing it with what the program gives. So a test is two things: a value to type in, and the result the program should give for it, worked out before you run the program.

A few tests cannot try every possible value, so the values have to be chosen well. Testers sort them into three kinds, and a good set of tests has some of each.

The three kinds

Take a speaker app. You type a volume, and it should print Volume set for a whole number from 0 to 10, and Volume must be 0 to 10 for anything else.

Normal data is a value well inside what is allowed, which the program should accept. For the speaker, 6 is normal data. It checks that the program works at all.
Boundary data sits at an edge of what is allowed: the smallest and the largest values that should be accepted, here 0 and 10. Testers also try the values just past each edge, here -1 and 11, which should be rejected. Some books call those boundary data too, and some call them invalid. Either way, test them.
Invalid data is a value the program should reject: a number far outside the range, such as 25, or a value of the wrong type, such as the word loud or the decimal 4.5. It checks that the program rejects bad values instead of using them or stopping. Invalid data is also called erroneous data.

Here is a test plan for the speaker, written before the program is run.

Value typed	Kind	Expected output
6	normal	Volume set
0	boundary	Volume set
10	boundary	Volume set
-1	just past an edge	Volume must be 0 to 10
11	just past an edge	Volume must be 0 to 10
25	invalid	Volume must be 0 to 10
loud	invalid	Volume must be 0 to 10

Why the edges matter most

Here is a first try at the speaker program.

volume = int(input())
if volume >= 0 and volume < 10:
    print("Volume set")
else:
    print("Volume must be 0 to 10")

import java.util.Scanner;

public static void main(String[] args) {
    Scanner scanner
        = new Scanner(System.in);
    int volume = Integer.parseInt(
        scanner.nextLine());
    if (volume >= 0 && volume < 10) {
        System.out.println("Volume set");
    } else {
        System.out.println(
            "Volume must be 0 to 10");
    }
}

Run the plan against it. For 6, 0, -1, 11 and 25 it prints exactly what the plan expects. Of the rows with a number, only 10 goes wrong: it prints Volume must be 0 to 10, although 10 is allowed. The test should be "10 or less", not "less than 10".

That is typical. A mistake in a comparison, such as "less than" where "or less" was meant, changes the result for one value only: the value at the edge. Every value well inside the range, and every value far outside it, gets the same result from the right test and the wrong one. So normal data alone would never find this bug. Only boundary data can.

Test both edges. A mistake can sit at the bottom edge as easily as the top, and a test of one edge says nothing about the other.

What invalid data finds

Now run the last row of the plan, loud. The program does not print either message. It stops with an error at the conversion, because it has no try. That is a second bug, and only invalid data could find it, because only invalid data is something other than a whole number.

Two common mistakes

Testing only normal values, because they are the ones you thought about while writing the program. They are the values least likely to find a bug.
Running the program first and then deciding the output looks right. Work out the expected result before you run each test, or a wrong answer can look believable.

Now try it yourself on the questions below.

Exercise 6.6.16.6.1 · Answer itBike hire test data

A bike-hire app asks how many hours a bike is wanted for. It accepts a whole number from 1 to 12, and rejects anything else with a message.

(a) State which kind of test data each of these values is: 7, 12 and seven. [3]

(b) The app gives the right result for all three values. Give one more test value that should be tried, and explain what it checks that the three values in (a) do not. [2]

[5]

Your answer

Exercise 6.6.26.6.2 · Answer itBooking boundary bug

A cafe takes table bookings for groups of 1 to 8 people. This program checks the group size. In Java it runs inside a main method, with Scanner imported.

size = int(input())
if size >= 1 and size <= 9:
    print("Booked")
else:
    print("Size must be 1 to 8")

Scanner scanner
    = new Scanner(System.in);
int size = Integer.parseInt(
    scanner.nextLine());
if (size >= 1 && size <= 9) {
    System.out.println("Booked");
} else {
    System.out.println(
        "Size must be 1 to 8");
}

(a) A student tests the program with 4 and with 12, and both give the right output. State which kind of test data 4 and 12 each are. [2]

(b) The program has a logic error. Identify a test value that shows it, and state what the program prints for that value. [2]

(c) The student adds a test with 8, and the program prints the right output for it too. Explain why the test with 8 does not show the error. [1]

[5]

Your answer