Specification-Based Testing: Exploring partitions

When we are testing our code, we can’t only test things that will give us a successful return.

Tests should be a way to help you find bugs, so in this step of the specification-based testing, we are going to understand the partitions of our code. And explore scenarios that could break our code or have bugs in it.

Here we also have an opportunity to expand our domain knowledge and generate questions for ourselves to help you find definition failures in the business rule or any lack of understanding.

Let’s explore:

Since this blog post is part of a series (check out the other parts in case you missed it). We are continuing to use the ADD with carry implementation that I did for the Chip-8 emulator that I have written.

Here’s the code:

// 8XY4
// ADD
int registerxIndex = ((instruction & 0x0F00) >> 8);
int registeryIndex = ((instruction & 0x00F0) >> 4);

int firstToSum = registers[registerxIndex];
int secondRegisterToSum = registers[registeryIndex];

int registersSum = (firstToSum + secondRegisterToSum);
registers[registerxIndex] = registersSum & 0xFF;
if (registersSum > 255) registers[registers.length - 1] = 1;
else registers[registers.length - 1] = 0;
pc += 2;

Some important things to consider about the code to proceed:

• The registers array is an array of bytes.
  • Each position of the array can hold a value to be processed to emulate the registers of the CPU.
• pc and registers are a class level attributes
  • pc is a primitive integer.
• This code is just a snippet of a long implementation

With these considerations, we can continue.

The idea here is to look at each partition of the code. We start by separating what are the partitions of the code.

In this code, we can identify some partitions that will be:

• The instruction value:
  • It can’t be null because is primitive.
  • An integer that identifies which is the operation
  • It contains 8-bits to identify the two indexes of the registers in the array
• The registers array
  • An array of bytes with size 16.
  • Is the primitive type of byte.
• The program counter (PC)

Now we can explore those partitions, and write down the possibilities of states each partition could assume. Since we already thought about the success cases in the other parts of this series, this time we will explore how the code behaves with some unexpected inputs we can think of, based on the information that we described above.

• Instruction value:
  • It can be not initialized.
  • It can be zero.
  • It can be a negative number.
  • It can be an invalid instruction value.
• The register array:
  • A null array (Not initialized).
  • An array with a size smaller than 16.
  • An array with a size bigger than 16.
  • All registers are set to 0.
  • A negative index
  • A positive index
  • All registers are set to random values.
• Program counter:
  • A negative value
  • Zero
  • Max negative value
  • Max positive value
• Combining instruction and register together:
  • A valid instruction with registers null
  • Valid registers with an invalid instruction
  • Both invalid instructions and null registers
  • An instruction that could provoke an array index out of bounds when accessing the registers.

These are just a few examples, you can create a ton of combinations and possibilities. Now you could write a test for each combination that you could find. But that’s too time-consuming and too expensive. And also does not make sense.

So how can you approach that?

Question yourself and your knowledge about the code.

Which of those unexpected behaviors is more likely to occur? Are the inputs sanitized before? What are the situations this code will be executed? Will this code be reused for every developer in the project like a utils class? Or will be something strictly attached to a single flow? Do I need to check null or was already checked?

Reflect a little bit, that will help you understand the feature/project more comprehensively.

If you don’t have all the answers explore the code, ask your team, ask the stakeholders if the answer is not in the code, and discover what is possible and what’s not.

If you tried some unexpected inputs and something breaks, check if it’s possible to happen or not.

It’s easy to think, I’ll add a null check here because I tried a null input and got a Null Pointer, but before adding this you have to check if it’s possible to happen in your code.

To exemplify let’s look at one of the partitions:

Let’s analyze the Instruction partition. Look at the unexpected values you thought of one by one and check what’s possible to happen.

• It can be not initialized.
  • This will not be possible, the emulator has to fetch the instruction to execute, so it always has a value in it.
• It can be zero.
  • It can happen, in this case, the instruction should be ignored and not change the values of the registers, a test case here would be necessary.
• It can be a negative number.
  • It can’t happen, each instruction is a combination of two bytes fetched in memory, which in the worst case would be 0.
• It can be an invalid instruction value.
  • It can happen like the 0, where the instruction cannot be decoded, a test would be necessary in this case.

This is an example, you can obtain this information by looking at the code, reading documentation, and asking questions. This will reduce the amount of test combinations and help you to test what matters.

Now the next time you are going to write a test, try it out, this will clear your mind and help you to write the most efficient test possible.

Want to learn more about this topic?

In the following days, I’ll dive into each of those steps in more detail, follow me, and do not miss my next blog posts of this series when it comes out.

In my next blog post, we are going to discuss how to check the boundaries of your code. The most common place to have bugs in it.

Stay tuned to learn more! Don’t miss out!

Willian Moya (@WillianFMoya) / X (twitter.com)

Willian Ferreira Moya | LinkedIn

Also, follow me here on dev.to