Archive

Archive for January, 2012

Testing versus Checking

January 26, 2012 Leave a comment

I stumble upon this blog post Testing vs. Checking by Michael Bolton which I found very interesting and a fundamental for software testers. I tried to summarize it and place it in a table to see the distinction clearly.

Checking Testing
confirming existing beliefs finding new information
a process of confirmation, verification, and validation a process of exploration, discovery, investigation, and learning
focused on making sure that the program doesn’t fail focused on “learning sufficiently everything that matters about how the program works and about how it might not work.” – James Bach
We are checking when:
* we’ve made a change to the code and we want to make sure that everything that worked before still works. When we have an assumption that’s important, we check to make sure the assumption holds.
* developers write and modify their code, creating automated routines that they run frequently to check to make sure that the code hasn’t broken		 We are testing when:
* we configure, operate, and observe a product with the intention of evaluating it, or with the intention of recognizing a problem that we hadn’t anticipated
* when we’re trying to find out about the extents and limitations of the product and its design, and when we’re largely driven by questions that haven’t been answered or even asked before
A check provides a binary result—true or false, yes or no A test has an open-ended result
is all about asking and answering the question, “Does this assertion pass or fail?” is about asking and answering the question “Is there a problem here?”
checking can prove the presence of bugs, but not their absence the process of finding out whether our checks have been good enough
machine-decidable and are, in and of themselves, value-neutral requires human, which makes it a sapient process
when done by a programmer, is mostly a quality assurance practice. The quality assurance angle: a programmer helps to assure the quality of his work by checking it. is not quality assurance, but acts in service to it; we supply information to programmers and managers who have the authority to make decisions about the project.


Checkers Testers
A person who needs a clear, complete, up-to-date, unambiguous specification to proceed Does not require the certainty of a perfect specification to make useful observations and inferences about the product
A person who needs a test script to proceed His job is to discover information; often that information is in terms of inconsistencies between what people think and what’s true in reality
A person who does nothing but to compare a program against some reference A tester’s task might be to reveal problems that occur when our excellent code calls buggy code in someone else’s library, for which we don’t have a specification

I also liked how James Bach describes testing, “questioning a product in order to evaluate it.” Evaluation of a product doesn’t assure its quality, but it can inform decisions that will have an impact on quality.

Though people may or may not agree with these, it made me understand where to draw the line. =) Thanks Michael!
Categories: quality assurance Tags: , ,
    • Design a site like this with WordPress.com
    Get started