Black Box Testing

By
P.Vinitha
Release Testing (Black box testing)
• Release testing is the process of testing a release
of the system that will be distributed to
customers.
• The primary goal of this process is to increase the
supplier's confidence that the system meets its
requirements.
• To demonstrate that the system meets its
requirements, it must be shown that it delivers
the specified functionality, performance and
dependability, and that it does not fail during
normal use.
Black Box Testing
• Black box testing, also called functional testing and
behavioral testing, focuses on determining whether
or not a program does what it is supposed to do
based on its functional requirements
Black Box Testing
• The tester presents inputs to the component or the system
and examines the corresponding outputs.
• If the outputs are not those predicted (i.e., if the outputs are
in set O,) then the test has detected a problem with the
software. Inputs causing
anomalous
Input test data I behaviour
e

System

Outputs which reveal

the presence of
Output test results Oe defects
Black Box Testing
• Black box testing attempts to find errors in the
external behavior of the code in the following
categories
– Incorrect or missing functionality
– Interface errors
– Errors in data structures used by interfaces
– Behavior or performance errors
– Initialization and termination errors
Guidelines for Black Box testing
• Choose inputs that force the system to
generate all error messages.
• Design inputs that cause input buffers to
overflow.
• Repeat the same input or series of inputs
numerous times.
• Force invalid outputs to be generated.
• Force computation results to be too large or
too small.
Advantages of Black Box Testing
• Efficient when used on Larger systems
• As the tester and developer are independent of each other,
test is balanced and unprejudiced
• Tester can be non-technical.
• There is no need of having detailed functional knowledge of
system to the tester.
• Tests will be done from a end user's point of view. Because
end user should accept the system. (This is reason,
sometimes this testing technique is also called as
Acceptance testing)
• Testing helps to identify the vagueness and contradiction in
functional specifications.
• Test cases can be designed as soon as the functional
specifications are complete
Disadvantages of Black Box Testing
• Test cases are tough and challenging to design,
without having clear functional specifications
• It is difficult to identify tricky inputs, if the test
cases are not developed based on specifications.
• It is difficult to identify all possible inputs in
limited testing time. So writing test cases is slow
and difficult
• Chances of having unidentified paths during this
testing
• Chances of having repetition of tests that are
already done by programmer.
Black-Box Testing Techniques
• Graph Based Testing Methods
• Equivalence Partitioning
• Boundary Value Analysis
• Comparison Testing
• Orthogonal Array Testing
Graph-based Testing
• Graph-based Testing is a black-box testing
technique that uses objects that are modeled
in software and the relationships among these
objects.
• Understanding the dynamics on how these
objects communicate and collaborate with
one another can derive test cases.
Graph-based Testing
• Create a graph of software objects and
identify the relationship of these objects.
• Using nodes and edges, create a graph of software
objects.
• Nodes represent the software objects
• Links represent the relationship between objects.
• Node weights describes the properties of the node.
Graph-based Testing
• Graph Notation

Object Directed Link Object

#2
#1

Node Weight
(Value)
Undirected Link Parallel Link

Object
#3
Graph-based Testing
• Sample Example

New
File
menu Select generates Document
Menu Window
Select (generation time <1.0 Sec)

Attributes:
Start dimension :
default Setting or
Is reperesented Contains Preferences
as Background color:
White
Document
Text Text Color: default
color or preferences
Graph-based Testing
• Guidelines for Graph-based Testing
1. Identify the start and stop points of the graph.
There should be an entry and exit nodes.
2. Name nodes and specify their properties.
3. Establish their relationship through the use of
edges. Specify the properties.
4. Derive test cases and ensure that there is node
and edge coverage.
Equivalence Partitioning
• Equivalence Testing is a black-box testing technique
that uses the input domain of the program.
• It divides the input domain into sets of data from
which test cases can be derived.
• Derived test cases are used to uncover errors that
reflect a class of errors.
• Thus, reduces the effort in testing the software.
• It makes use of equivalence classes, which are sets of
valid and invalid states
• that an input may be in.
Equivalence Partitioning
• Guidelines in Identifying Equivalence Classes
1. Input Condition is specified as a range of value. The
test case is one valid input, and two invalid
equivalence classes.
2. Input Condition requires a specific value. The test
case is one valid, and two invalid equivalence
classes.
3. Input condition specifies a member of a set. The test
case is one valid and one invalid.
4. Input condition is Boolean. The test case is one
valid, and one invalid
Equivalence Partitioning
• consider a text message code of registering a mobile number
to a text service of getting traffic reports. Assume that the
message requires the following structure:
Boundary Value Testing
• Boundary Value Analysis (BVA) is a Functional
Testing technique where the extreme
boundary values are chosen.
• Boundary values include maximum,
minimum, just inside/outside boundaries,
typical values, and error values.
Boundary Value Testing
• Guidelines in Deriving Test Cases Using BVA
– If input condition specifies range bounded by
values and b, the test cases should be designed
with values and b and just above and just below a
and b.
– If input condition specifies number of values, the
test cases should be developed that exercise the
minimum and maximum numbers. Values just
above and below the minimum and maximum are
also tested.
Comparison Testing
• Black-box testing for safety critical systems in which
independently developed implementations of
redundant systems are tested for conformance to
specifications.
• When redundant systems are developed, separate
software engineering teams develop independent
versions of an application using the same specification.
• Each version is tested with the same test data to
ensure that all provide the same output.
• Then all versions are executed in parallel with real-
time comparison of results to ensure consistency.
Comparison Testing
• Often equivalence class partitioning is used to
develop a common set of test cases for each
implementation.
• If the output from each version is the same then
it is assumed that all the implementations are
correct.
• If the output is different then careful analysis is
done.
• Even though more than one implementations are
developed only a single version will be used in
the delivered computer based system
Orthogonal Array Testing
• Orthogonal array testing is applied to
problems where the input domain is relatively
small but too large to accommodate
exhaustive testing.
• This method is useful for finding errors
associated with the software component.
Orthogonal Array Testing
• Orthogonal arrays are two dimensional arrays
of numbers which possess the interesting
quality that by choosing any two columns in
the array you receive an even distribution of
all the pair-wise combinations of values in the
array
terminology for working with
orthogonal arrays
• Runs: the number of rows in the array. This
directly translates to the number of test cases
that will be generated by the OATS technique.
• Factors: the number of columns in an array. This
directly translates to the maximum number of
variables that can be handled by this array.
• Levels: the maximum number of values that can
be taken on by any single factor. An orthogonal
array will contain values from 0 to Levels-1.
• Orthogonal arrays are most often named
following the pattern LRuns(LevelsFactors).
Orthogonal Array Testing
• How to use this technique
1. Decide how many independent variables will be tested for
interaction. This will map to the Factors of the array.
2. Decide the maximum number of values that each
independent variable will take on. This will map to the
Levels of the array.
3. Find a suitable orthogonal array with the smallest number
of Runs.3 A suitable array is one that has at least as many
Factors as needed from Step 1 and has at least as many
levels for each of those factors as decided in Step 2.
4. Map the Factors and values onto the array.
5. Choose values for any "left over" Levels.
6. Transcribe the Runs into test cases, adding any
particularly suspicious combinations that aren't
generated.
Orthogonal Array Testing
• Send function of a fax application
• Four parameters p1,p2,p3,p4
• Each takes three discrete values
• P1 takes on values
P1=1, Send it now
P1=2, Send it one hour later
P1=3, Send it after midnight.
• P2, P3 and P$ also takes values 1,2 and 3
Orthogonal Array Testing
(L9 orthogonal Array for fax send)
Test Parameters
Test Cases
P1 P2 P3 P4
1 1 1 1 1
2 1 2 2 2
3 1 3 3 3
4 2 1 2 3
5 2 2 3 1
6 2 3 1 2
7 3 1 3 2
8 3 2 1 3
9 3 3 2 1
Orthogonal Array Testing
• Can identify
– Single mode fault
– Double mode fault
– Multimode fault