Sự khác biệt giữa system testing và integration testing

Giống: đều là test toàn bộ hệ thông
Khác:
system testing: test trên môi trường phát triển, người test là đội dev
integration testing: test trên môi trường triển khai, người test là khách hàng

3 kỹ thuật blackbox testing

Blackbox Testing
	Equivalence partitioning
	Boundary Value Analysis
	Decision Table

3 kỹ thuật này dùng để cho dev tìm ra những bug thông thường hay mắc phải. (Vì bình thường dev chỉ code theo suy nghĩ của mình chứ không phải trên quan điểm của tester)

Differences Between Black Box Testing and White Box Testing

The Differences Between Black Box Testing and White Box Testing are listed below.

Criteria	Black Box Testing	White Box Testing
Definition	Black Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is NOT known to the tester	White Box Testing is a software testing method in which the internal structure/ design/ implementation of the item being tested is known to the tester.
Levels Applicable To	Mainly applicable to higher levels of testing:Acceptance Testing System Testing	Mainly applicable to lower levels of testing:Unit Testing Integration Testing
Responsibility	Generally, independent Software Testers	Generally, Software Developers
Programming Knowledge	Not Required	Required
Implementation Knowledge	Not Required	Required
Basis for Test Cases	Requirement Specifications	Detail Design

For a combination of the two testing methods, see Gray Box Testing.

WHITE BOX TESTING Fundamentals

DEFINITION
White Box Testing (also known as Clear Box Testing, Open Box Testing, Glass Box Testing, Transparent Box Testing, Code-Based Testing or Structural Testing) is a software testing method in which the internal structure/ design/ implementation of the item being tested is known to the tester. The tester chooses inputs to exercise paths through the code and determines the appropriate outputs. Programming know-how and the implementation knowledge is essential. White box testing is testing beyond the user interface and into the nitty-gritty of a system.
This method is named so because the software program, in the eyes of the tester, is like a white/ transparent box; inside which one clearly sees.
Definition by ISTQB

• white-box testing: Testing based on an analysis of the internal structure of the component or
  system.
• white-box test design technique: Procedure to derive and/or select test cases based on an
  analysis of the internal structure of a component or system.

EXAMPLE
A tester, usually a developer as well, studies the implementation code of a certain field on a webpage, determines all legal (valid and invalid) AND illegal inputs and verifies the outputs against the expected outcomes, which is also determined by studying the implementation code.
White Box Testing is like the work of a mechanic who examines the engine to see why the car is not moving.
LEVELS APPLICABLE TO
White Box Testing method is applicable to the following levels of software testing:

• Unit Testing: For testing paths within a unit.
• Integration Testing: For testing paths between units.
• System Testing: For testing paths between subsystems.

However, it is mainly applied to Unit Testing.
WHITE BOX TESTING ADVANTAGES

• Testing can be commenced at an earlier stage. One need not wait for the GUI to be available.
• Testing is more thorough, with the possibility of covering most paths.

WHITE BOX TESTING DISADVANTAGES

• Since tests can be very complex, highly skilled resources are required, with thorough knowledge of programming and implementation.
• Test script maintenance can be a burden if the implementation changes too frequently.
• Since this method of testing is closely tied with the application being testing, tools to cater to every kind of implementation/platform may not be readily available.

White Box Testing is contrasted with Black Box Testing. Read the Differences between Black Box Testing and White Box Testing.

How to use Decision Tables

A decision table is an excellent tool to use in both testing and requirements management. Essentially it is a structured exercise to formulate requirements when dealing with complex business rules. Decision tables are used to model complicated logic. They can make it easy to see that all possible combinations of conditions have been considered and when conditions are missed, it is easy to see this.

Click Here To Download Your Practical Requirements Strategy Now (it’s free)

Let’s take an example scenario for an ATM where a decision table would be of use.
A customer requests a cash withdrawal. One of the business rules for the ATM is that the ATM machine pays out the amount if the customer has sufficient funds in their account or if the customer has the credit granted. Already, this simple example of a business rule is quite complicated to describe in text. A decision table makes the same requirements clearer to understand:


In a decision table, conditions are usually expressed as true (T) or false (F). Each column in the table corresponds to a rule in the business logic that describes the unique combination of circumstances that will result in the actions. The table above contains three different business rules, and one of them is the “withdrawal is granted if the requested amount is covered by the balance.” It is normal to create at least one test case per column, which results in full coverage of all business rules.

One advantage of using decision tables is that they make it possible to detect combinations of conditions that would otherwise not have been found and therefore not tested or developed. The requirements become much clearer and you often realize that some requirements are illogical, something that is hard to see when the requirements are only expressed in text.

A disadvantage of the technique is that a decision table is not equivalent to complete test cases containing step-by-step instructions of what to do in what order. When this level of detail is required, the decision table has to be further detailed into test cases.

Decision tables can be used in all situations where the outcome depends on the combinations of different choices, and that is usually very often. In many systems there are tons of business rules where decision tables add a lot of value.

Decision tables should best be constructed during system design, since they become useful to both developers and testers. The requirements specialist also becomes more confident that everything that is important is actually documented. If there are no decision tables, testers can create them during test design to be able to write better test cases.

Click Here To Download Your Practical Requirements Strategy Now (it’s free)

Step 1 – Analyze the requirement and create the first column

Requirement: “Withdrawal is granted if requested amount is covered by the balance or if the customer is granted credit to cover the withdrawal amount”.

Express conditions and resulting actions in a list so that they are either TRUE or FALSE. In this case there are two conditions, “withdrawal amount ≤ balance” and “credit granted”. There is one result, the withdrawal is granted.

Step 2: Add Columns

Calculate how many columns are needed in the table. The number of columns depends on the number of conditions and the number of alternatives for each condition. If there are two conditions and each condition can be either true or false, you need 4 columns. If there are three conditions there will be 8 columns and so on.
Mathematically, the number of columns is 2 ^conditions. In this case 2² = 4 columns.
Number of columns that is needed:

The bottom line is that you should create more smaller decision tables instead of fewer larger ones, otherwise you run the risk of the decision tables being so large as to be unmanageable. Test the technique by picking areas with fewer business rules.

Now is the time to fill in the T (TRUE) and F (FALSE) for the conditions. How do you do that? The simplest is to say that it should look like this:
Row 1: TF
Row 2: TTFF
Row 3: TTTTFFFF

For each row, there is twice as many T and F as the previous line.
Repeat the pattern above from left to right for the entire row. In other words, for a table with 8 columns, the first row will read TFTFTFTF, the second row will read TTFFTTFF and the third row will read TTTTFFFF.



Tip: There are Excel templates to fill in decision tables if you need help.

Step 3: Reduce the table

Mark insignificant values with “-“. If the requested amount is less than or equal to the account balance it does not matter if credit is granted. In the next step, you can delete the columns that have become identical.



Check for invalid combinations. Invalid combinations are those that cannot happen, for example, that someone is both an infant and senior. Mark them somehow, e.g. with “X”. In this example, there are no invalid combinations.

Finish by removing duplicate columns. In this case, the first and third column are equal, therefore one of them is removed.

Step 4: Determine actions

Enter actions for each column in the table. You will be able to find this information in the requirement. Name the columns (the rules). They may be named R1/Rule 1, R2/Rule 2 and so on, but you can also give them more descriptive names.

Step 5: Write test cases

Write test cases based on the table. At least one test case per column gives full coverage of all business rules.

• Test case for R1: balance = 200, requested withdrawal = 200. Expected result: withdrawal granted.
• Test case for R2: balance = 100, requested withdrawal = 200, credit granted. Expected result: withdrawal granted.
• Test case for R3: balance = 100, requested withdrawal = 200, no credit. Expected Result: withdrawal denied.

Summary

Decision tables are a good way to describe requirements when there are several business rules that interact together. Using decision tables it becomes easier for the requirements specialist to write requirements which cover all conditions. As to the tester, it becomes easier for them to write complete test cases.
Write decision tables early, then they’ll become useful for requirements specialists, developers, end-users and testers.

Boundary Value Analysis & Equivalence Partitioning with Examples

Practically, due to time and budget considerations, it is not possible to perform exhausting testing for each set of test data, especially when there is a large pool of input combinations.

• We need an easy way or special techniques that can select test cases intelligently from the pool of test-case, such that all test scenarios are covered.
• We use two techniques - Equivalence Partitioning & Boundary Value Analysis testing techniques to achieve this.

In this tutorial, we will learn

• What is Boundary Testing?
• What is Equivalent Class Partitioning?
• Example 1: Equivalence and Boundary Value
• Example 2: Equivalence and Boundary Value
• Why Equivalence & Boundary Analysis Testing

What is Boundary Testing?

Boundary testing is the process of testing between extreme ends or boundaries between partitions of the input values.

• So these extreme ends like Start- End, Lower- Upper, Maximum-Minimum, Just Inside-Just Outside values are called boundary values and the testing is called "boundary testing".
• The basic idea in boundary value testing is to select input variable values at their:

1. Minimum
2. Just above the minimum
3. A nominal value
4. Just below the maximum
5. Maximum

In Boundary Testing, Equivalence Class Partitioning plays a good role

Boundary Testing comes after the Equivalence Class Partitioning.

What is Equivalent Class Partitioning?

Equivalent Class Partitioning is a black box technique (code is not visible to tester) which can be applied to all levels of testing like unit, integration, system, etc. In this technique, you divide the set of test condition into a partition that can be considered the same.

• It divides the input data of software into different equivalence data classes.
• You can apply this technique, where there is a range in input field.

Example 1: Equivalence and Boundary Value

• Let's consider the behavior of tickets in the Flight reservation application, while booking a new flight.

• Ticket values 1 to 10 are considered valid & ticket is booked. While value 11 to 99 are considered invalid for reservation and error message will appear, "Only ten tickets may be ordered at one time."

Here is the test condition

1. Any Number greater than 10 entered in the reservation column (let say 11) is considered invalid.
2. Any Number less than 1 that is 0 or below, then it is considered invalid.
3. Numbers 1 to 10 are considered valid
4. Any 3 Digit Number say -100 is invalid.

We cannot test all the possible values because if done, the number of test cases will be more than 100. To address this problem, we use equivalence partitioning hypothesis where we divide the possible values of tickets into groups or sets as shown below where the system behavior can be considered the same.

The divided sets are called Equivalence Partitions or Equivalence Classes. Then we pick only one value from each partition for testing. The hypothesis behind this technique is that if one condition/value in a partition passes all others will also pass. Likewise, if one condition in a partition fails, all other conditions in that partition will fail.

Boundary Value Analysis- in Boundary Value Analysis, you test boundaries between equivalence partitions

In our earlier example instead of checking, one value for each partition you will check the values at the partitions like 0, 1, 10, 11 and so on. As you may observe, you test values at both valid and invalid boundaries. Boundary Value Analysis is also called range checking.
Equivalence partitioning and boundary value analysis are closely related and can be used together at all levels of testing.

Example 2: Equivalence and Boundary Value

Suppose a password field accepts minimum 6 characters and maximum 10 characters
That means results for values in partitions 0-5, 6-10, 11-14 should be equivalent

Test Scenario #	Test Scenario Description	Expected Outcome
1	Enter 0 to 5 characters in password field	System should not accept
2	Enter 6 to 10 characters in password field	System should accept
3	Enter 11 to 14 character in password field	System should not accept

Examples 3: Input Box should accept the Number 1 to 10

Here we will see the Boundary Value Test Cases

Test Scenario Description	Expected Outcome
Boundary Value = 0	System should NOT accept
Boundary Value = 1	System should accept
Boundary Value = 2	System should accept
Boundary Value = 9	System should accept
Boundary Value = 10	System should accept
Boundary Value = 11	System should NOT accept

Why Equivalence & Boundary Analysis Testing

1. This testing is used to reduce very large number of test cases to manageable chunks.
2. Very clear guidelines on determining test cases without compromising on the effectiveness of testing.
3. Appropriate for calculation-intensive applications with large number of variables/inputs

Summary:

• Boundary Analysis testing is used when practically it is impossible to test large pool of test cases individually
• Two techniques - Equivalence Partitioning & Boundary Value Analysis testing techniques is used
• In Equivalence Partitioning, first you divide a set of test condition into a partition that can be considered.
• In Boundary Value Analysis you then test boundaries between equivalence partitions
• Appropriate for calculation-intensive applications with variables that represent physical quantities

BLACK BOX TESTING Fundamentals

Black Box Testing, also known as Behavioral Testing, is a software testing method in which the internal structure/ design/ implementation of the item being tested is not known to the tester. These tests can be functional or non-functional, though usually functional.

This method is named so because the software program, in the eyes of the tester, is like a black box; inside which one cannot see. This method attempts to find errors in the following categories:

• Incorrect or missing functions
• Interface errors
• Errors in data structures or external database access
• Behavior or performance errors
• Initialization and termination errors

Definition by ISTQB

• black box testing: Testing, either functional or non-functional, without reference to the
  internal structure of the component or system.
• black box test design technique: Procedure to derive and/or select test cases based on an
  analysis of the specification, either functional or non-functional, of a component or system
  without reference to its internal structure.

EXAMPLE
A tester, without knowledge of the internal structures of a website, tests the web pages by using a browser; providing inputs (clicks, keystrokes) and verifying the outputs against the expected outcome.
LEVELS APPLICABLE TO
Black Box Testing method is applicable to the following levels of software testing:

• Integration Testing
• System Testing
• Acceptance Testing

The higher the level, and hence the bigger and more complex the box, the more black box testing method comes into use.
BLACK BOX TESTING TECHNIQUES
Following are some techniques that can be used for designing black box tests.

• Equivalence partitioning: It is a software test design technique that involves dividing input values into valid and invalid partitions and selecting representative values from each partition as test data.
• Boundary Value Analysis: It is a software test design technique that involves determination of boundaries for input values and selecting values that are at the boundaries and just inside/ outside of the boundaries as test data.
• Cause Effect Graphing: It is a software test design technique that involves identifying the cases (input conditions) and effects (output conditions), producing a Cause-Effect Graph, and generating test cases accordingly.

BLACK BOX TESTING ADVANTAGES

• Tests are done from a user’s point of view and will help in exposing discrepancies in the specifications.
• Tester need not know programming languages or how the software has been implemented.
• Tests can be conducted by a body independent from the developers, allowing for an objective perspective and the avoidance of developer-bias.
• Test cases can be designed as soon as the specifications are complete.

BLACK BOX TESTING DISADVANTAGES

• Only a small number of possible inputs can be tested and many program paths will be left untested.
• Without clear specifications, which is the situation in many projects, test cases will be difficult to design.
• Tests can be redundant if the software designer/ developer has already run a test case.
• Ever wondered why a soothsayer closes the eyes when foretelling events? So is almost the case in Black Box Testing.

Black Box Testing is contrasted with White Box Testing. Read Differences between Black Box Testing and White Box Testing.

Black box testing