Dynamic software testing Assessment 3

Assessment item 3

Dynamic Testing

Task

Assignment 3 walks you through the full range of developer based dynamic software testing.

In Part A, you need to critically analyse the provided case study and design documents to determine the areas of highest product and technical risk. You then need to create a minimal ‘Master Test Plan’ in which you specify the amounts and types of testing you will apply to different aspects of the application.

In Part B, you are required to implement a full range of dynamic tests for a particular use case, ranging from low level unit tests, through at least two levels of integration testing, to system testing.

Part C asks you to engage in some critical reflection on the effectiveness and utility of different aspects of dynamic testing.

Please note that it is recognised that standard practice is to specify user acceptance tests before development activities occur, and to conduct unit testing as classes are initially coded. Unfortunately, the semester timetable makes this impractical.

Also note that the focus in this assignment is on the process and techniques applied, not the complexity of the code. It is recognised that the process applied is ‘way over the top’ for the size of problem addressed.

Finally, please note that the code supplied is deliberately NOT bug free. The purpose of your testing is to expose those bugs. You are specifically instructed NOT to modify the code to eliminate those bugs. You are ONLY to implement and report on the results of your testing.

Tasks

Part A – Master Test Plan (10 marks)

You are provided with a case study and a set of design documents and diagrams which support a use case. You are required to analyse the product and technical risks implied by these artefacts and develop a minimal ‘Master Test Plan’ which addresses these risks.

Quality criteria for a minimal master test plan include:

1. Provides an overview of the project objectives
2. Identifies testing objectives in terms of problem domain outcomes
3. Identifies areas of product and technical risk
4. Identifies a test strategy for each risk (test level at which risk is addressed, extent and coverage of testing, including static testing approaches)
5. For each test level, identifies the test approach (test environment, test goals, entry criteria, exit criteria, defect response)

Part B – Dynamic Testing (80 marks)

You are provided with an implementation and a ‘master test plan’ which specifies the areas of concern, types of testing and coverage required for the application. You are required to implement a suite of dynamic tests to enact that plan using best practice testing techniques and procedures.

Unit Testing - 30 marks

Implement a suite of unit tests according to the master test plan using best practice unit test techniques and processes.

These include:

1. Tests are well organised and well named
2. Tests are well structured (Arrange, act, assert internal structure)
3. Tests are isolated (Tests only one code unit)
4. Tests are independent (Can be run in any order, don’t depend on external services or state)
5. Tests for error conditions included
6. Tests achieve appropriate level of coverage (as specified in the master test plan)
7. Each test tests only one thing (No logic in test, in general one assert per test)
8. Tests are automated (Do not require visual inspection to differentiate passing or failing)
9. Makes effective use of a test framework and test substitute library as appropriate (such as Junit, and Mockito)
10. Tests separated from production code

Integration Testing - 30 marks

Implement a suite of integration tests according to the master test plan using best practice integration test techniques and processes.

Use a ‘bottom up’ or ‘top down’ approach to check interaction between entities and between control and entity classes. Using your ‘master test plan’ as a guide, make sure you test the most critical and/or risky system operation. You are not required to test boundary (user interaction) classes.

Guidelines for best practice integration testing include:

1. Tests are well organised and well named
2. Tests are well structured (Arrange, act, assert internal structure)
3. Tests are isolated to only those code units included in the intended level of integration.
4. Tests are independent (Can be run in any order, don’t depend on external services or state)
5. Tests for error conditions included
6. Tests achieve appropriate level of coverage (as specified in the master test plan)
7. Each test tests only one thing (no logic in test, in general one assert per test)
8. Tests separated from production code
9. All levels of integration are addressed
10. Tests are automated (Do not require visual inspection to differentiate passing or failing)
11. Makes effective use of a test framework and test substitute library as appropriate (such as Junit, and Mockito)

System Tests - 20 marks

On the basis of the requirement and design documentation identify some system test cases which can be used to check correct functionality in the implemented use case. Specify system test procedures, test data, and expected results. A template for a system test will be available in the Resources section of the subject's Interact site.

Carry out the system tests you specified. Use the test data and procedures specified and check that it meets stated acceptance criteria. Report on the success or otherwise of the test. Include screenshots as evidence of your results in your test reports.

System test quality criteria include:

1. Tests are well organised (test cases are organised into test scenarios addressing specific functional areas.
2. Test are well named
3. Preconditions are completely and concretely specified.
4. Test data are completely and concretely specified
5. Postconditions are completely and concretely specified
6. Test steps and expected results for each step are completely and concretely specified
7. Tests are objectively evaluated and supporting evidence for test results is provided

Part C – Reflective Practice (10 marks)

Write a reflective report in which you critically evaluate the following questions:

1. Which aspect of testing was the most effective? i.e. Which aspect of testing discovered the most defects for the effort expended?
2. How relevant and useful are the guidelines for good unit testing practices?
3. How easy is it to achieve comprehensive test coverage? What proportion of programming effort should be allocated to testing?
4. How effective is dynamic testing compared to static code review in terms of discovering defects?

Marking criteria and standards

Master Test Plan (10 marks)

Quality criteria:

1. Provides an overview of the project objectives
2. Identifies test objectives in terms of problem domain outcomes
3. Identifies areas of product and technical risk
4. Identifies a test strategy for each risk (test level at which risk is addressed, extent and coverage of testing, including static testing approaches))
5. For each test level, identifies the test approach (test environment, test goals, entry criteria, exit criteria, defect response)

Unit Testing (20 marks)

Quality criteria:

1. Tests are well organised and well named
2. Tests are well structured (Arrange, act, assert internal structure)
3. Tests are isolated (Tests only one code unit)
4. Tests are independent (Can be run in any order, don’t depend on external services or state)
5. Tests for error conditions included
6. Tests achieve appropriate level of coverage (as specified in the master test plan)
7. Each test tests only one thing (no logic in test, in general one assert per test)
8. Tests are automated (Do not require visual inspection to differentiate passing or failing)
9. Makes effective use of a test framework and test substitute library as appropriate (such as Junit, and Mockito)
10. Tests separated from production code

Integration Testing (20 marks)

Quality criteria:

1. Tests are well organised and well named
2. Tests are well structured (Arrange, act, assert internal structure)
3. Tests are isolated to only those code units included in the intended level of integration.
4. Tests are independent (Can be run in any order, don’t depend on external services or state)
5. Tests for error conditions included
6. All levels of integration are addressed
7. Tests achieve appropriate level of coverage (as specified in the master test plan)
8. Each test tests only one thing (no logic in test, in general one assert per test)
9. Tests are automated (Do not require visual inspection to differentiate passing or failing)
10. Makes effective use of a test framework and test substitute library as appropriate (such as Junit, and Mockito)
11. Tests separated from production code

System Tests - 20 marks

Quality criteria:

1. Tests are well organised (test cases are organised into test scenarios addressing specific functional areas.
2. Test are well named
3. Preconditions are completely and concretely specified.
4. Test data are completely and concretely specified
5. Postconditions are completely and concretely specified
6. Test steps and expected results for each step are completely and concretely specified
7. Tests are objectively evaluated and supporting evidence for test results is provided

Reflective Practice (10 marks)

Quality Criteria:

1. Responses address all questions
2. Responses are comprehensive, objective, and insightful
3. Responses are consistent with an external view, and supported by available evidence
4. Responses go beyond description and display critical analysis.
5. Logical conclusions and implications for future practice are stated.
6. Analysis and conclusions drawn are logically justified and/or correct in terms of subject material
7. All content relates directly to critical reflection, analysis, and argument for conclusions reached
8. Responses are concise, relevant, easy to read, and easy to understand.

Presentation
You should submit a zip file through containing the following:

1. A document containing your Master Test Plan
2. Your Development Directory containing all source and test code
3. A summary document explaining where test files for each section of the assignment can be found
4. A document containing your responses to the reflective practice task.

If you are required to submit using TurnItIn, submit the following:A Word or PDF document containing the concatenated contents of:

1. Your master plan
2. A screen shot showing the directory structure and contained files of your test development environment
3. The source code for all tests you developed, with each test file separately identified
4. Your responses to the reflective practice task.