Validating Tests with the Coverage Analyzer in Development Systems

Why Use Code Coverage Analysis in Development Systems

As a developer, you are responsible for ensuring that your code is tested completely before it is shipped. You can write unit tests with ABAP Unit and functional or integration tests with eCATT. But how can you verify that your test suite actually tests your code thoroughly?

The answer to this question is: With the Coverage Analyzer (transaction SCOV). With the Coverage Analyzer, you can measure the code coverage of your unit and integration tests. Code coverage is no replacement for systematic planning of testing for path coverage and branch coverage. But code coverage is a useful indicator of the quality and completeness of your tests.

Coverage Analyzer features for developers include these:

• For an overview and for reporting, the Coverage Analyzer provides you with statistics on code coverage of programs, procedures, branches, and statements.

• To evaluate the quality of the code coverage and to find gaps in your tests, you can drill down from program-level code coverage statistics to your source code. There you can display exactly which branches and statements are tested and which are not.

• You can verify that your integration tests replicate actual user scenarios by comparing your test code coverage with the code coverage of production users.

Prerequisites for Monitoring Test Code Coverage

Before you can use the Coverage Analyzer, you must check these prerequisites:

• Is the Coverage Analyzer active in your development system?

  The Coverage Analyzer must be activated in the system for you to see code coverage results.

  For a developer, it is most efficient to be able to verify your test code in your development system. You will want to verify the code coverage of your unit tests as you implement them.

  Because the Coverage Analyzer has little impact on performance, there is no reason not to activate it in development systems. You should therefore press to activate the Coverage Analyzer in your development system if this is not already the case.

  Verifying the code coverage of your tests in an integration or consolidation system is of course possible, but it is less efficient. You must wait for transports of unit tests and integration tests to arrive in the system. You must then run your tests separately in the integration or consolidation system in order to collect the code coverage data.

• Have you defined a test group for your development project in the Coverage Analyzer?

  A test group lets you segregate the code coverage produced by specific users from all other code coverage statistics generated in the system. You can view and reset your group's code coverage in isolation in the Coverage Analyzer.

  Defining a test group is optional. You can always see system-wide code coverage statistics in test group ALL.

  SAP recommends for performance reasons that you define a maximum of 10 test groups in a system. You therefore will probably not be able to define a test group only for yourself.

  But defining a test group may make sense if you have a discrete group of developers working on a project. Or if you run Code Inspector test or other automated tests under a specific user.

  In such cases, test groups will make it easier and more convenient to make code coverage evaluation part of your test development.

  When you want to run a test, you can simply reset the test group in transaction SCOV, the Coverage Analyzer. After you have run the tests, you can view their code coverage in isolation, uncontaminated by all of the other activity in the system.

Using the Coverage Analyzer in a Development System

1. Implement unit tests, eCATT function tests, and integration tests for your programs.

2. To check the code coverage of unit tests, do the following:

   • Create a favorite in the ABAP Unit Browser for running your unit tests.

   • Select the code coverage option and run the tests.

   • Follow the link in the results display of the Browser to the special display of code coverage in the Browser.

3. If you are using the ABAP Unit Browser, then you do not need to reset the statistics in the Coverage Analyzer.

   But before you run tests with other tools, such as the Code Inspector, you can optionally reset your code coverage statistics. The reset helps ensure that the code coverage results are not contaminated by activity in the system other than your tests.

   Limit the reset as precisely as possible to your own programs, so that you do not inconvenience other users.

4. Optionally, automate the execution of your ABAP Unit tests with the Code Inspector (transaction SCI).

   If the Code Inspector runs with a special user, then you can create a test group for the user in the Coverage Analyzer.

   You can see the code coverage results in the Details Display.

5. If you run eCATT tests and integration tests in your development system, then you can view the code coverage in the Details Display of your development system.

Additional Guidelines for Using Code Coverage Effectively

• Use the Coverage Analyzer for quick ad hoc checks of test code coverage in your development systems. Large-scale QA testing with code coverage analysis belongs in integration or consolidation systems.

  The reason: The Coverage Analyzer resets the current code coverage for a processing block whenever the block is changed. Since programs are frequently changed in a development system, you will probably find that trying to organize systematic large-scale testing in a development system is more trouble than it is worth.

• Use Favorites in the ABAP Unit Browser to view the code coverage of your unit tests.

  Using favorites to group and run your ABAP Unit tests in the ABAP Unit Browser has a special advantage: In the test results you can display the code coverage of the tests you have just run in isolation from other code coverage results. You do not need to have a test group in the Coverage Analyzer for yourself. You do not need to reset Coverage Analyzer statistics beforehand.

  Here is how to request code coverage in the ABAP Unit browser.

  Here is help for understanding the code coverage statistics.

• Use the Code Inspector (transaction SCI) for automated execution of your unit tests.

  Use the Foreground Scheduler (transaction STPFE) for automated execution of your CATT/eCATT tests.

  You can view the results of these automated tests in the Coverage Analyzer (transaction SCOV).

  You should define test groups in SCOV in order to isolate results by programming project, up to the recommended limit of 10 groups per system.

  You will probably want to reset the code coverage for your test group before running the Code Inspector check.

  For the Code Inspector, you will want to define an object set and a check variant in transaction SCI. For ABAP Unit tests, you must activate Dynamic Tests in the check variant.

  You can then define and run the inspection.