Visual error detection is just the first stage of runtime analysis. We also need to understand exactly what happens during the run. For that purpose, runtime analysis should be based on exact measurements of parameters vital for the application's execution:
- Runtime performance
- Memory usage
- Code coverage
Again, we will look at examples to understand this runtime analysis capability.
Measurement example 1: Function list view
Function list view is a typical runtime analysis view that can be generated with a specialized runtime analysis tool such as Quantify (Figure 5). It presents all important methods and/or objects of an application in tables that can be sorted by number of measured parameters; this allows developers analyzing code to find what methods used the most available memory at that point in time, as well as the slowest functions, the age of objects, and so forth.
This view provides exact information about the number of calls to methods, time spent in methods only, time spent and memory accumulated in selected methods and all their descendants, and so on.
Figure 5: Quantify function list view for a Visual C++ Application
Measurement example 2: Function detail view
A runtime analysis tool such as Quantify can also extend the information in measurement example 1 to include information about the distribution of measured data between calling methods and descendants. This is shown in the function detail view (Figure 6). This view highlights callers and descendants that contribute to a performance or memory hotspot - information that can help detect the exact cause of a performance or memory bottleneck.
Figure 6: Quantify function detail view for a Visual C#.NET application in Visual Studio.NET
Measurement example 3: Method coverage module view
As we explained earlier, in some cases - and especially when assessing the value of available testing methods - it is useful to measure the percentage of code covered while testing, or simply to mark all the methods that haven't been tested after a series of tests. You can do this with a tool such as PureCoverage®, which yields precise information about untested and dead code vs. tested code (Figure 7).
Figure 7: PureCoverage display of code coverage on the method level for a mixed C#.NET and VB.NET Application in Visual Studio.NET