Page 292 - DCAP305_PRINCIPLES_OF_SOFTWARE_ENGINEERING

Principles of Software Engineering

Notes or the ability of the proposed structure to satisfy performance criteria. Execution testing can be
performed in any of the following manners:
1. Using hardware and software monitors
2. Simulating the functioning of all or part of the system using a simulation model
3. Creating a quick and dirty program(s) to evaluate the approximate performance of a
completed system.
Execution testing may be executed on-site or off-site for the performance of the test. For example,
execution testing can be performed on hardware and software before being acquired, or may
be done after the application system has been completed. The earlier the technique is used, the
higher the assurance that the completed application will meet the performance criteria.
Examples of the use of execution testing:
1. Calculating turnaround time on transactions processed through the application.
2. Determining that the hardware and software selected provide the optimum processing
capability.
3. Using software monitors to determine that the program code is effectively used.

Dynamic program instrumentation and analysis enables many applications including intrusion
detection and prevention, unfortunately, because these analyses are executed in line with
program execution, they can substantially impact system performance, greatly reducing their
utility. For example, analyses commonly used for detecting buffer overflows or use of undefined
memory routinely incur overheads on the order of slowed by the analysis. Often analyses whose
performance impact would be prohibitive if done inline can run with surprisingly minimal lag
if run in parallel. Third, after sight can run analyses offline for situations where analyses are not
known beforehand or are not time critical, such as when debugging. After sight is a general-
purpose analysis framework. Any analysis that can run in the critical path of execution can
run in after sight, as long as that analysis does not change the execution (this would break the
determinism that after sight’s replay mechanism relies upon). Also, after sight makes the entire
system state at each instruction boundary available for analyses, providing greater generality than
approaches based on sampling. Further, logs originating from the VMM can be replayed and
analyzed in different execution environments (e.g., a simulator or VMM). This flexibility greatly
eases program instrumentation and enables a variety of optimizations. We have implemented
an after sight prototype on the x86 architecture, building on the record and replay capability of
VMware Workstation. Our framework enables replay on the QEMU whole-system emulator,
which supports easy instrumentation during replay and analysis. With this framework, we have
implemented an online security analysis that can be used to detect buffer overflow attacks on
running systems. We also implemented an analysis that can perform checks for memory safety
and heap overflows, and we used this analysis to discover several new and serious bugs in
VMware ESX Server, Linux, and Windows applications.
14.4.3 The Case for Decoupled Analysis

After sight improve dynamic analysis by decoupling the analysis from the main workload,
while still providing the analysis with the identical, complete sequence of states from the main
workload. This combination of decoupling and reproducibility improves dynamic analysis in the
following ways. First, after sight allows analyses to be added to a running system without fear
of breaking the main workload, because after sight runs analyses on a separate virtual machine
from the main workload, new analyses can beaded without changing the running application,
operating system, or virtual machine monitor of the main workload. Second, after sight offers
users several choices along the safety/performance spectrum. Users who can tolerate some
lag between the analysis and the workload can improve the performance of the workload and
still get best-effort safety or offline analysis, while users who require synchronous safety can

286 LOVELY PROFESSIONAL UNIVERSITY