Kelly Androutsopoulos

Failed error propagation (FEP) is known to hamper software testing, yet it remains poorly understood. We introduce an information theoretic formulation of FEP that is based on measures of conditional entropy. This formulation considers the situation in which we are interested in the potential for an incorrect program state at statement s to fail to propagate to incorrect output. We define five metrics that differ in two ways: whether we only consider parts of the program that can be reached after executing s and whether we restrict attention to a single program path of interest .We give the results of experiments in which it was found that on average one in 10 tests suffered from FEP, earlier studies having shown that this figure can vary significantly between programs. The experiments also showed that our metrics are well-correlated with FEP. Our empirical study involved 30 programs, for which we executed a total of 7,140,000 test cases. The results reveal that the metrics differ in their performance but the Spearman rank correlation with failed error propagation is close to 0.95 for two of the metrics. These strong correlations in an experimental setting, in which all information about both FEP and conditional entropy is known, open up the possibility in the longer term of devising inexpensive information theory based metrics that allow us to minimise the effect of FEP.

Runtime monitoring is performed during system execution to detect whether the system's behaviour deviates from that described by requirements. To support this activity we have developed a monitoring framework that expresses the requirements to be monitored in event calculus - a formal temporal first order language. Following an investigation of how this framework could be used to monitor security requirements, in this paper we propose patterns for expressing three basic types of such requirements, namely confidentiality, integrity and availability. These patterns aim to ease the task of specifying confidentiality, integrity and availability requirements in monitorable forms by non-expert users. The paper illustrates the use of these patterns using examples of an industrial case study.