Pat Langley

When modeling a probability distribution with a Bayesian network, we are faced with the problem of how to handle continuous variables. Most previous work has either solved the problem by discretizing, or assumed that the data are generated by a single Gaussian. In this paper we abandon the normality assumption and instead use statistical methods for nonparametric density estimation. For a naive Bayesian classifier, we present experimental results on a variety of natural and artificial domains, comparing two methods of density estimation: assuming normality and modeling each conditional distribution with a single Gaussian; and using nonparametric kernel density estimation. We observe large reductions in error on several natural and artificial data sets, which suggests that kernel estimation is a useful tool for learning Bayesian models.

In this paper we present an average-case analysis of the Bayesian classifier, a simple probabilistic induction algorithm that fares remarkably well on many learning tasks. Our analysis assumes a monotone conjunctive target concept, Boolean attributes that are independent of each other and that follow a single distribution, and the absence of attribute noise. We first calculate the probability that the algorithm will induce an arbitrary pair of concept descriptions; we then use this expression to compute the probability of correct classification over the space of instances. The analysis takes into account the number of training instances, the number of relevant and irrelevant attributes, the distribution of these attributes, and the level of class noise. In addition, we explore the behavioral implications of the analysis by presenting predicted learning curves for a number of artificial domains. We also give experimental results on these domains as a check on our reasoning. Finally, we ...

Scientific discovery is often regarded as romantic and creative -- and hence unanalyzable -- whereas the everyday process of verifying discoveries is sober and more suited to analysis. Yet this fascinating exploration of how scientific work proceeds argues that however sudden the moment of discovery may seem, the discovery process can be described and modeled. Using the methods and concepts of contemporary information-processing psychology (or cognitive science) the authors develop a series of artificial-intelligence programs that can simulate the human thought processes used to discover scientific laws. The programs -- BACON, DALTON, GLAUBER, and STAHL -- are all largely data-driven, that is, when presented with series of chemical or physical measurements they search for uniformities and linking elements, generating and checking hypotheses and creating new concepts as they go along. Scientific Discovery examines the nature of scientific research and reviews the arguments for and against a normative theory of discovery; describes the evolution of the BACON programs, which discover quantitative empirical laws and invent new concepts; presents programs that discover laws in qualitative and quantitative data; and ties the results together, suggesting how a combined and extended program might find research problems, invent new instruments, and invent appropriate problem representations. Numerous prominent historical examples of discoveries from physics and chemistry are used as tests for the programs and anchor the discussion concretely in the history of science.