Page 135 - DCAP310_INTRODUCTION_TO_ARTIFICIAL_INTELLIGENCE_AND_EXPERT_SYSTEMS

Unit 7: Probabilistic Reasoning

7.1.1 Inference Notes

Bayesian probability is one of the different interpretations of the concept of probability and
belongs to the category of evidential probabilities. The Bayesian interpretation of probability
can be seen as an extension of the branch of mathematical logic known as propositional logic
that enables reasoning with propositions whose truth or falsity is uncertain. To evaluate the
probability of a hypothesis, the Bayesian probabilist specifies some prior probability, which is
then updated in the light of new, relevant data.
The Bayesian interpretation provides a standard set of procedures and formulae to perform this
calculation. Bayesian probability interprets the concept of probability as “an abstract concept, a
quantity that we assign theoretically, for the purpose of representing a state of knowledge, or
that we calculate from previously assigned probabilities,” in contrast to interpreting it as a
frequency or “propensity” of some phenomenon.
The term “Bayesian” refers to the 18th century mathematician and theologian Thomas Bayes,
who provided the first mathematical treatment of a non-trivial problem of Bayesian inference.
Nevertheless, it was the French mathematician Pierre-Simon Laplace who pioneered and
popularised what is now called Bayesian probability.

Broadly speaking, there are two views on Bayesian probability that interpret the probability
concept in different ways. According to the objectivist view, the rules of Bayesian statistics can
be justified by requirements of rationality and consistency and interpreted as an extension of
logic. According to the subjectivist view, probability quantifies a “personal belief”. Many modern
machine learning methods are based on objectivist Bayesian principles. In the Bayesian view, a
probability is assigned to a hypothesis, whereas under the frequentist view, a hypothesis is
typically being assigned a probability.
7.1.2 Brief Introduction to the Bayesian Approach

Broadly speaking, there are two views on Bayesian probability that interpret the ‘probability’
concept in different ways. For objectivists, probability objectively measures the plausibility of
propositions, i.e. the probability of a proposition corresponds to a reasonable belief everyone
(even a “robot”) sharing the same knowledge should share in accordance with the rules of
Bayesian statistics, which can be justified by requirements of rationality and consistency. For
subjectivists, probability corresponds to a ‘personal belief’. For subjectivists, rationality and
coherence constrain the probabilities a subject may have, but allow for substantial variation
within those constraints. The objective and subjective variants of Bayesian probability differ
mainly in their interpretation and construction of the prior probability.

Notes The use of Bayesian probabilities as the basis of Bayesian inference has been
supported by several arguments, such as the Cox axioms, the Dutch book argument,
arguments based on decision theory and de Finetti’s theorem.

Axiomatic Approach

Richard T. Cox showed that Bayesian updating follows from several axioms, including two
functional equations and a controversial hypothesis of differentiability. It is known that Cox’s
1961 development (mainly copied by Jay Ness) is non-rigorous, and in fact a counter example
has been found by Halpern. The assumption of differentiability or even continuity is questionable
since the Boolean algebra of statements may only be finite. Other axiomatizations have been
suggested by various authors to make the theory more rigorous.

LOVELY PROFESSIONAL UNIVERSITY 129