Home

Bayesian

A Bayesian believes that the best way of dealing with uncertainty is via probability theory; and at any point in time he is prepared to weigh up all the evidence and put a probability on any given hypothesis. Anyone who bets on the football, for instance, is acting as a Bayesian. Note that Bayesian arguments are prescriptive, not descriptive; it turns out that we are poor Bayesians.
Sewell (2006)

"One principal approach of statistical inference is Bayesian estimation, which incorporates reasonable expectations or prior judgments (perhaps based on previous studies), as well as new observations or experimental results. Another method is the likelihood approach, in which “prior probabilities” are eschewed in favour of calculating a value of the parameter that would be most “likely” to produce the observed distribution of experimental outcomes."
Encyclopædia Britannica, 2005.

"The other strand of inductive probability treats the probabilities of theories as a property of our attitude towards them; such probabilities are then interpreted, roughly speaking, as measuring degrees of belief. This is called the subjectivist or personalist interpretation. The scientific methodology based on this idea is usually referred to as the methodology of Bayesianism because of the prominent role it assigns to a famous result of the probability calculus known as Bayes's Theorem."
Howson and Urbach (1993) page 11

"According to the Bayesian view, scientific and indeed much of everyday reasoning is conducted in probabilistic terms."
Howson and Urbach (1993) page 1

"The sharpest and most persistent objection to the Bayesian approach has been that it treats certain subjective factors as relevant to the scientific appraisal of theories."
Howson and Urbach (1993)

"Like any theorem in probability, at the technical level Bayes' theorem merely provides a form of “uncertainty accounting”, which asserts that the left-hand side of the equation must equal the right-hand side. The interest and controversy, of course, lie in the interpretation and assumed scope of the formal inputs to the two sides of the equation—and it is here that past and present commentators part company in their responses to the idea that Bayes' theorem can or should be regarded as a central feature of the statistical learning process. At the heart of the controversy is the issue of the philosophical interpretation of probability‘objective or subjective?‘and the appropriateness and legitimacy of basing a scientific theory on the latter."
Bernardo and Smith (2000)

"Mathematically, Bayes' formula is merely a statement of conditional probability, and as such its validity is not in question. What has been questioned is its applicability to general problems of scientific inference. The difficulties concern
a. the meaning of probability, and
b. the choice of, and necessity for, the prior distribution."
Box and Tiao (1973)

"Good (1965, p. 10) suggests that ‘one is more or less a Bayesian depending on the precision with which one is prepared to make intuitive probability estimates’, although Good (1971b) uses the term ‘Bayesian’ in a much wider sense."
Walley, 1991, page 478

"Several different kinds of Bayesians exist, but it seems to me that the essential defining property of a Bayesian is that he regards it as meaningful to talk about the probability P(H | E) of a hypothesis H, given evidence E. Consequently, he will make more use of Bayes' theorem than a non-Bayesian will. Bayes' theorem itself is a trivial consequence of the product axiom of probability, and it is not a belief in this theorem that makes a person a Bayesian. Rather it is a readiness to incorporate intuitive probability into statistical theory and practice, and into the philosophy of science and of the behavior of humans, animals, and automata, and in an understanding of all forms of communication, and everything."
[...]
"An extreme Bayesian believes that every intuitive probability is precise, whereas less extreme Bayesians regard intuitive probabilities as only partially ordered so that each probability merely lies in some interval of values."
[...]
"One is more or less a Bayesian depending on the precision with which one is prepared to make intuitive probability estimates."
Good (1965)

Workshop Statistics: Discovery with Data, A Bayesian Approach, An Introduction to Bayesian Thinking

A dynamic Bayesian network is a Bayesian network that represents sequences of variables. These sequences are often time-series (for example in speech recognition) or sequences of symbols (for example protein sequences). The hidden Markov model and the Kalman Filter can be considered as the most simple dynamic Bayesian networks. http://en.wikipedia.org/wiki/Nonlinear_filter Particle filtering is to on-line learning what Markov chain Monte Carlo (MCMC) is to batch learning; and particle filtering is to non-linear non-Gaussian state-space models what the Kalman filter is to linear Gaussian state-space models.

Papers

Links