| 
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

References on the Cox Proof

Page history last edited by PBworks 16 years, 8 months ago

Some Key References on the Cox Proof of Bayesianism

R. T. Cox's theorem says that any consistent system of inference is isomorphic (mappable-onto) probability theory with Bayesian updating, so if you are using a non-Bayesian system of reasoning under uncertainty (including orthodox Neyman-Pearson statistics, neural nets, or belief functions) you are caught on this dilemma:

  • Either your system is inconsistent, in that it gives different answers when the same problem is posed in different ways
  • Or your system is just Bayesianism in disguise, or a special case of it.

The literature about the Cox proof that I am familiar with forms a three-act drama.

 

Act One

 

Cox, R. T. (1946) "Probability, frequency and reasonable expectation" American Journal of Physics

 

Cox, R. T. (1961) The Algebra of Probable Inference reissued by Johns Hopkins University Press 2001

Algebra of Probable Inference is a very influential book (see the gushing reviews on Amazon US). He shows how the Bayes-LaPlace system of probability follows from desiderata on a consistent system of inference, relates this to Shannon's information theory as a measure of uncertainty and shows how classic results including the law of large numbers follow from his two axioms.

 

Jaynes, E. T. (2003) Probability Theory: The Logic of Science. Cambridge: Cambridge University Press

This is the posthumous publication of a book that was gradually created by Jaynes in his later years as a "summing up" of his academic career. Chapter 2 is a guide through the Cox proof and Jaynes spells out with relish the disastrous consequences for orthodox statisticians and other non-Bayesians.

See also two reviews of the book in the March 2004 issue of SIAM News (Society for Industrial and Applied Mathematics). The review by Persi Diaconis hits right on the head why this book is so delightful to read and Terrance Fine's review hits right on the head why it is flawed and not suitable as an undergrad textbook. As a graduate student in inductive logic, I found it the single most useful reference I came across.

 

Act Two

 

Paris, J. (1995) The Uncertain Reasoner's Companion Cambridge: Cambridge University Press

Cox and Jaynes presented the proof in a way that was easy to follow, but which did not spell out all the premises that are essential to the argument. Paris sets out the proof with appropriate rigour and shows that it depends on an assumption that the possible values of plausibility have to form a dense set. The derivation of the negation rule is also shown to be less straightforward than in the Cox and Jaynes presentations.

 

Halpern, J. Y. (1999) "A counter-example to the theorems of Cox and Fine" Journal of AI research, 10, 67-85

Halpern provides a numerical example of a belief function which obeys all the Cox axioms apart from the controversial one.

 

Halpern, J. Y. (1999) "Cox's Theorem Revisited" Journal of Artificial Intelligence Research v11 pp429-435 (citeseer info)

Halpern looks at three ways to patch the hole in the proof, although he considers them "rather strong and, arguably, not natural". Specifically, his concerns are about the domains over which the system of reasoning is supposed to apply. If we are considering a finite domain of propositions, then Cox's controversial premise does not go through. One way around this is to imagine that the system will be applied to a potentially infinite number of finite domains.

 

Act Three

 

Snow, P. (1998) "On the Correctness and Reasonableness of Cox's Theorem for Finite Domains" Computational Intelligence v14 n3 pp452-459

Snow offers defences of the controversial axiom and argues that it was intentionally part of Cox's argument, but obscured by his discursive style. Here's one argument: subjective probabilities ought to match objective probabilities. Objective probabilities might take any real value from 0 to 1, hence so should subjective probabilities.

"The patch Paris fashioned is harmonious with Cox's writings after 1946, and appropriate for the applications which Cox made of his own theorem. Thus, there is no counterexample, and no lively controversy about the deductive correctness of Cox's conclusions."

 

Arnborg, S. and G. Sjödin. (2001) "What is the plausibility of probability?" working paper (citeseer info)

This proof brings in what the authors call "Extended Probability" which uses the mathematics of infinitesimals. Unlike Cox, they do not make an assumption that probabilities are real numbers. They show that any plausibility measure fitting their weaker assumptions can be embedded in a system of extended probability.

"[T]here is no simple way around the normative claims of the Robust Extended Bayes' method. So if one uses an alternative method, it will sooner or later have to be evaluated against the standard scale of rationality."

They admit that for pragmatic reasons we would probably not use infinitesimals but simply assume with Cox that plausibilities are real-valued.

 

Van Horn, K. S. (2003) "Constructing a Logic of Plausible Inference: a Guide To Cox's Theorem" International Journal of Approximate Reasoning v34 n1 pp3-24. (citeseer info)

This provides an abbreviated proof of the theorem, and sets out the justification for each premise in an impressively clear way. The approach here conditions on a state of information which may include non-propositional information. Amongst other consequences, this removes the constraint that in a finite field of propositions the set of possible plausibility values has to be finite.

Whereas Halpern has misgivings about the Cox proof because it does not allow finite domains, Van Horn is more optimistic because it is not the job of logic to be restricted to a finite domain.

"We (humanity) would have found it difficult to make any significant progress in mathematics if we had been required to come up with new rules of logic for every new domain we wished to investigate. It is the very fact that we have identified widely-applicable rules of logic, to be used in nearly every domain, that allows us to reason with confidence when entering new conceptual territory."

 

Conclusion

I use the term "Bayesian fundamentalism" for the proposition that any system of inference properly so-called has to be Bayesian in character or inconsistent. By publishing his theorem, Cox started a trend for Bayesian fundamentalism which is visible today in some work on uncertain reasoning and philosophy of science. Other scientists and philosophers are understandably wary of any fundamentalism. More recent work has set out the proof with more mathematical rigour than Cox's own presentation, and found an axiom that seems to be doing an awful lot of the work, casting some doubt on whether the theorem is as final and definitive as is claimed.

 

However, the controversial axiom is not superfluous but is demanded by both theoretical and pragmatic requirements. A general theory of logic which will apply to an indefinite number of domains has in effect a dense set of potential plausibilities. Arnborg, and Sjödin's paper is more cautious in its assumptions, but their extended probability systems could not be implemented in a computer, so if we wanted to automate inference, we would have to approximate with good old-fashioned probability. Cox and Jaynes seem to have assumed real-valued plausibility not to hide a gap in the argument, but out of good pragmatic sense.

 

Critiquing the mathematical content of these arguments is beyond my mathematical competence, so I am reliant on peer review to assure me that the proofs actually are proofs. It is only because the above papers are especially well-written that I can follow the argument at all, but it looks as though Bayesian fundamentalism is vindicated.


See also the Wikipedia entry for Cox's Theorem

Comments (0)

You don't have permission to comment on this page.