The Annals of Statistics

Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse Problems

Imre Csiszar

Full-text: Open access

Abstract

An attempt is made to determine the logically consistent rules for selecting a vector from any feasible set defined by linear constraints, when either all $n$-vectors or those with positive components or the probability vectors are permissible. Some basic postulates are satisfied if and only if the selection rule is to minimize a certain function which, if a "prior guess" is available, is a measure of distance from the prior guess. Two further natural postulates restrict the permissible distances to the author's $f$-divergences and Bregman's divergences, respectively. As corollaries, axiomatic characterizations of the methods of least squares and minimum discrimination information are arrived at. Alternatively, the latter are also characterized by a postulate of composition consistency. As a special case, a derivation of the method of maximum entropy from a small set of natural axioms is obtained.

Article information

Source
Ann. Statist. Volume 19, Number 4 (1991), 2032-2066.

Dates
First available in Project Euclid: 12 April 2007

Permanent link to this document
https://projecteuclid.org/euclid.aos/1176348385

Digital Object Identifier
doi:10.1214/aos/1176348385

Mathematical Reviews number (MathSciNet)
MR1135163

Zentralblatt MATH identifier
0753.62003

JSTOR
links.jstor.org

Subjects
Primary: 62A99: None of the above, but in this section
Secondary: 68T01: General 94A17: Measures of information, entropy 92C55: Biomedical imaging and signal processing [See also 44A12, 65R10, 94A08, 94A12]

Keywords
Image reconstruction linear constraitns logically consistent inference minimum discrimination information nonlinear projection nonsymmetric distance selection rules

Citation

Csiszar, Imre. Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse Problems. Ann. Statist. 19 (1991), no. 4, 2032--2066. doi:10.1214/aos/1176348385. https://projecteuclid.org/euclid.aos/1176348385


Export citation