## The Annals of Statistics

- Ann. Statist.
- Volume 19, Number 4 (1991), 2032-2066.

### Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse Problems

#### Abstract

An attempt is made to determine the logically consistent rules for selecting a vector from any feasible set defined by linear constraints, when either all $n$-vectors or those with positive components or the probability vectors are permissible. Some basic postulates are satisfied if and only if the selection rule is to minimize a certain function which, if a "prior guess" is available, is a measure of distance from the prior guess. Two further natural postulates restrict the permissible distances to the author's $f$-divergences and Bregman's divergences, respectively. As corollaries, axiomatic characterizations of the methods of least squares and minimum discrimination information are arrived at. Alternatively, the latter are also characterized by a postulate of composition consistency. As a special case, a derivation of the method of maximum entropy from a small set of natural axioms is obtained.

#### Article information

**Source**

Ann. Statist., Volume 19, Number 4 (1991), 2032-2066.

**Dates**

First available in Project Euclid: 12 April 2007

**Permanent link to this document**

https://projecteuclid.org/euclid.aos/1176348385

**Digital Object Identifier**

doi:10.1214/aos/1176348385

**Mathematical Reviews number (MathSciNet)**

MR1135163

**Zentralblatt MATH identifier**

0753.62003

**JSTOR**

links.jstor.org

**Subjects**

Primary: 62A99: None of the above, but in this section

Secondary: 68T01: General 94A17: Measures of information, entropy 92C55: Biomedical imaging and signal processing [See also 44A12, 65R10, 94A08, 94A12]

**Keywords**

Image reconstruction linear constraitns logically consistent inference minimum discrimination information nonlinear projection nonsymmetric distance selection rules

#### Citation

Csiszar, Imre. Why Least Squares and Maximum Entropy? An Axiomatic Approach to Inference for Linear Inverse Problems. Ann. Statist. 19 (1991), no. 4, 2032--2066. doi:10.1214/aos/1176348385. https://projecteuclid.org/euclid.aos/1176348385