The Role of Sufficiency and of Estimation in Thermodynamics

Abstract

The purpose of this paper is to point out (and to use) the relations of certain statistical concepts with "statistical" thermodynamics. (A) It is observed that Gibbs's "canonical distribution" of energy is precisely what statisticians have later labeled a "distribution of the exponential type". It follows that a rigorous treatment of the canonical law can be based upon the concept of "sufficiency", which is thereby related to the physical idea of "thermal equilibrium" and to the "zero-th principle of thermodynamics". In other words, the theory of physical fluctuations can be based upon "principles" very similar to those of the "phenomenological", or "classical, non-statistical" thermodynamics. Naturally, our results will be less detailed than those of statistical mechanics. However, the foundations of the latter theory still raise a host of unanswered problems, and it seems good in the meantime to show that the less powerful phenomenological theory has a wider scope than is commonly thought (see also [15]). The possibility of a purely phenomenological approach to statistical thermodynamics is not in itself a new idea. A procedure somewhat similar to ours has indeed been long ago suggested in Szilard's admirable, but very difficult and neglected, paper [18]--not to be confused with his [19]. Of course, Szilard used a quite different vocabulary; but, with hindsight, one may now say that he has co-invented the concept of sufficiency with R. A. Fisher; by showing that, under certain regularity conditions, Gibbs's canonical law is the only probability distribution with a single scalar sufficient statistic, Szilard also anticipated the results of G. Darmois [2], B. O. Koopman [10] and E. J. G. Pitman [16], but was partly anticipated by Poincare [17]. (B) The second thesis of the paper is independent of Szilard, and concerns the concept of temperature. For systems with a canonical energy, the temperature is the parameter of the Gibbs distribution; as such it is undefined for isolated systems with a determined energy. However, it is necessary to generalize the concept of temperature to isolated systems. Several definitions have been proposed and, although they all safely converge mutually for the usual very large systems, the temperature remains mathematically ambiguous for small isolated systems; it also becomes physically meaningless. We shall show that the temperature for systems-in-isolation should be viewed as a statistical estimate of the parameter of a conjectural canonical distribution, from which the presently isolated system may be presumed to have once been drawn. This interpretation explains the nature of the ambiguity of the concept of temperature; it also meets the actual practice of physicists; finally, some of the a priori conditions, which the physicists impose upon their "estimators", turn out to correspond to the statistical conditions of consistency, unbiasedness, and efficiency. Physicists also use two very interesting variants of consistency and unbiasedness, which we shall study under the names of "self-consistency" and "self-unbiasedness". The most commonly used temperature, due to Ludwig Boltzmann, turns out to be the maximum likelihood estimator. In summary, we hope to show that it is a great pity that mathematical and physical statistics should have developed largely independently of each other, while using the same concepts. By combining the rigor of modern statistics with the intuitive vigor of thermodynamics, both should be served well. However, as things stand, the mathematical statistician should not hope to unearth in the literature of physics any result as yet unknown to him. An important open problem suggested by this paper is the following. When sufficiency and estimation are defined in the most general terms, it seems that one should also be able to generalize the scope of thermodynamics. However, an approach such as that of P. R. Halmos and L. J. Savage [5] could not be applied to thermodynamics without substantial restrictions, as we shall show in Section 7. It remains to study these restrictions in greater detail, before one can assert that a non-void generalization of thermodynamics is possible. The problem is addressed to both mathematicians and physicists. We shall strive to reduce to the minimum the detailed knowledge of physics required to read this paper. If the reader's appetite for information about thermodynamics has been awakened, he could do no better than to make use of references [11] and [20].