Open Access
September, 1986 Stochastic Complexity and Modeling
Jorma Rissanen
Ann. Statist. 14(3): 1080-1100 (September, 1986). DOI: 10.1214/aos/1176350051


As a modification of the notion of algorithmic complexity, the stochastic complexity of a string of data, relative to a class of probabilistic models, is defined to be the fewest number of binary digits with which the data can be encoded by taking advantage of the selected models. The computation of the stochastic complexity produces a model, which may be taken to incorporate all the statistical information in the data that can be extracted with the chosen model class. This model, for example, allows for optimal prediction, and its parameters are optimized both in their values and their number. A fundamental theorem is proved which gives a lower bound for the code length and, therefore, for prediction errors as well. Finally, the notions of "prior information" and the "useful information" in the data are defined in a new way, and a related construct gives a universal test statistic for hypothesis testing.


Download Citation

Jorma Rissanen. "Stochastic Complexity and Modeling." Ann. Statist. 14 (3) 1080 - 1100, September, 1986.


Published: September, 1986
First available in Project Euclid: 12 April 2007

zbMATH: 0602.62008
MathSciNet: MR856807
Digital Object Identifier: 10.1214/aos/1176350051

Primary: 62A99
Secondary: 60F99 , 62F03 , 62M10

Keywords: Coding , inference , model selection criteria , number of parameters , prediction

Rights: Copyright © 1986 Institute of Mathematical Statistics

Vol.14 • No. 3 • September, 1986
Back to Top