The Annals of Statistics

Stochastic Complexity and Modeling

Jorma Rissanen

Full-text: Open access


As a modification of the notion of algorithmic complexity, the stochastic complexity of a string of data, relative to a class of probabilistic models, is defined to be the fewest number of binary digits with which the data can be encoded by taking advantage of the selected models. The computation of the stochastic complexity produces a model, which may be taken to incorporate all the statistical information in the data that can be extracted with the chosen model class. This model, for example, allows for optimal prediction, and its parameters are optimized both in their values and their number. A fundamental theorem is proved which gives a lower bound for the code length and, therefore, for prediction errors as well. Finally, the notions of "prior information" and the "useful information" in the data are defined in a new way, and a related construct gives a universal test statistic for hypothesis testing.

Article information

Ann. Statist. Volume 14, Number 3 (1986), 1080-1100.

First available in Project Euclid: 12 April 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Primary: 62A99: None of the above, but in this section
Secondary: 62M10: Time series, auto-correlation, regression, etc. [See also 91B84] 62F03: Hypothesis testing 60F99: None of the above, but in this section

Inference number of parameters model selection criteria prediction coding


Rissanen, Jorma. Stochastic Complexity and Modeling. Ann. Statist. 14 (1986), no. 3, 1080--1100. doi:10.1214/aos/1176350051.

Export citation