The Annals of Statistics

Yule’s “nonsense correlation” solved!

Philip A. Ernst, Larry A. Shepp, and Abraham J. Wyner

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

In this paper, we resolve a longstanding open statistical problem. The problem is to mathematically prove Yule’s 1926 empirical finding of “nonsense correlation” [J. Roy. Statist. Soc. 89 (1926) 1–63], which we do by analytically determining the second moment of the empirical correlation coefficient \begin{eqnarray*}&&\theta:=\frac{\int_{0}^{1}W_{1}(t)W_{2}(t)\,dt-\int_{0}^{1}W_{1}(t)\,dt\int_{0}^{1}W_{2}(t)\,dt}{\sqrt{\int_{0}^{1}W^{2}_{1}(t)\,dt-(\int_{0}^{1}W_{1}(t)\,dt)^{2}}\sqrt{\int_{0}^{1}W^{2}_{2}(t)\,dt-(\int_{0}^{1}W_{2}(t)\,dt)^{2}}},\end{eqnarray*} of two independent Wiener processes, $W_{1},W_{2}$. Using tools from Fredholm integral equation theory, we successfully calculate the second moment of $\theta$ to obtain a value for the standard deviation of $\theta$ of nearly 0.5. The “nonsense” correlation, which we call “volatile” correlation, is volatile in the sense that its distribution is heavily dispersed and is frequently large in absolute value. It is induced because each Wiener process is “self-correlated” in time. This is because a Wiener process is an integral of pure noise, and thus its values at different time points are correlated. In addition to providing an explicit formula for the second moment of $\theta$, we offer implicit formulas for higher moments of $\theta$.

Article information

Source
Ann. Statist., Volume 45, Number 4 (2017), 1789-1809.

Dates
Received: February 2016
Revised: August 2016
First available in Project Euclid: 28 June 2017

Permanent link to this document
https://projecteuclid.org/euclid.aos/1498636874

Digital Object Identifier
doi:10.1214/16-AOS1509

Mathematical Reviews number (MathSciNet)
MR3670196

Zentralblatt MATH identifier
06773291

Subjects
Primary: 60J65: Brownian motion [See also 58J65] 60G15: Gaussian processes
Secondary: 60G05: Foundations of stochastic processes

Keywords
Volatile correlation Wiener processes integral equations

Citation

Ernst, Philip A.; Shepp, Larry A.; Wyner, Abraham J. Yule’s “nonsense correlation” solved!. Ann. Statist. 45 (2017), no. 4, 1789--1809. doi:10.1214/16-AOS1509. https://projecteuclid.org/euclid.aos/1498636874


Export citation

References

  • [1] Aldrich, J. (1995). Correlations genuine and spurious in Pearson and Yule. Statist. Sci. 10 364–376.
  • [2] Boas, R. P. Jr. (1954). Entire Functions. Academic Press, New York.
  • [3] Erdös, P. and Kac, M. (1946). On certain limit theorems of the theory of probability. Bull. Amer. Math. Soc. 52 292–302.
  • [4] Feller, W. (1957). An Introduction to Probability Theory and Its Applications. Wiley, New York.
  • [5] Granger, C. W. J. and Newbold, D. (1974). Spurious regression in econometrics. J. Econometrics 2 111–120.
  • [6] Hendry, D. F. (1986). Economic modelling with cointegrated variables: An overview. Oxf. Bull. Econ. Stat. 48 201–212.
  • [7] Logan, B. F., Mallows, C. L., Rice, S. O. and Shepp, L. A. (1973). Limit distributions of self-normalized sums. Ann. Probab. 1 788–809.
  • [8] Logan, B. F. and Shepp, L. A. (1968). Real zeros of random polynomials. Proc. Lond. Math. Soc. (3) 18 29–35.
  • [9] Magnus, J. R. (1986). The exact moments of a ratio of quadratic forms in normal variables. Ann. Écon. Stat. 4 95–109.
  • [10] Mann, M. E., Zhang, Z., Hughes, M. K., Bradley, R. S., Miller, S. K., Rutherford, S. and Ni, F. (2008). Proxy-based reconstructions of hemispheric and global surface temperature variations over the past two millenia. Proc. Natl. Acad. Sci. USA 105 13252–13257.
  • [11] McShane, B. B. and Wyner, A. J. (2011). A statistical analysis of multiple temperature proxies: Are reconstructions of surface temperatures over the last 1000 years reliable? Ann. Appl. Stat. 5 5–44.
  • [12] Phillips, P. C. B. (1986). Understanding spurious regressions in econometrics. J. Econometrics 33 311–340.
  • [13] Phillips, P. C. B. (1998). New tools for understanding spurious regressions. Econometrica 66 1299–1325.
  • [14] Rogers, L. C. G. and Shepp, L. A. (2006). The correlation of the maxima of correlated Brownian motions. J. Appl. Probab. 43 880–883.
  • [15] Shepp, L. A. (1979). The joint density of the maximum and its location for a Wiener process with drift. J. Appl. Probab. 16 423–427.
  • [16] Yule, G. U. (1926). Why do we sometimes get nonsense correlations between time series? A study in sampling and the nature of time series. J. Roy. Statist. Soc. 89 1–63.