Stochastic Systems

Models with hidden regular variation: Generation and detection

Bikramjit Das and Sidney I. Resnick

Full-text: Open access

Abstract

We review the notions of multivariate regular variation (MRV) and hidden regular variation (HRV) for distributions of random vectors and then discuss methods for generating models exhibiting both properties concentrating on the non-negative orthant in dimension two. Furthermore we suggest diagnostic techniques that detect these properties in multivariate data and indicate when models exhibiting both MRV and HRV are plausible fits for the data. We illustrate our techniques on simulated data, as well as two real Internet data sets.

Article information

Source
Stoch. Syst., Volume 5, Number 2 (2015), 195-238.

Dates
Received: March 2014
First available in Project Euclid: 23 December 2015

Permanent link to this document
https://projecteuclid.org/euclid.ssy/1450879454

Digital Object Identifier
doi:10.1214/14-SSY141

Mathematical Reviews number (MathSciNet)
MR3442427

Zentralblatt MATH identifier
1346.60073

Subjects
Primary: 28A33: Spaces of measures, convergence of measures [See also 46E27, 60Bxx] 60G17: Sample path properties 60G51: Processes with independent increments; Lévy processes 60G70: Extreme value theory; extremal processes

Keywords
Regular variation multivariate heavy tails hidden regular variation tail estimation conditional extreme value model

Citation

Das, Bikramjit; Resnick, Sidney I. Models with hidden regular variation: Generation and detection. Stoch. Syst. 5 (2015), no. 2, 195--238. doi:10.1214/14-SSY141. https://projecteuclid.org/euclid.ssy/1450879454


Export citation

References

  • [1] Anderson, P. L. and Meerschaert, M. M. Modeling river flows with heavy tails. Water Resources Research, 34 (9): 2271–2280, 1998. ISSN 1944-7973. URL http://dx.doi.org/10.1029/98WR01449.
  • [2] Bingham, N. H., Goldie, C. M., and Teugels, J. L. Regular Variation. Cambridge University Press, 1987.
  • [3] Bollobás, B., Borgs, C., Chayes, J., and Riordan, O. Directed scale-free graphs. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms (Baltimore, 2003), pages 132–139, New York, 2003. ACM.
  • [4] Breiman, L. On some limit theorems similar to the arc-sin law. Theory Probab. Appl., 10: 323–331, 1965.
  • [5] Crovella, M. and Bestavros, A. Self-similarity in world wide web traffic: Evidence and possible causes. In Proceedings of the ACM SIGMETRICS ’96 International Conference on Measurement and Modeling of Computer Systems, volume 24, pages 160–169, 1996.
  • [6] Crovella, M., Bestavros, A., and Taqqu, M. S. Heavy-tailed probability distributions in the world wide web. In Taqqu, M. S., Adler, R., Feldman, R. eds, A Practical Guide to Heavy Tails: Statistical Techniques for Analysing Heavy Tailed Distributions. Birkhäuser, Boston, 1999.
  • [7] Das B. and Resnick, S. I. Conditioning on an extreme component: Model consistency with regular variation on cones. Bernoulli, 17 (1): 226–252, 2011a. ISSN 1350-7265.
  • [8] Das, B. and Resnick, S. I. Detecting a conditional extreme value model. Extremes, 14 (1): 29–61, 2011b.
  • [9] Das, B., Embrechts, P., and Fasen, V. Four theorems and a financial crisis. The International Journal of Approximate Reasoning, 54 (6): 701–716, 2013.
  • [10] Das, B., Mitra, A., and Resnick, S. Living on the multi-dimensional edge: Seeking hidden risks using regular variation. Advances in Applied Probability, 45 (1): 139–163, 2013. http://arxiv.org/abs/1108.5560.
  • [11] de Haan, L. and Ferreira, A. Extreme Value Theory: An Introduction. Springer-Verlag, New York, 2006.
  • [12] Drees, H., de Haan, L., and Resnick, S. I. How to make a Hill plot. Ann. Statist., 28 (1): 254–274, 2000.
  • [13] Durrett, R. T. Random Graph Dynamics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 2010. ISBN 978-0-521-15016-3.
  • [14] Embrechts, P., Kluppelberg, C., and Mikosch, T. Modelling Extreme Events for Insurance and Finance. Springer-Verlag, Berlin, 1997.
  • [15] Guerin, C. A., Nyberg, H., Perrin, O., Resnick, S. I., Rootzén, H., and Stărică, C. Empirical testing of the infinite source poisson data traffic model. Stochastic Models, 19 (2): 151–200, 2003.
  • [16] Heffernan, J. E. and Resnick, S. I. Limit laws for random vectors with an extreme component. Ann. Appl. Probab., 17 (2): 537–571, 2007. ISSN 1050-5164.
  • [17] Hernandez-Campos, F., Jeffay, K., Park, C., Marron, J. S., and Resnick, S. I. Extremal dependence: Internet traffic applications. Stoch. Models, 21 (1): 1–35, 2005. ISSN 1532-6349.
  • [18] Hult, H. and Lindskog, F. Regular variation for measures on metric spaces. Publ. Inst. Math. (Beograd) (N.S.), 80(94): 121–140, 2006. ISSN 0350-1302. http://dx.doi.org/10.2298/PIM0694121H.
  • [19] Ibragimov, R., Jaffee, D., and Walden, J. Diversification disasters. Journal of Financial Economics, 99 (2): 333–348, 2011.
  • [20] Jessen, A. H. and Mikosch, T. Regularly varying functions. Publ. Inst. Math. (Beograd) (N.S.), 80 (94): 171–192, 2006.
  • [21] Lindskog, F., Resnick, S. I., and Roy, J. Regularly varying measures on metric spaces: Hidden regular variation and hidden jumps. Probability Surveys, 11: 270–314, 2014. http://arxiv.org/abs/1307.5803.
  • [22] Maulik, K. and Resnick, S. I. Characterizations and examples of hidden regular variation. Extremes, 7 (1): 31–67, 2005.
  • [23] Pickands, J. Statistical inference using extreme order statistics. Ann. Statist., 3: 119–131, 1975.
  • [24] Resnick, S. I. Point processes, regular variation and weak convergence. Adv. Applied Probability, 18: 66–138, 1986.
  • [25] Resnick, S. I. Hidden regular variation, second order regular variation and asymptotic independence. Extremes, 5 (4): 303–336 (2003), 2002. ISSN 1386-1999.
  • [26] Resnick, S. I. Heavy-Tail Phenomena: Probabilistic and Statistical Modeling. Springer Series in Operations Research and Financial Engineering. Springer-Verlag, New York, 2007. ISBN: 0-387-24272-4.
  • [27] Resnick, S. I. Extreme Values, Regular Variation and Point Processes. Springer, New York, 2008. ISBN 978-0-387-75952-4. Reprint of the 1987 original.
  • [28] Resnick, S. I. and Samorodnitsky, G. Tauberian theory for multivariate regularly varying distributions with application to preferential attachment networks. Extremes, 2015. http://arxiv.org/pdf/1406.6395. http://dx.doi.org/10.007/s10687-015-0216-2.
  • [29] Resnick, S. I. and Stărică, C. Smoothing the Hill estimator. Adv. Applied Probab., 29: 271–293, 1997.
  • [30] Samorodnitsky, G., Resnick, S., Towsley, D., Davis, R., Willis, A., and Wan, P. Nonstandard regular variation of in-degree and out-degree in the preferential attachment model. Journal of Applied Probability, 53 (1), 2016. http://arxiv.org/pdf/1405.4882.pdf.
  • [31] Smith, R. L. Statistics of extremes, with applications in environment, insurance and finance. In Finkenstadt, B. and Rootzén, H. eds, SemStat: Seminaire Europeen de Statistique, Exteme Values in Finance, Telecommunications, and the Environment, pages 1–78. Chapman-Hall, London, 2003.
  • [32] Weller, G. B. and Cooley, D. A sum characterization of hidden regular variation with likelihood inference via expectation-maximization. Biometrika, 101 (1): 17–36, 2014. ISSN 0006-3444. http://dx.doi.org/10.1093/biomet/ast046.