Tbilisi Mathematical Journal

Majorizatiuon and Zipf-Mandelbrot law

Naveed Latif, Đilda Pečarić, and Josip Pečarić

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

In this paper we show how the Zipf-Mandelbrot law is connected to the theory of majorization. Firstly we consider the Csiszár $f$-divergence for the Zipf-Mandelbrot law and then develop important majorization inequalities for these divergences. We also discuss some special cases for our generalized results by using the Zipf-Mandelbrot law. As applications, we present the majorization inequalities for various distances obtaining by some special convex functions in the Csiszár $f$-divergence for Z-M law like the Rényi $\alpha$-order entropy for Z-M law, variational distance for Z-M law, the Hellinger distance for Z-M law, $\chi^{2}$-distance for Z-M law and triangular discrimination for Z-M law. At the end, we give important applications of the Zipf's law in linguistics and obtain the bounds for the Kullback-Leibler divergence of the distributions associated to the English and the Russian languages.

Note

The publication was supported by the Ministry of Education and Science of the Russian Federation (the Agreement number No. 02.a03.21.0008.) This publication is partially supported by Royal Commission for Jubail and Yanbu, Kingdom of Saudi Arabia.

Article information

Source
Tbilisi Math. J., Volume 11, Issue 3 (2018), 1-27.

Dates
Received: 22 July 2017
Accepted: 21 February 2018
First available in Project Euclid: 3 October 2018

Permanent link to this document
https://projecteuclid.org/euclid.tbilisi/1538532023

Digital Object Identifier
doi:10.32513/tbilisi/1538532023

Mathematical Reviews number (MathSciNet)
MR3757497

Zentralblatt MATH identifier
1369.94404

Subjects
Primary: 94A15: Information theory, general [See also 62B10, 81P94]
Secondary: 94A17: Measures of information, entropy 26A51: Convexity, generalizations 26D15: Inequalities for sums, series and integrals

Keywords
Majorization inequailty Csiszár $f$-divergence Zipf-Mandelbrot law Zipf's law in linguistic Rényi $\alpha$-order entropy variational distance Hellinger discrimination $\chi^{2}$-distance and triangular discrimination

Citation

Latif, Naveed; Pečarić, Đilda; Pečarić, Josip. Majorizatiuon and Zipf-Mandelbrot law. Tbilisi Math. J. 11 (2018), no. 3, 1--27. doi:10.32513/tbilisi/1538532023. https://projecteuclid.org/euclid.tbilisi/1538532023


Export citation

References

  • M. A. Khan, N. Latif, and J. Pečarić, Generalization of majorization theorem, J. Math. Inequal., 9(3) (2015), 847-872.
  • M. A. Khan, N. Latif, I. Perić and J. Pečarić, On majorization for matrices, Math. Balkanica, New Series Vol. 27, 2013, Fasc. 1-2.
  • M. A. Khan, M. Niezgoda and J. Pečari\' c, On a refinement of the majorization type inequality, Demonstratio Math., 44(1) (2011), 49-57.
  • M. A. Khan, S. Khalid and J. Pečari\' c, Refinements of some majorization type inequalities, J. Math. Inequal.,7(1) (2013), 73-92.
  • C. J. C. Burgas, T, Shaked, E. Renshaw, A. Lazier, M. Deeds, N. Hamilton and G. Hallender, Learning to rank using dradient descent, In Proceedings of the Twenty-Second International Conference on Machine Learning (ICML), 89-96, 2005.
  • Z. Cao, T. Qin, T. Y. Liu, M. F. Tsai and H. Li, Learning to rank: from pair-wise approach to listwise approach, In Proceedings of the Twenty-Second International Conference on Machine Learning (ICML), 129-136, 2007.
  • I. Csisz$\acute{a}$r, Information-type measures of difference of probability distributions and indirect observations, Studia Sci Math Hungar 1967; $\mathbf{2}$: 299-318.
  • I. Csisz$\acute{a}$r, Information measure: A critical survey, Trans. 7th Prague Conf. on Info. Th., Statist. Decis. Funct., Random Processes and 8th European Meeting of Statist., $\mathbf{B}$, Academia Prague, 1978, 73-86.
  • S. S. Dragomir, Inequalities for Csiszar f-Divergence in Information Theory, RGMIA Monograph, 2001, URL: http://rgmia.org/papers/Csiszar/ULBCfDKLDApp.pdf.
  • V. Diodato, Dictionary of Bibliometrics, Haworth Press, New York: 1994.
  • L. Egghe and R. Rousseau, Introduction to Informetrics. Quantitative Methods in Library, Documentation and Information Science, Elsevier Science Publishers, New York: 1990.
  • A. Gelbukh and G. Sidorov, Zipf and Heaps laws, Coeffiecients Depend on Language, Proc. CICLing-2001, Conference on Intelligent Text Processing and Computational Linguistics, Maxico City.
  • R. Herbrich, T. P. Minka and T. Graepel, ${\rm{TrueSkill^{TM}}}$: A Bayesian skill rating system, In Advances in Nueral Information Processing Systems (NIPS), $\mathbf{19}$, 569-576.
  • T. Jaochims, A support vector method for multivariable performance measures, In Proceedings of the Twenty-Second International Conference on Machine Learning (ICML), 377-384, 2005.
  • N. Latif, \DJ. Pečarić and J. Pečarić, Majorization, Csiszar divergence and Zipf-Mandelbrot law, J. Inequal. Appl., 2017, 2017:197, 1-15.
  • N. Latif, J. Pečarić and I. Perić, On Majorization, Favard's and Berwald's Inequalities, Annals of Functional Analysis, 2(2011), no. 1, 31-50.
  • W. Li, Random texts exhibits Zipf's-law-like word frequency distribution, IEEE Transactions on Information Theory, 38($\mathbf{6}$): 1842-1845, 1992.
  • B. Mandelbrot, Information Theory and Psycholinguistics: A Theory of Words Frequencies. In Reading in Mathematical Social Science. (ed.) P. Lazafeld, N. Henry. Cambridge MA, MIT Press: 1966.
  • A. W. Marshall, I. Olkin and Barry C. Arnold, Inequalities: Theory of Majorization and Its Applications (Second Edition), Springer Series in Statistics, New York 2011.
  • M. Mati$\acute{c}$, C. E. M. Pearce and J. Pe$\check{c}$ari$\acute{c}$, Improvements of some bounds on entropy measures in information theory, Math. Inequal. Appl., $\mathbf{1}$ (1998), 295-304.
  • M. Mati$\acute{c}$, C. E. M. Pearce and J. Pe$\check{c}$ari$\acute{c}$, On an inequality for the entropy of a probability distribution, Acta Math. Hungarica, $\mathbf{85}$ (1999), 345-349.
  • M. Mati$\acute{c}$, C. E. M. Pearce and J. Pe$\check{c}$ari$\acute{c}$, Shannon's and related inequalities in information theory. Survey on classical inequalities, editor Themistocles M. Rassias, Kluwer Academic Publ., 2000, 127-164.
  • M. Mati$\acute{c}$, C. E. M. Pearce and J. Pe$\check{c}$ari$\acute{c}$, Some refinements of Shannon's inequalities, ANZIAM J. (formerly J. Austral. Math. Soc. Ser. B) $\mathbf{43}$ (2002), 493-511.
  • G. A. Miller, Language and Communication, McGraw-Hill, New York, 1951.
  • M. A. Montemurro, Beyond the Zipf-Mandelbrot law in quantitative linguistics. URL: arXiv:cond-mat/0104066v2, (2001).
  • D. Mouillot, A. Lepretre, Introduction of relative abundance distribution (RAD) indices, estimated from the rank-frequency diagrams (RFD), to assess changes in community diversity, Environmental Monitoring and Assessment, Springer, 63 $(\mathbf{2})$, 2000, p. 279-295.
  • M. E. J. Newman, Power laws, Pareto distributions and Zipf's law, URL: arXiv:cond-mat/0412004.
  • C. P. Niculescu and L. E. Persson, Convex Functions and their Applications, A contemporary Approach, CMS Books in Mathematics, $\mathbf{23}$, Springer-Verlage, New York, 2006.
  • J. Pečarić, F. Proschan and Y. L. Tong, Convex functions, Partial Orderings and Statistical Applications, Academic Press, New York, 1992.
  • J. Rennie and N. Srebro, Fast maximum margin matrix factorization for collaborative prediction, In Proceedings of the Twenty-Second International Conference on Machine Learning (ICML), 713-719, 2005.
  • Z. K. Silagadze, Citations and the Zipf-Mandelbrot Law, Complex Systems, (1997) $\mathbf{11}$, p: 487-499.
  • C. Tullo and J. R. Hurford, Modelling Zipfian Distributions in Language, URL: http://www.lel.ed.ac.uk/~jim/zipfjrh.pdf.
  • F. Xia, T. Y. Liu, J. Wang, W. Zhang and H. Li, Listwise approach to learning to rank - theory and algorithms. In Proceedings of the Twenty-Second International Conference on Machine Learning (ICML), 1192-1199, 2008.