The Annals of Probability

Reflection Groups, Generalized Schur Functions, and the Geometry of Majorization

Morris L. Eaton and Michael D. Perlman

Full-text: Open access


Let $G$ be a closed subgroup of the orthogonal group $O(n)$ acting on $R^n$. A real-valued function $f$ on $R^n$ is called $G$-monotone (decreasing) if $f(y) \geqq f(x)$ whenever $y \precsim x$, i.e., whenever $y \in C(x)$, where $C(x)$ is the convex hull of the $G$-orbit of $x$. When $G$ is the permutation group $\mathscr{P}_n$ the ordering $\sim$ is the majorization ordering of Schur, and the $\mathscr{P}_n$-monotone functions are the Schur-concave functions. This paper contains a geometrical study of the convex polytopes $C(x)$ and the ordering $\precsim$ when $G$ is any closed subgroup of $O(n)$ that is generated by reflections, which includes $\mathscr{P}_n$ as a special case. The classical results of Schur (1923), Ostrowski (1952), Rado (1952), and Hardy, Littlewood and Polya (1952) concerning majorization and Schur functions are generalized to reflection groups. It is shown that a smooth $G$-invariant function $f$ is $G$-monotone iff $(r'x)(r'\nabla f(x))\leqq 0$ for all $x \in R^n$ and all $r \in R^n$ such that the reflection across the hyperplane $\{z\mid r'z = 0\}$ is in $G$. Furthermore, it is shown that the convolution (relative to Lebesgue measure) of two nonnegative $G$-monotone functions is again $G$-monotone. The latter extends a theorem of Marshall and Olkin (1974) concerning $\mathscr{P}_n$, and has applications to probability inequalities arising in multivariate statistical analysis.

Article information

Ann. Probab., Volume 5, Number 6 (1977), 829-860.

First available in Project Euclid: 19 April 2007

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier


Primary: 26A84
Secondary: 26A86 50B35 52A25 62H99: None of the above, but in this section

Orthogonal transformations groups generated by reflections Coxeter groups roots fundamental region $G$-orbit convex hull convex polytope convex polyhedral cone extreme point extreme ray edge $G$-monotone majorization Schur-concave function convolution


Eaton, Morris L.; Perlman, Michael D. Reflection Groups, Generalized Schur Functions, and the Geometry of Majorization. Ann. Probab. 5 (1977), no. 6, 829--860. doi:10.1214/aop/1176995655.

Export citation