## Duke Mathematical Journal

### A notion of rank for unitary representations of reductive groups based on Kirillov's orbit method

#### Abstract

We introduce a new notion of rank for unitary representations of semisimple groups over a local field of characteristic zero. The theory is based on Kirillov's method of orbits for nilpotent groups over local fields. When the semisimple group is a classical group, we prove that the new theory is essentially equivalent to Howe's theory of $N$-rank (see [Ho4], [L2], [Sc]). Therefore our results provide a systematic generalization of the notion of a small representation (in the sense of Howe) to exceptional groups. However, unlike previous works that used ad hoc methods to study different types of classical groups (and some exceptional ones; see [We], [LS]), our definition is simultaneously applicable to both classical and exceptional groups. The most important result of this article is a general “purity” result for unitary representations which demonstrates how similar partial results in these authors' works should be formulated and proved for an arbitrary semisimple group in the language of Kirillov's theory. The purity result is a crucial step toward studying small representations of exceptional groups. New results concerning small unitary representations of exceptional groups will be published in a forthcoming paper [S]

#### Article information

Source
Duke Math. J. Volume 136, Number 1 (2007), 1-49.

Dates
First available: 4 December 2006

http://projecteuclid.org/euclid.dmj/1165244878

Digital Object Identifier
doi:10.1215/S0012-7094-07-13611-1

Mathematical Reviews number (MathSciNet)
MR2271294

Zentralblatt MATH identifier
1111.22013

#### Citation

Salmasian, Hadi. A notion of rank for unitary representations of reductive groups based on Kirillov's orbit method. Duke Mathematical Journal 136 (2007), no. 1, 1--49. doi:10.1215/S0012-7094-07-13611-1. http://projecteuclid.org/euclid.dmj/1165244878.

#### References

• J. Adams, Nonlinear covers of real groups, Int. Math. Res. Not. 2004, no. 75, 4031--4047.
• J. Adams and D. Vogan, eds., Representation Theory of Lie Groups (Park City, Utah, 1998), IAS/Park City Math. Ser. 8, Amer. Math. Soc., Providence, 2000.
• A. Borel, Linear Algebraic Groups, 2nd ed., Grad. Texts in Math. 126, Springer, New York, 1991.
• N. Bourbaki, Lie Groups and Lie Algebras, Chapters 4--6, trans. Andrew Pressley, Elem. Math. (Berlin), Springer, Berlin, 2002.
• I. D. Brown, Dual topology of a nilpotent Lie group, Ann. Sci. École Norm. Sup. (4) 6 (1973), 407--411.
• L. J. Corwin and F. P. Greenleaf, Representations of Nilpotent Lie Groups and Their Applications, Part I: Basic Theory and Examples, Cambridge Stud. Adv. Math. 18, Cambridge Univ. Press, Cambridge, 1990.
• V. V. Deodhar, On central extensions of rational points of algebraic groups, Amer. J. Math. 100 (1978), 303--386.
• M. Duflo, Théorie de Mackey pour les groupes de Lie algébriques, Acta Math. 149 (1982), 153--213.
• A. Dvorsky and S. Sahi, Tensor products of singular representations and an extension of the $\theta$-correspondence, Selecta Math. (N.S.) 4 (1998), 11--29.
• —, Explicit Hilbert spaces for uncertain unipotent representations, II, Invent. Math. 138 (1999), 203--224.
• W. T. Gan and G. Savin, On minimal representations definitions and properties, Represent. Theory 9 (2005), 46--93.
• D. Ginzburg, S. Rallis, and D. Soudry, A tower of theta correspondences for $G\sb 2$, Duke Math. J. 88 (1997), 537--624.
• M. I. Graev and A. A. Kirillov, Theory of group representations'' (in Russian), in Proc. Internat. Congr. Math. (Moscow, 1966), Izdat. Mir,'' Moscow, 1968, 373--379.; translation in 31 Invited Addresses (8 in Abstract) at the International Congress of Mathematicians (Moscow, 1966), Amer. Math. Soc. Transl. Ser. 2 70, Amer. Math. Soc., Providence, 1968.
• B. H. Gross and N. R. Wallach, On quaternionic discrete series representations, and their continuations, J. Reine Angew. Math. 481 (1996), 73--123.
• S. Helgason, Differential Geometry, Lie Groups, and Symmetric Spaces, Grad. Stud. Math. 34, Amer. Math. Soc., Providence, 2001.
• R. Howe, $\theta$-series and invariant theory'' in Automorphic Forms, Representations and $L$-Functions, Part 1 (Corvallis, Ore., 1977), Proc. Sympos. Pure Math. 33, Amer. Math. Soc., Providence, 1979, 275--285.
• —, On the role of the Heisenberg group in harmonic analysis, Bull. Amer. Math. Soc. (N.S.) 3 (1980), 821--843.
• —, Automorphic forms of low rank'' in Noncommutative Harmonic Analysis and Lie Groups (Marseille, 1980), Lecture Notes in Math. 880, Springer, Berlin, 1981, 211--248.
• —, On a notion of rank for unitary representations of the classical groups'' in Harmonic Analysis and Group Representations, Liguori, Naples, 1982, 223--331.
• —, Small unitary representations of classical groups'' in Group Representations, Ergodic Theory, Operator Algebras, and Mathematical Physics (Berkeley, 1984), Math. Sci. Res. Inst. Publ. 6, Springer, New York, 1987, 121--150.
• —, Oscillator representation, analytic preliminaries, unpublished notes, 1977.
• R. E. Howe and C. C. Moore, Asymptotic properties of unitary representations, J. Funct. Anal. 32 (1979), 72--96.
• R. Howe, G. Ratcliff, and N. Wildberger, Symbol mappings for certain nilpotent groups'' in Lie Group Representations, III (College Park, Md., 1982/1983), Lecture Notes in Math. 1077, Springer, Berlin, 1984, 288--320.
• J. E. Humphreys, Linear Algebraic Groups, Grad. Texts in Math. 21, Springer, New York, 1975.
• A. Joseph, A preparation theorem for the prime spectrum of a semisimple Lie algebra, J. Algebra 48 (1977), 241--289.
• D. Kazhdan, Some applications of the Weil representation, J. Analyse Mat. 32 (1977), 235--248.
• A. A. Kirillov, Unitary representations of nilpotent Lie groups (in Russian), Uspekhi Mat. Nauk 17, no. 4 (1962), 57--110.
• —, Merits and demerits of the orbit method, Bull. Amer. Math. Soc. (N.S.) 36 (1999), 433--488.
• —, Lectures on the Orbit Method, Grad. Stud. Math. 64, Amer. Math. Soc., Providence, 2004.
• J.-S. Li, On the classification of irreducible low rank unitary representations of classical groups, Compositio Math. 71 (1989), 29--48.
• —, Singular unitary representations of classical groups, Invent. Math. 97 (1989), 237--255.
• —, Automorphic forms with degenerate Fourier coefficients, Amer. J. Math. 119 (1997), 523--578.
• J.-S. Li and C.-B. Zhu, On the decay of matrix coefficients for exceptional groups, Math. Ann. 305 (1996), 249--270.
• H. Y. Loke and G. Savin, Rank and matrix coefficents for simply laced groups, preprint, 2005.
• G. W. Mackey, Induced representations of locally compact groups, I, Ann. of Math. (2) 55 (1952), 101--139.
• —, The Theory of Unitary Group Representations, Chicago Lectures in Math., Univ. of Chicago Press, Chicago, 1976.
• K. Magaard and G. Savin, Exceptional $\Theta$-correspondences, I, Compositio Math. 107 (1997), 89--123.
• C. C. Moore, Extensions and low dimensional cohomology theory of locally compact groups, I, II, Trans. Amer. Math. Soc. 113 (1964), 40--63.
• —, Decomposition of unitary representations defined by discrete subgroups of nilpotent groups, Ann. of Math. (2) 82 (1965), 146--182.
• H. Oh, Uniform pointwise bounds for matrix coefficients of unitary representations and applications to Kazhdan constants, Duke Math. J. 113 (2002), 133--192.
• V. Platonov and A. Rapinchuk, Algebraic Groups and Number Theory, Pure Appl. Math. 139, Academic Press, Boston, 1994.
• H. Salmasian, Isolatedness of the minimal representation and minimal decay of exceptional groups, Manuscripta Math. 120 (2006), 39--52.
• R. Scaramuzzi, A notion of rank for unitary representations of general linear groups, Trans. Amer. Math. Soc. 319 (1990), 349--379.
• J.-P. Serre, Lie Algebras and Lie Groups, 2nd ed., Lecture Notes in Math. 1500, Springer, Berlin, 1992.
• M. E. Taylor, Noncommutative Harmonic Analysis, Math. Surveys Monogr. 22, Amer. Math. Soc., Providence, 1986.
• J. Tits, Tabellen zu den einfachen Lie Gruppen und ihren Darstellungen, Lecture Notes in Math. 40, Springer, Berlin, 1967.
• P. Torasso, Méthode des orbites de Kirillov-Duflo et représentations minimales des groupes simples sur un corps local de caractéristique nulle, Duke Math. J. 90 (1997), 261--377.
• A. Weil, Sur certains groupes d'$\!$opérateurs unitaires, Acta Math. 111 (1964), 143--211.
• —, Basic Number Theory, 3rd ed., Grundlehren Math. Wiss. 144, Springer, New York, 1974.
• M. H. Weissman, The Fourier-Jacobi map and small representations, Represent. Theory 7 (2003), 275--299.