If $G$ is a connected complex Lie group that is the complexification of a compact Lie group $U$, a “prehomogeneous vector space” for $G$ is a complex finite-dimensional vector $V$ together with a holomorphic representation of $G$ on $V$ such that $G$ has an open orbit in $V$. The open orbit is necessarily unique. Easy examples include the standard representation of $GL(n,\mathbb C)$ on $\mathbb C^n$, the standard representation of $Sp(n,\mathbb C)$ on $\mathbb C^{2n}$, the action of $K^{\mathbb C}$ on $\mathfrak p^+$ when $G/K$ is Hermitian, and certain actions obtained from the standard Vogan diagrams of some of the indefinite orthogonal groups.

The question that is to be studied is the decomposition of the symmetric algebra $S(V)$ under $U$. For any prehomogeneous vector space, the symmetric algebra $S(V)$ embeds in a natural $U$ equivariant fashion into $L^2(U/U_v)$, where $U_v$ is the subgroup of $U$ fixing a point $v$ in $V$ whose $G$ orbit is open. This fact gives a first limitation on what representations can occur in $S(V)$.

A “nilpotent element” $e$ in a finite-dimensional Lie algebra $\mathfrak g$ is an element for which $\text{ad}\,e$ is nilpotent. If $\mathfrak g$ is complex semisimple, the Jacobson–Morozov Theorem says that such an $e$, if nonzero, can be embedded in an “$\mathfrak s \mathfrak l_2$ triple” $(h,\,e,\, f)$, spanning a copy of $\mathfrak s \mathfrak l(2,\mathbb C)$.

When a complex semisimple Lie algebra is graded as $\bigoplus \mathfrak g^k$, $\text{ad}\,\mathfrak g^0$ provides a representation of $\mathfrak g^0$ on $\mathfrak g_1$, and Vinberg’s Theorem says that the result yields a prehomogeneous vector space. All such gradings arise from parabolic subalgebras of $\mathfrak g$. The examples above of the action of $K^{\mathbb C}$ on $\mathfrak f p^+$ and of certain actions obtained from indefinite orthogonal groups are prehomogeneous vector spaces of this kind.

For the first of these two examples, the action of $K$ on $S(\mathfrak p ^+)$ is described by a theorem of Schmid. In the special case of $SU(m,n)$, this theorem reduces to a classical theorem about the action of the product of two unitary groups on the space of polynomials on a matrix space. For the second of these two examples, the action on the symmetric algebra can be analyzed by using this classical theorem in combination with Littlewood’s Theorem about restricting representations from unitary groups to orthogonal groups.

In the general case of Vinberg’s Theorem, if $v$ is suitably chosen in the prehomogeneous vector space $V$, then $U/(U_v)_0$ fibers by a succession of three compact symmetric spaces, and hence $L^2(U/(U_v)_0)$ can be analyzed by iterating various branching theorems for compact symmetric spaces. This fact gives a second limitation on what representations can occur in $S(V)$.