## Journal of the Mathematical Society of Japan

### The Penrose transform for $Sp(n,R)$ and singular unitary representations

Hideko SEKIGUCHI

#### Abstract

We give a general definition of the Radon-Penrose transform for a Zuckerman-Vogan derived functor module of a reductive Lie group $G$, which maps from the Dolbeault cohomology group over a pseudo-Kähler homogeneous manifold into the space of smooth sections of a vector bundle over a Riemannian symmetric space. Furthermore, we formulate a functorial property between two Penrose transforms in the context of the Kobayashi theory of discretely decomposable restrictions of unitary representations.

Based on this general theory, we study the Penrose transform for a family of singular unitary representations of $Sp(n,R)$ in details. We prove that the image of the Penrose transform is exactly the space of global holomorphic solutions of the system of partial differential equations of minor determinant type of odd degree over the bounded symmetric domain of type CI, which is biholomorphic to the Siegel upper half space. This system might be regarded as a generalization of the Gauss-Aomoto-Gelfand hypergeometric differential equations to higher order. We also find a new phenomenon that the kernel of the Penrose transform is non-zero, which we determine explicitly by means of representation theory.

#### Article information

Source
J. Math. Soc. Japan, Volume 54, Number 1 (2002), 215-253.

Dates
First available in Project Euclid: 5 October 2007

Permanent link to this document
https://projecteuclid.org/euclid.jmsj/1191593961

Digital Object Identifier
doi:10.2969/jmsj/1191593961

Mathematical Reviews number (MathSciNet)
MR1864934

Zentralblatt MATH identifier
1020.32015

#### Citation

SEKIGUCHI, Hideko. The Penrose transform for $Sp(n,R)$ and singular unitary representations. J. Math. Soc. Japan 54 (2002), no. 1, 215--253. doi:10.2969/jmsj/1191593961. https://projecteuclid.org/euclid.jmsj/1191593961