Given two algebraic groups G, H over a field k, we investigate the representability of the functor of morphisms (of schemes) and the subfunctor of homomorphisms (of algebraic groups) . We show that is represented by a group scheme, locally of finite type, if the k-vector space is finite-dimensional; the converse holds if H is not étale. When G is linearly reductive and H is smooth, we show that is represented by a smooth scheme M; moreover, every orbit of H acting by conjugation on M is open.
"Homomorphisms of Algebraic Groups: Representability and Rigidity." Michigan Math. J. 72 51 - 76, August 2022. https://doi.org/10.1307/mmj/20217214