The moduli space $\mathcal{M}_g$ of nonsingular projective curves of genus $g$ is compactified into the moduli $\overline{\mathcal{M}}_g$ of Deligne-Mumford stable curves of genus $g$. We compactify in a similar way the moduli space of abelian varieties by adding some mildly degenerating limits of abelian varieties.
A typical case is the moduli space of Hesse cubics. Any Hesse cubic is GIT-stable in the sense that its $\mathrm{SL}(3)$-orbit is closed in the semistable locus, and conversely any GIT-stable planar cubic is one of Hesse cubics. Similarly in arbitrary dimension, the moduli space of abelian varieties is compactified by adding only GIT-stable limits of abelian varieties (§ 14).
Our moduli space is a projective “fine” moduli space of possibly degenerate abelian schemes with non-classical non-commutative level structure over $\mathbf{Z}[\zeta_{N},1/N]$ for some $N\geq 3$. The objects at the boundary are singular schemes, called PSQASes, projectively stable quasi-abelian schemes.