We provide an existence theory for gradient Gibbs measures for $\mathbb{Z}$-valued spin models on regular trees which are not invariant under translations of the tree, assuming only summability of the transfer operator. The gradient states we obtain are delocalized. The construction we provide for them starts from a two-layer hidden Markov model representation in a setup which is not invariant under tree-automorphisms, involving internal q-spin models. The proofs of existence and lack of translation invariance of infinite-volume gradient states are based on properties of the local pseudo-unstable manifold of the corresponding discrete dynamical systems of these internal models, around the free state, at large q.