Our main aim is to present a geometrically meaningful formula for the fundamental solutions to a second order sub-elliptic differential equation and to the heat equation associated with a sub-elliptic operator in the sub-Riemannian geometry on the unit sphere $S^3$. Our method is based on the Hamiltonian-Jacobi approach, where the corresponding Hamitonian system is solved with mixed boundary conditions. A closed form of the modified action is given. It is a sub-Riemannian invariant and plays the role of a distance on $S^3$.