Let three homogeneous real linear forms be given in a three-dimensional real space. Their moduli give a mapping of the space into another space. In the second space, we consider the convex hull of images of all integer points of the first space except its origin. This convex hull is called the modular polyhedron. The best integer approximations to the root subspaces of these forms are given by the integer points whose images lie on the boundary of the modular polyhedron. Here we study the properties of the modular polyhedron and use them for the construction of an algorithm generalizing continued fraction. The algorithm gives the best approximations, and it is periodic for cubic irrationalities with positive discriminant. Attempts to generalize continued fraction were made by Euler, Jacobi, Dirichlet, Hermite, Poincare, Hurwitz, Klein, Minkowski, Voronoi, and by many others.