Optimization of mean-field spin glasses

Abstract

Mean-field spin glasses are families of random energy functions (Hamiltonians) on high-dimensional product spaces. In this paper, we consider the case of Ising mixed p-spin models,; namely, Hamiltonians ${\mathit{H}}_{\mathit{N}}:{\mathrm{\Sigma }}_{\mathit{N}}\to \mathbb{R}$ on the Hamming hypercube ${\mathrm{\Sigma }}_{\mathit{N}}={\{\pm 1\}}^{\mathit{N}}$, which are defined by the property that ${\{{\mathit{H}}_{\mathit{N}}(\mathit{\sigma })\}}_{\mathit{\sigma }\in {\mathrm{\Sigma }}_{\mathit{N}}}$ is a centered Gaussian process with covariance $\mathbb{E}\{{\mathit{H}}_{\mathit{N}}({\mathit{\sigma }}_1){\mathit{H}}_{\mathit{N}}({\mathit{\sigma }}_2)\}$ depending only on the scalar product $⟨{\mathit{\sigma }}_1,{\mathit{\sigma }}_2⟩$.

The asymptotic value of the optimum ${max}_{\mathit{\sigma }\in {\mathrm{\Sigma }}_{\mathit{N}}}{\mathit{H}}_{\mathit{N}}(\mathit{\sigma })$ was characterized in terms of a variational principle known as the Parisi formula, first proved by Talagrand and, in a more general setting, by Panchenko. The structure of superlevel sets is extremely rich and has been studied by a number of authors. Here, we ask whether a near optimal configuration σ can be computed in polynomial time.

We develop a message passing algorithm whose complexity per-iteration is of the same order as the complexity of evaluating the gradient of ${\mathit{H}}_{\mathit{N}}$, and characterize the typical energy value it achieves. When the p-spin model ${\mathit{H}}_{\mathit{N}}$ satisfies a certain no-overlap gap assumption, for any $\mathit{\epsilon }>0$, the algorithm outputs $\mathit{\sigma }\in {\mathrm{\Sigma }}_{\mathit{N}}$ such that ${\mathit{H}}_{\mathit{N}}(\mathit{\sigma })\ge (1-\mathit{\epsilon }){max}_{{\mathit{\sigma }}^{\prime }}{\mathit{H}}_{\mathit{N}}({\mathit{\sigma }}^{\prime })$, with high probability. The number of iterations is bounded in N and depends uniquely on ε. More generally, regardless of whether the no-overlap gap assumption holds, the energy achieved is given by an extended variational principle, which generalizes the Parisi formula.