Bayesian inference and uncertainty quantification in a general class of nonlinear inverse regression models is considered. Analytic conditions on the regression model $\{\mathcal{G}(\mathit{\theta }):\mathit{\theta }\in \mathrm{\Theta }\}$ and on Gaussian process priors for θ are provided such that semiparametrically efficient inference is possible for a large class of linear functionals of θ. A general Bernstein–von Mises theorem is proved that shows that the (non-Gaussian) posterior distributions are approximated by certain Gaussian measures centred at the posterior mean. As a consequence, posterior-based credible sets are valid and optimal from a frequentist point of view. The theory is illustrated with two applications with PDEs that arise in nonlinear tomography problems: an elliptic inverse problem for a Schrödinger equation, and inversion of non-Abelian X-ray transforms. New analytical techniques are deployed to show that the relevant Fisher information operators are invertible between suitable function spaces.