This paper considers the problem of reconstructing $n$ independent uniform spins $X_{1},\dots,X_{n}$ living on the vertices of an $n$-vertex graph $G$, by observing their interactions on the edges of the graph. This captures instances of models such as (i) broadcasting on trees, (ii) block models, (iii) synchronization on grids, (iv) spiked Wigner models. The paper gives an upper bound on the mutual information between two vertices in terms of a bond percolation estimate. Namely, the information between two vertices’ spins is bounded by the probability that these vertices are connected when edges are opened with a probability that “emulates” the edge-information. Both the information and the open-probability are based on the Chi-squared mutual information. The main results allow us to re-derive known results for information-theoretic nonreconstruction in models (i)–(iv), with more direct or improved bounds in some cases, and to obtain new results, such as for a spiked Wigner model on grids. The main result also implies a new subadditivity property for the Chi-squared mutual information for symmetric channels and general graphs, extending the subadditivity property obtained by Evans–Kenyon–Peres–Schulman (Ann. Appl. Probab. 10 (2000) 410–433) for trees. Some cases of nonsymmetrical channels are also discussed.
References
[1] Abbe, E. (2017). Community detection and stochastic block models: Recent developments. J. Mach. Learn. Res. 18 Paper No. 177, 86. 1403.62110[1] Abbe, E. (2017). Community detection and stochastic block models: Recent developments. J. Mach. Learn. Res. 18 Paper No. 177, 86. 1403.62110
[2] Abbe, E., Bandeira, A. S., Bracher, A. and Singer, A. (2014). Decoding binary node labels from censored edge measurements: Phase transition and efficient recovery. IEEE Trans. Netw. Sci. Eng. 1 10–22.[2] Abbe, E., Bandeira, A. S., Bracher, A. and Singer, A. (2014). Decoding binary node labels from censored edge measurements: Phase transition and efficient recovery. IEEE Trans. Netw. Sci. Eng. 1 10–22.
[3] Abbe, E., Massoulié, L., Montanari, A., Sly, A. and Srivastava, N. (2017). Group Synchronization on Grids. Available at arXiv:1706.08561. 1706.08561[3] Abbe, E., Massoulié, L., Montanari, A., Sly, A. and Srivastava, N. (2017). Group Synchronization on Grids. Available at arXiv:1706.08561. 1706.08561
[4] Abbe, E. and Montanari, A. (2015). Conditional random fields, planted constraint satisfaction, and entropy concentration. Theory Comput. 11 413–443. 1351.68190 10.4086/toc.2015.v011a017[4] Abbe, E. and Montanari, A. (2015). Conditional random fields, planted constraint satisfaction, and entropy concentration. Theory Comput. 11 413–443. 1351.68190 10.4086/toc.2015.v011a017
[6] Banks, J. and Moore, C. (2016). Information-theoretic thresholds for community detection in sparse networks. Available at arXiv:1601.02658. 1601.02658[6] Banks, J. and Moore, C. (2016). Information-theoretic thresholds for community detection in sparse networks. Available at arXiv:1601.02658. 1601.02658
[7] Banks, J., Moore, C., Neeman, J. and Netrapalli, P. (2016). Information-theoretic thresholds for community detection in sparse networks. Proc. of COLT.[7] Banks, J., Moore, C., Neeman, J. and Netrapalli, P. (2016). Information-theoretic thresholds for community detection in sparse networks. Proc. of COLT.
[8] Banks, J., Moore, C., Vershynin, R., Verzelen, N. and Xu, J. (2018). Information-theoretic bounds and phase transitions in clustering, sparse PCA, and submatrix localization. IEEE Trans. Inform. Theory 64 4872–4994. 1401.94065 10.1109/TIT.2018.2810020[8] Banks, J., Moore, C., Vershynin, R., Verzelen, N. and Xu, J. (2018). Information-theoretic bounds and phase transitions in clustering, sparse PCA, and submatrix localization. IEEE Trans. Inform. Theory 64 4872–4994. 1401.94065 10.1109/TIT.2018.2810020
[9] Chin, P., Rao, A. and Vu, V. (2015). Stochastic block model and community detection in the sparse graphs: A spectral algorithm with optimal rate of recovery. Available at arXiv:1501.05021.. 1501.05021[9] Chin, P., Rao, A. and Vu, V. (2015). Stochastic block model and community detection in the sparse graphs: A spectral algorithm with optimal rate of recovery. Available at arXiv:1501.05021.. 1501.05021
[10] Coja-Oghlan, A., Krzakala, F., Perkins, W. and Zdeborová, L. (2018). Information-theoretic thresholds from the cavity method. Adv. Math. 333 694–795. 1397.82013 10.1016/j.aim.2018.05.029[10] Coja-Oghlan, A., Krzakala, F., Perkins, W. and Zdeborová, L. (2018). Information-theoretic thresholds from the cavity method. Adv. Math. 333 694–795. 1397.82013 10.1016/j.aim.2018.05.029
[11] Cover, T. M. and Thomas, J. A. (1991). Elements of Information Theory. Wiley Series in Telecommunications. Wiley, New York. 0762.94001[11] Cover, T. M. and Thomas, J. A. (1991). Elements of Information Theory. Wiley Series in Telecommunications. Wiley, New York. 0762.94001
[12] Csiszár, I. (1967). Information-type measures of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar. 2 299–318.[12] Csiszár, I. (1967). Information-type measures of difference of probability distributions and indirect observations. Studia Sci. Math. Hungar. 2 299–318.
[13] Deshpande, Y., Abbe, E. and Montanari, A. (2017). Asymptotic mutual information for the balanced binary stochastic block model. Inf. Inference 6 125–170. 1383.62021[13] Deshpande, Y., Abbe, E. and Montanari, A. (2017). Asymptotic mutual information for the balanced binary stochastic block model. Inf. Inference 6 125–170. 1383.62021
[15] Donoho, D. L., Maleki, A. and Montanari, A. (2009). Message-passing algorithms for compressed sensing. Proc. Natl. Acad. Sci. USA 106 18914–18919.[15] Donoho, D. L., Maleki, A. and Montanari, A. (2009). Message-passing algorithms for compressed sensing. Proc. Natl. Acad. Sci. USA 106 18914–18919.
[17] Evans, W., Kenyon, C., Peres, Y. and Schulman, L. J. (2000). Broadcasting on trees and the Ising model. Ann. Appl. Probab. 10 410–433. 1052.60076 10.1214/aoap/1019487349 euclid.aoap/1019487349[17] Evans, W., Kenyon, C., Peres, Y. and Schulman, L. J. (2000). Broadcasting on trees and the Ising model. Ann. Appl. Probab. 10 410–433. 1052.60076 10.1214/aoap/1019487349 euclid.aoap/1019487349
[18] Grimmett, G. (1999). Percolation, 2nd ed. Grundlehren der Mathematischen Wissenschaften 321. Springer, Berlin. MR1707339 0926.60004[18] Grimmett, G. (1999). Percolation, 2nd ed. Grundlehren der Mathematischen Wissenschaften 321. Springer, Berlin. MR1707339 0926.60004
[19] Guo, D., Shamai, S. and Verdú, S. (2005). Mutual information and minimum mean-square error in Gaussian channels. IEEE Trans. Inform. Theory 51 1261–1282. 1309.94099 10.1109/TIT.2005.844072[19] Guo, D., Shamai, S. and Verdú, S. (2005). Mutual information and minimum mean-square error in Gaussian channels. IEEE Trans. Inform. Theory 51 1261–1282. 1309.94099 10.1109/TIT.2005.844072
[20] Heimlicher, S., Lelarge, M. and Massoulié, L. (2012). Community detection in the labelled stochastic block model. Available at arXiv:1209.2910. 1209.2910[20] Heimlicher, S., Lelarge, M. and Massoulié, L. (2012). Community detection in the labelled stochastic block model. Available at arXiv:1209.2910. 1209.2910
[21] Javanmard, A., Montanari, A. and Ricci-Tersenghi, F. (2016). Phase transitions in semidefinite relaxations. Proc. Natl. Acad. Sci. USA 113 E2218–E2223. 1359.62188 10.1073/pnas.1523097113[21] Javanmard, A., Montanari, A. and Ricci-Tersenghi, F. (2016). Phase transitions in semidefinite relaxations. Proc. Natl. Acad. Sci. USA 113 E2218–E2223. 1359.62188 10.1073/pnas.1523097113
[22] Kesten, H. and Stigum, B. P. (1966). A limit theorem for multidimensional Galton–Watson processes. Ann. Math. Stat. 37 1211–1223. 0203.17401 10.1214/aoms/1177699266 euclid.aoms/1177699266[22] Kesten, H. and Stigum, B. P. (1966). A limit theorem for multidimensional Galton–Watson processes. Ann. Math. Stat. 37 1211–1223. 0203.17401 10.1214/aoms/1177699266 euclid.aoms/1177699266
[24] Mossel, E., Neeman, J. and Sly, A. (2012). Stochastic block models and reconstruction. Available at arXiv:1202.1499 [math.PR]. 1202.1499 1350.05154 10.1214/15-AAP1145 euclid.aoap/1472745457[24] Mossel, E., Neeman, J. and Sly, A. (2012). Stochastic block models and reconstruction. Available at arXiv:1202.1499 [math.PR]. 1202.1499 1350.05154 10.1214/15-AAP1145 euclid.aoap/1472745457
[25] Perry, A., Wein, A. S. and Bandeira, A. S. (2016). Statistical limits of spiked tensor models. Available at arXiv:1612.07728. 1612.07728 07199304 10.1214/19-AIHP960 euclid.aihp/1580720488[25] Perry, A., Wein, A. S. and Bandeira, A. S. (2016). Statistical limits of spiked tensor models. Available at arXiv:1612.07728. 1612.07728 07199304 10.1214/19-AIHP960 euclid.aihp/1580720488
[26] Perry, A., Wein, A. S., Bandeira, A. S. and Moitra, A. (2018). Optimality and sub-optimality of PCA I: Spiked random matrix models. Ann. Statist. 46 2416–2451. 1404.62065 10.1214/17-AOS1625 euclid.aos/1534492840[26] Perry, A., Wein, A. S., Bandeira, A. S. and Moitra, A. (2018). Optimality and sub-optimality of PCA I: Spiked random matrix models. Ann. Statist. 46 2416–2451. 1404.62065 10.1214/17-AOS1625 euclid.aos/1534492840
[27] Polyanskiy, Y. and Wu, Y. (2018). Application of information-percolation method to reconstruction problems on graphs. Available at arXiv:1806.04195. 1806.04195 07195621 10.4171/MSL/10[27] Polyanskiy, Y. and Wu, Y. (2018). Application of information-percolation method to reconstruction problems on graphs. Available at arXiv:1806.04195. 1806.04195 07195621 10.4171/MSL/10
[28] Polyanskiy, Y. and Wu, Y. (2017). Strong data-processing inequalities for channels and Bayesian networks. In Convexity and Concentration. IMA Vol. Math. Appl. 161 211–249. Springer, New York. 1419.60021[28] Polyanskiy, Y. and Wu, Y. (2017). Strong data-processing inequalities for channels and Bayesian networks. In Convexity and Concentration. IMA Vol. Math. Appl. 161 211–249. Springer, New York. 1419.60021
[29] Saade, A., Krzakala, F., Lelarge, M. and Zdeborová, L. (2015). Spectral detection in the censored block model. Available at arXiv:1502.00163. 1502.00163[29] Saade, A., Krzakala, F., Lelarge, M. and Zdeborová, L. (2015). Spectral detection in the censored block model. Available at arXiv:1502.00163. 1502.00163
[30] Stam, A. J. (1959). Some inequalities satisfied by the quantities of information of Fisher and Shannon. Inf. Control 2 101–112. 0085.34701 10.1016/S0019-9958(59)90348-1[30] Stam, A. J. (1959). Some inequalities satisfied by the quantities of information of Fisher and Shannon. Inf. Control 2 101–112. 0085.34701 10.1016/S0019-9958(59)90348-1
