Electronic Journal of Statistics

Convergence and rates for fixed-interval multiple-track smoothing using $k$-means type optimization

Matthew Thorpe and Adam M. Johansen

Full-text: Open access


We address the task of estimating multiple trajectories from unlabeled data. This problem arises in many settings, one could think of the construction of maps of transport networks from passive observation of travellers, or the reconstruction of the behaviour of uncooperative vehicles from external observations, for example. There are two coupled problems. The first is a data association problem: how to map data points onto individual trajectories. The second is, given a solution to the data association problem, to estimate those trajectories. We construct estimators as a solution to a regularized variational problem (to which approximate solutions can be obtained via the simple, efficient and widespread $k$-means method) and show that, as the number of data points, $n$, increases, these estimators exhibit stable behaviour. More precisely, we show that they converge in an appropriate Sobolev space in probability and with rate $n^{-1/2}$.

Article information

Electron. J. Statist., Volume 10, Number 2 (2016), 3693-3722.

Received: October 2015
First available in Project Euclid: 3 December 2016

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G20: Asymptotic properties

Asymptotics $k$-means non-parametric regression rates of convergence variational methods


Thorpe, Matthew; Johansen, Adam M. Convergence and rates for fixed-interval multiple-track smoothing using $k$-means type optimization. Electron. J. Statist. 10 (2016), no. 2, 3693--3722. doi:10.1214/16-EJS1209. https://projecteuclid.org/euclid.ejs/1480734075

Export citation


  • [1] M. Aerts, G. Claeskens, and M. P. Wand. Some theory for penalized spline generalized additive models., Journal of Statistical Planning and Inference, 103(1-2):455–470, 2002.
  • [2] A. Antos. Improved minimax bounds on the test and training distortion of empirically designed vector quantizers., Information Theory, IEEE Transactions on, 51(11) :4022–4032, 2005.
  • [3] D. Arthur, B. Manthey, and H. Roeglin. $k$-means has polynomial smoothed complexity. In, Proceedings of the 50th Symposium on Foundations of Computer Science, 2009.
  • [4] B. Auder and A. Fischer. Projection-based curve clustering., Journal of Statistical Computation and Simulation, 82(8) :1145–1168, 2012.
  • [5] P. L. Bartlett, T. Linder, and G. Lugosi. The minimax distortion redundancy in empirical quantizer design., Information Theory, IEEE Transactions on, 44(5) :1802–1813, 1998.
  • [6] J. C. Bezdek., Pattern Recognition with Fuzzy Objective Function Algorithms. Springer, 2013.
  • [7] G. Biau, L. Devroye, and G. Lugosi. On the performance of clustering in Hilbert spaces., Information Theory, IEEE Transactions on, 54(2):781–790, 2008.
  • [8] P. S. Bradley, O. L. Mangasarian, and W. N. Street. Clustering via concave minimization. In, Advances in Neural Information Processing Systems, pages 368–374, 1997.
  • [9] A. Braides., $\Gamma$-Convergence for Beginners. Oxford University Press, 2002.
  • [10] P. A. Chou. The distortion of vector quantizers trained on $n$ vectors decreases to the optimum as $O_p(1/n)$. In, Information Theory, Proceedings of 1994 IEEE International Symposium on, page 457, 1994.
  • [11] D. D. Cox. Asymptotics for $M$-type smoothing splines., The Annals of Statistics, 11(2):530–551, 1983.
  • [12] D. D. Cox. Approximation of method of regularization estimators., The Annals of Statistics, 16(2):694–712, 1988.
  • [13] P. Craven and G. Wahba. Smoothing noisy data with spline functions., Numerische Mathematik, 31(4):377–403, 1979.
  • [14] J. A. Cuesta and C. Matran. The strong law of large numbers for $k$-means and best possible nets of Banach valued random variables., Probability Theory and Related Fields, 78(4):523–534, 1988.
  • [15] J. A. Cuesta-Albertos and R. Fraiman. Impartial trimmed $k$-means for functional data., Computational Statistics & Data Analysis, 51(10) :4864–4877, 2007.
  • [16] G. Dal Maso., An Introduction to $\Gamma$-Convergence. Springer, 1993.
  • [17] S. Dasgupta and Y. Freund. Random projection trees for vector quantization., Information Theory, IEEE Transactions on, 55(7) :3229–3242, 2009.
  • [18] R. C. de Amorim and B. Mirkin. Minkowski metric, feature weighting and anomalous cluster initializing in K-means clustering., Pattern Recognition, 45(3) :1061–1075, 2012.
  • [19] A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM Algorithm., Journal of the Royal Statistical Society, Series B, 39:2–38, 1977.
  • [20] R. M. Dudley., Real Analysis and Probability. Cambridge University Press, 2002.
  • [21] E. A. Feinberg, P. O. Kasyanov, and N. V. Zadoianchuk. Fatou’s lemma for weakly converging probabilities., Theory of Probability & Its Applications, 58(4):683–689, 2014.
  • [22] A. Fischer. Quantization and clustering with Bregman divergences., Journal of Multivariate Analysis, 101(9) :2207–2221, 2010.
  • [23] N. García Trillos and D. Slepčev. Continuum limit of Total Variation on point clouds., Archive for Rational Mechanics and Analysis, pages 1–49, 2015.
  • [24] N. García Trillos and D. Slepčev. On the rate of convergence of empirical measures in $\infty$-transportation distance., Canadian Journal of Mathematics, 67 :1358–1383, 2015.
  • [25] J. Hartigan. Asymptotic distributions for clustering criteria., The Annals of Statistics, 6(1):117–131, 1978.
  • [26] T. Laloë. $L_1$-quantization and clustering in Banach spaces., Mathematical Methods of Statistics, 19(2):136–150, 2010.
  • [27] J. Lember. On minimizing sequences for $k$-centres., Journal of Approximation Theory, 120:20–35, 2003.
  • [28] C. Levrard. Nonasymptotic bounds for vector quantization in Hilbert spaces., The Annals of Statistics, 43(2):592–619, 2015.
  • [29] K.-C. Li. Asymptotic optimality for $C_p$, $C_L$, cross-validation and generalized cross-validation: Discrete index set., The Annals of Statistics, 15(3):958–975, 1987.
  • [30] T. Linder., Principles of Nonparametric Learning, chapter Learning-Theoretic Methods in Lossy Data Compression, pages 163–210. Springer, 2002.
  • [31] T. Linder, G. Lugosi, and K. Zeger. Rates of convergence in the source coding theorem, in empirical quantizer design, and in universal lossy source coding., Information Theory, IEEE Transactions on, 40(6) :1728–1740, 1994.
  • [32] S. Lloyd. Least squares quantization in PCM., IEEE Transactions on Information Theory, 28(2):129–137, 1982.
  • [33] D. W. Nychka and D. D. Cox. Convergence rates for regularized solutions of integral equations from discrete noisy data., The Annals of Statistics, 17(2):556–572, 1989.
  • [34] D. Pollard. Strong consistency of $k$-means clustering., The Annals of Statistics, 9(1):135–140, 1981.
  • [35] D. Pollard. A central limit theorem for empirical processes., Journal of the Australian Mathematical Society (Series A), 33(2):235–248, 1982.
  • [36] D. Pollard. A central limit theorem for $k$-means clustering., The Annals of Statistics, 4(10):919–926, 1982.
  • [37] D. L. Ragozin. Error bounds for derivative estimates based on spline smoothing of exact or noisy data., Journal of Approximation Theory, 37(4):335–355, 1983.
  • [38] P. L. Speckman. Spline smoothing and optimal rates of convergence in nonparametric regression models., The Annals of Statistics, 13(3):970–983, 1985.
  • [39] P. L. Speckman and D. Sun. Asymptotic properties of smoothing parameter selection in spline smoothing. Technical report, Department of Statistics, University of Missouri, 2001.
  • [40] C. J. Stone. Optimal global rates of convergence for nonparametric regression., The Annals of Statistics, 10(4) :1040–1053, 1982.
  • [41] T. Tarpey and K. K. J. Kinateder. Clustering functional data., Journal of Classification, 2003.
  • [42] M. Thorpe, F. Theil, A. M. Johansen, and N. Cade. Convergence of the $k$-means minimization problem using $\Gamma$-convergence., SIAM Journal on Applied Mathematics, 75(6) :2444–2474, 2015.
  • [43] F. I. Utreras. Optimal smoothing of noisy data using spline functions., SIAM Journal on Scientific and Statistical Computing, 2(3):349–362, 1981.
  • [44] F. I. Utreras. Natural spline functions, their associated eigenvalue problem., Numerische Mathematik, 42(1):107–117, 1983.
  • [45] F. I. Utreras. Smoothing noisy data under monotonicity constraints existence, characterization and convergence rates., Numerische Mathematik, 47(4):611–625, 1985.
  • [46] G. Wahba., Spline models for observational data. Society for Industrial and Applied Mathematics (SIAM), 1990.
  • [47] M. P. Wand. On the optimal amount of smoothing in penalised spline regression., Biometrika, 86(4):936–940, 1999.