## Communications in Applied Mathematics and Computational Science

### On a scalable nonparametric denoising of time series signals

#### Abstract

Denoising and filtering of time series signals is a problem emerging in many areas of computational science. Here we demonstrate how the nonparametric computational methodology of the finite element method of time series analysis with $H 1$ regularization can be extended for denoising of very long and noisy time series signals. The main computational bottleneck is the inner quadratic programming problem. Analyzing the solvability and utilizing the problem structure, we suggest an adapted version of the spectral projected gradient method (SPG-QP) to resolve the problem. This approach increases the granularity of parallelization, making the proposed methodology highly suitable for graphics processing unit (GPU) computing. We demonstrate the scalability of our open-source implementation based on PETSc for the Piz Daint supercomputer of the Swiss Supercomputing Centre (CSCS) by solving large-scale data denoising problems and comparing their computational scaling and performance to the performance of the standard denoising methods.

#### Article information

Source
Commun. Appl. Math. Comput. Sci., Volume 13, Number 1 (2018), 107-138.

Dates
Received: 20 June 2017
Accepted: 30 October 2017
First available in Project Euclid: 28 March 2018

Permanent link to this document
https://projecteuclid.org/euclid.camcos/1522202441

Digital Object Identifier
doi:10.2140/camcos.2018.13.107

Mathematical Reviews number (MathSciNet)
MR3778322

Zentralblatt MATH identifier
1385.37084

#### Citation

Pospíšil, Lukáš; Gagliardini, Patrick; Sawyer, William; Horenko, Illia. On a scalable nonparametric denoising of time series signals. Commun. Appl. Math. Comput. Sci. 13 (2018), no. 1, 107--138. doi:10.2140/camcos.2018.13.107. https://projecteuclid.org/euclid.camcos/1522202441

#### References

• N. Agmon, Y. Alhassid, and R. D. Levine, An algorithm for finding the distribution of maximal entropy, J. Comput. Phys. 30 (1979), no. 2, 250–258.
• S. Balay, S. Abhyankar, M. F. Adams, J. Brown, P. Brune, K. Buschelman, L. Dalcin, V. Eijkhout, W. D. Gropp, D. Kaushik, M. G. Knepley, D. A. May, L. C. McInnes, K. Rupp, B. F. Smith, S. Zampini, H. Zhang, and H. Zhang, PETSc, users manual ANL-95/11, rev. 3.7, Argonne National Laboratory, 2016.
• J. Barzilai and J. M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal. 8 (1988), no. 1, 141–148.
• D. P. Bertsekas, Nonlinear programming, 2nd ed., Athena Scientific, 1999.
• E. G. Birgin, J. M. Martínez, and M. Raydan, Nonmonotone spectral projected gradient methods on convex sets, SIAM J. Optim. 10 (2000), no. 4, 1196–1211.
• ––––, Algorithm 813: SPG - software for convex-constrained optimization, ACM Trans. Math. Softw. 27 (2001), no. 3, 340–349.
• S. Boyd and L. Vandenberghe, Convex optimization, Cambridge University, 2004.
• E. J. Candès and M. B. Wakin, An introduction to compressive sampling, IEEE Signal Proc. Mag. 25 (2008), no. 2, 21–30.
• A.-L. Cauchy, Méthode générale pour la résolution des systèmes d'équations simultanées, Comptes Rendus Hebd. Séances Acad. Sci. 25 (1847), 536–538.
• Y. Chen and X. Ye, Projection onto a simplex, preprint, 2011.
• A. Coates, B. Huval, T. Wang, D. J. Wu, A. Y. Ng, and B. Catanzaro, Deep learning with COTS HPC systems, Proceedings of the 30th International Conference on Machine Learning (S. Dasgupta and D. McAllester, eds.), Proceedings of Machine Learning Research, no. 28, 2013, pp. 1337–1345.
• Y.-H. Dai and R. Fletcher, Projected Barzilai–Borwein methods for large-scale box-constrained quadratic programming, Numer. Math. 100 (2005), no. 1, 21–47.
• Y.-H. Dai and L.-Z. Liao, $R$-linear convergence of the Barzilai and Borwein gradient method, IMA J. Numer. Anal. 22 (2002), no. 1, 1–10.
• Z. Dostál, Optimal quadratic programming algorithms: with applications to variational inequalities, Springer Optimization and Its Applications, no. 23, Springer, 2009.
• Z. Dostál and L. Pospíšil, Minimizing quadratic functions with semidefinite Hessian subject to bound constraints, Comput. Math. Appl. 70 (2015), no. 8, 2014–2028.
• ––––, Optimal iterative QP and QPQC algorithms, Ann. Oper. Res. 243 (2016), no. 1–2, 5–18.
• W. Enders, Applied econometric time series, 4th ed., Wiley, 2015.
• G. Fourestey, B. Cumming, L. Gilly, and T. C. Schulthess, First experiences with validating and using the Cray power management database tool, Cray User Group proceedings, 2014.
• S. Gerber and I. Horenko, On inference of causality for discrete state models in a multiscale context, Proc. Natl. Acad. Sci. USA 111 (2014), no. 41, 14651–14656.
• ––––, Improving clustering by imposing network information, Sci. Adv. 1 (2015), no. 7, 1500163.
• L. Grippo, F. Lampariello, and S. Lucidi, A nonmonotone line search technique for Newton's method, SIAM J. Numer. Anal. 23 (1986), no. 4, 707–716.
• P. C. Hansen and D. P. O'Leary, The use of the $L$-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput. 14 (1993), no. 6, 1487–1503.
• V. Hapla, Massively parallel quadratic programming solvers with applications in mechanics, Ph.D. thesis, Technical University of Ostrava, 2016.
• V. Hapla, D. Horák, L. Pospíšil, M. Čermák, A. Vašatová, and R. Sojka, Solving contact mechanics problems with PERMON, High performance computing in science and engineering: second international conference (T. Kozubek, R. Blaheta, J. Šístek, M. Rozložník, and M. Čermák, eds.), Lecture Notes in Computer Science, no. 9611, Springer, 2015, pp. 101–115.
• T. J. Hastie and R. J. Tibshirani, Generalized additive models, Monographs on Statistics and Applied Probability, no. 43, Chapman and Hall, 1990.
• M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Research Nat. Bur. Standards 49 (1952), 409–436.
• I. Horenko, On simultaneous data-based dimension reduction and hidden phase identification, J. Atmos. Sci. 65 (2008), no. 6, 1941–1954.
• ––––, On robust estimation of low-frequency variability trends in discrete Markovian sequences of atmospheric circulation patterns, J. Atmos. Sci. 66 (2009), no. 7, 2059–2072.
• ––––, Finite element approach to clustering of multidimensional time series, SIAM J. Sci. Comput. 32 (2010), no. 1, 62–83.
• ––––, On clustering of non-stationary meteorological time series, Dynam. Atmos. Oceans 49 (2010), no. 2–3, 164–187.
• ––––, On the identification of nonstationary factor models and their application to atmospheric data analysis, J. Atmos. Sci. 67 (2010), no. 5, 1559–1574.
• ––––, Nonstationarity in multifactor models of discrete jump processes, memory, and application to cloud modeling, J. Atmos. Sci. 68 (2011), no. 7, 1493–1506.
• ––––, On analysis of nonstationary categorical data time series: dynamical dimension reduction, model selection, and applications to computational sociology, Multiscale Model. Simul. 9 (2011), no. 4, 1700–1726.
• D. E. King, Dlib-ml: a machine learning toolkit, J. Mach. Learn. Res. 10 (2009), 1755–1758.
• A. J. Laub, Matrix analysis for scientists and engineers, Society for Industrial and Applied Mathematics, 2005.
• F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche, and B. Arnaldi, A review of classification algorithms for EEG-based brain–computer interfaces, J. Neural Eng. 4 (2007), no. 2, R1.
• J. H. Macke, M. Opper, and M. Bethge, Common input explains higher-order correlations and entropy in a simple model of neural population activity, Phys. Rev. Lett. 106 (2011), no. 20, 208102.
• P. Metzner, L. Putzig, and I. Horenko, Analysis of persistent nonstationary time series and applications, Commun. Appl. Math. Comput. Sci. 7 (2012), no. 2, 175–229.
• M. Mudelsee, Climate time series analysis: classical statistical and bootstrap methods, 2nd ed., Atmospheric and Oceanographic Sciences Library, no. 51, Springer, 2014.
• T. Munson, J. Sarich, S. Wild, S. Benson, and L. C. McInnes, Toolkit for Advanced Optimization $($TAO$)$, users manual ANL/MCS-TM-322, rev. 3.5, Argonne National Laboratory, 2014.
• K. Murphy, Hidden Markov model $($HMM$)$ toolbox for Matlab, 2005.
• J. Nocedal and S. J. Wright, Numerical optimization, Springer, 1999.
• L. Pospíšil, Development of algorithms for solving minimizing problems with convex quadratic function on special convex sets and applications, Ph.D. thesis, Technical University of Ostrava, 2015.
• L. Pospíšil, I. Horenko, P. Gagliardini, and W. Sawyer, PASC-inference: open-source HPC library of nonparametric Bayesian causality inference methods, 2017.
• M. Raydan M., Convergence properties of the Barzilai and Borwein gradient method, Ph.D. thesis, Rice University, 1991.
• K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, preprint, 2015.
• F. Stimberg, M. Opper, G. Sanguinetti, and A. Ruttor, Inference in continuous-time change-point models, Neural Information Processing Systems Conference 2011 (J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, eds.), Advances in Neural Information Processing Systems, no. 24, 2011, pp. 2717–2725.
• L. A. Vese and C. Le Guyader, Variational methods in image processing, CRC, 2016.
• G. Wahba, Spline models for observational data, CBMS-NSF Regional Conference Series in Applied Mathematics, no. 59, Society for Industrial and Applied Mathematics, 1990.
• Y. Wiaux, L. Jacques, G. Puy, A. M. M. Scaife, and P. Vandergheynst, Compressed sensing imaging techniques for radio interferometry, Mon. Not. R. Astron. Soc. 395 (2009), no. 3, 1733–1742.
• S. J. Wright, Primal-dual interior-point methods, Society for Industrial and Applied Mathematics, 1997.