Annals of Statistics

On the optimal reconstruction of partially observed functional data

Alois Kneip and Dominik Liebl

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text

Abstract

We propose a new reconstruction operator that aims to recover the missing parts of a function given the observed parts. This new operator belongs to a new, very large class of functional operators which includes the classical regression operators as a special case. We show the optimality of our reconstruction operator and demonstrate that the usually considered regression operators generally cannot be optimal reconstruction operators. Our estimation theory allows for autocorrelated functional data and considers the practically relevant situation in which each of the $n$ functions is observed at $m_{i}$, $i=1,\dots ,n$, discretization points. We derive rates of consistency for our nonparametric estimation procedures using a double asymptotic. For data situations, as in our real data application where $m_{i}$ is considerably smaller than $n$, we show that our functional principal components based estimator can provide better rates of convergence than conventional nonparametric smoothing methods.

Article information

Source
Ann. Statist., Volume 48, Number 3 (2020), 1692-1717.

Dates
Received: September 2018
Revised: May 2019
First available in Project Euclid: 17 July 2020

Permanent link to this document
https://projecteuclid.org/euclid.aos/1594972835

Digital Object Identifier
doi:10.1214/19-AOS1864

Mathematical Reviews number (MathSciNet)
MR4124340

Zentralblatt MATH identifier
07241608

Subjects
Primary: 62M20: Prediction [See also 60G25]; filtering [See also 60G35, 93E10, 93E11] 62H25: Factor analysis and principal components; correspondence analysis 62G05: Estimation 62G08: Nonparametric regression

Keywords
Functional data analysis functional principal components incomplete functions

Citation

Kneip, Alois; Liebl, Dominik. On the optimal reconstruction of partially observed functional data. Ann. Statist. 48 (2020), no. 3, 1692--1717. doi:10.1214/19-AOS1864. https://projecteuclid.org/euclid.aos/1594972835


Export citation

References

  • Bosq, D. (2000). Linear Processes in Function Spaces: Theory and Applications. Lecture Notes in Statistics 149. Springer, New York.
  • Cai, T. T. and Hall, P. (2006). Prediction in functional linear regression. Ann. Statist. 34 2159–2179.
  • Cardot, H., Mas, A. and Sarda, P. (2007). CLT in functional linear regression models. Probab. Theory Related Fields 138 325–361.
  • Cludius, J., Hermann, H., Matthes, F. C. and Graichen, V. (2014). The merit order effect of wind and photovoltaic electricity generation in Germany 2008–2016: Estimation and distributional implications. Energy Econ. 44 302–313.
  • Delaigle, A. and Hall, P. (2013). Classification using censored functional data. J. Amer. Statist. Assoc. 108 1269–1283.
  • Delaigle, A. and Hall, P. (2016). Approximating fragmented functional data by segments of Markov chains. Biometrika 103 779–799.
  • Delaigle, A., Hall, P., Huang, W. and Kneip, A. (2018). Estimating the covariance of fragmented and other incompletely observed functional data. Working Paper.
  • Descary, M.-H. and Panaretos, V. M. (2019). Recovering covariance from functional fragments. Biometrika 106 145–160.
  • Fanone, E., Gamba, A. and Prokopczuk, M. (2013). The case of negative day-ahead electricity prices. Energy Econ. 35 22–34.
  • Goldberg, Y., Ritov, Y. and Mandelbaum, A. (2014). Predicting the continuation of a function with applications to call center data. J. Statist. Plann. Inference 147 53–65.
  • Gromenko, O., Kokoszka, P. and Sojka, J. (2017). Evaluation of the cooling trend in the ionosphere using functional regression with incomplete curves. Ann. Appl. Stat. 11 898–918.
  • Hall, P. and Horowitz, J. L. (2007). Methodology and convergence rates for functional linear regression. Ann. Statist. 35 70–91.
  • Hall, P., Müller, H.-G. and Wang, J.-L. (2006). Properties of principal component methods for functional and longitudinal data analysis. Ann. Statist. 34 1493–1517.
  • Hirth, L. (2013). The market value of variable renewables: The effect of solar wind power variability on their relative price. Energy Econ. 38 218–236.
  • Horváth, L. and Kokoszka, P. (2012). Inference for Functional Data with Applications. Springer Series in Statistics. Springer, New York.
  • Kneip, A. and Liebl, D. (2020). Supplement to “On the optimal reconstruction of partially observed functional data.” https://doi.org/10.1214/19-AOS1864SUPP.
  • Kraus, D. (2015). Components and completion of partially observed functional data. J. R. Stat. Soc. Ser. B. Stat. Methodol. 77 777–801.
  • Li, Y. and Hsing, T. (2010). Uniform convergence rates for nonparametric regression and principal component analysis in functional/longitudinal data. Ann. Statist. 38 3321–3351.
  • Liebl, D. (2013). Modeling and forecasting electricity spot prices: A functional data perspective. Ann. Appl. Stat. 7 1562–1592.
  • Liebl, D. (2019). Nonparametric testing for differences in electricity prices: The case of the Fukushima nuclear accident. Ann. Appl. Stat. 13 1128–1146.
  • Nicolosi, M. (2010). Wind power integration and power system flexibility—an empirical analysis of extreme events in Germany under the new negative price regime. Energy Policy 38 7257–7268.
  • Ramsay, J. O. and Silverman, B. W. (2005). Functional Data Analysis, 2nd ed. Springer Series in Statistics. Springer, New York.
  • Rice, J. A. and Silverman, B. W. (1991). Estimating the mean and covariance structure nonparametrically when the data are curves. J. Roy. Statist. Soc. Ser. B 53 233–243.
  • Weigt, H. (2009). Germany’s wind energy: The potential for fossil capacity replacement and cost saving. Appl. Energy 86 1857–1863.
  • Weron, R. (2014). Electricity price forecasting: A review of the state-of-the-art with a look into the future. Int. J. Forecast. 30 1030–1081.
  • Yao, F., Müller, H.-G. and Wang, J.-L. (2005a). Functional data analysis for sparse longitudinal data. J. Amer. Statist. Assoc. 100 577–590.
  • Yao, F., Müller, H.-G. and Wang, J.-L. (2005b). Functional linear regression analysis for longitudinal data. Ann. Statist. 33 2873–2903.
  • Zhang, J.-T. and Chen, J. (2007). Statistical inferences for functional data. Ann. Statist. 35 1052–1079.
  • Zhang, X. and Wang, J.-L. (2016). From sparse to dense functional data and beyond. Ann. Statist. 44 2281–2321.

Supplemental materials

  • Supplemental paper. The supplemental paper contains the proofs of our theoretical results.