The Annals of Statistics

On the systematic and idiosyncratic volatility with large panel high-frequency data

Xin-Bing Kong

Full-text: Access denied (no subscription detected)

We're sorry, but we are unable to provide you with the full text of this article because we are not able to identify you as a subscriber. If you have a personal subscription to this journal, then please login. If you are already logged in, then you may need to update your profile to register your subscription. Read more about accessing full-text


In this paper, we separate the integrated (spot) volatility of an individual Itô process into integrated (spot) systematic and idiosyncratic volatilities, and estimate them by aggregation of local factor analysis (localization) with large-dimensional high-frequency data. We show that, when both the sampling frequency $n$ and the dimensionality $p$ go to infinity and $p\geq C\sqrt{n}$ for some constant $C$, our estimators of High dimensional Itô process; common driving process; specific driving process, integrated High dimensional Itô process, common driving process, specific driving process, systematic and idiosyncratic volatilities are $\sqrt{n}$ ($n^{1/4}$ for spot estimates) consistent, the best rate achieved in estimating the integrated (spot) volatility which is readily identified even with univariate high-frequency data. However, when $Cn^{1/4}\leq p<C\sqrt{n}$, aggregation of $n^{1/4}$-consistent local estimates of systematic and idiosyncratic volatilities results in $p$-consistent (not $\sqrt{n}$-consistent) estimates of integrated systematic and idiosyncratic volatilities. Even more interesting, when $p<Cn^{1/4}$, the integrated estimate has the same convergence rate as the spot estimate, both being $p$-consistent. This reveals a distinctive feature from aggregating local estimates in the low-dimensional high-frequency data setting. We also present estimators of the integrated (spot) idiosyncratic volatility matrices as well as their inverse matrices under some sparsity assumption. We finally present a factor-based estimator of the inverse of the spot volatility matrix. Numerical studies including the Monte Carlo experiments and real data analysis justify the performance of our estimators.

Article information

Ann. Statist., Volume 46, Number 3 (2018), 1077-1108.

Received: March 2016
Revised: January 2017
First available in Project Euclid: 3 May 2018

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62M05: Markov processes: estimation 62G20: Asymptotic properties
Secondary: 60J75: Jump processes 60G20: Generalized stochastic processes

High dimensional Itô process common driving process specific driving process


Kong, Xin-Bing. On the systematic and idiosyncratic volatility with large panel high-frequency data. Ann. Statist. 46 (2018), no. 3, 1077--1108. doi:10.1214/17-AOS1578.

Export citation


  • Aït-Sahalia, Y., Mykland, P. A. and Zhang, L. (2016). How often to sample a continuous-time process in the presence of market microstructure noise. Rev. Financ. Stud. 18 3519–416. DOI:10.1007/0-387-28359-5_1.
  • Aït-Sahalia, Y. and Xiu, D. (2016). Principal component analysis of high frequency data. NBER Working Paper No. 21584. Available at
  • Aït-Sahalia, Y. and Xiu, D. (2017). Using principal component analysis to estimate a high dimensional factor model with high-frequency data. J. Econometrics. To appear. DOI:10.1016/j.jeconom.2017.08.015.
  • Andersen, T. G., Bollerslev, T., Diebold, F. X. and Labys, P. (2003). Modeling and forecasting realized volatility. Econometrica 71 579–625.
  • Bai, J. (2003). Inferential theory for factor models of large dimensions. Econometrica 71 135–171.
  • Bai, J. and Ng, S. (2003). Determining the number of factors in approximate factor models. Econometrica 70 191–221.
  • Barndorff-Nielsen, O. E., Hansen, P. R., Lunde, A. and Shephard, N. (2008). Designing realized kernels to measure the ex post variation of equity prices in the presence of noise. Econometrica 76 1481–1536.
  • Barndorff-Nielsen, O. E. and Shephard, N. (2002). Econometric analysis of realized volatility and its use in estimating stochastic volatility models. J. R. Stat. Soc. Ser. B. Stat. Methodol. 64 253–280.
  • Bickel, P. J. and Levina, E. (2008). Covariance regularization by thresholding. Ann. Statist. 36 2577–2604.
  • Cai, T. and Liu, W. (2011). Adaptive thresholding for sparse covariance matrix estimation. J. Amer. Statist. Assoc. 106 672–684.
  • Chamberlain, G. and Rothschild, M. (1983). Arbitrage, factor structure, and mean-variance analysis on large asset markets. Econometrica 51 1281–1304.
  • Chen, S. X. and Xu, Z. (2014). On implied volatility for options—Some reasons to smile and more to correct. J. Econometrics 179 1–15.
  • Christensen, K., Kinnebrock, S. and Podolskij, M. (2010). Pre-averaging estimators of the ex-post covariance matrix in noisy diffusion models with non-synchronous data. J. Econometrics 159 116–133.
  • Davis, C. and Kahan, W. M. (1970). The rotation of eigenvectors by a perturbation. III. SIAM J. Numer. Anal. 7 1–46.
  • Fan, J., Furger, A. and Xiu, D. (2016). Incorporating global industrial classification standard into portfolio allocation: A simple factor-based large covariance matrix estimator with high-frequency data. J. Bus. Econom. Statist. 34 489–503.
  • Fan, J., Li, Y. and Yu, K. (2012). Vast volatility matrix estimation using high-frequency data for portfolio selection. J. Amer. Statist. Assoc. 107 412–428.
  • Fan, J., Liao, Y. and Mincheva, M. (2013). Large covariance estimation by thresholding principal orthogonal complements. J. R. Stat. Soc. Ser. B. Stat. Methodol. 75 603–680.
  • Jacod, J., Li, Y., Mykland, P. A., Podolskij, M., and Vetter, M. (2013). Microstructure noise in the continuous case: The pre-averaging approach. Stochastic Process. Appl. 119 2249–2276.
  • Jacod, J. and Rosenbaum, M. (2013). Quarticity and other functionals of volatility: Efficient estimation. Ann. Statist. 41 1462–1484.
  • Jacod, J. and Todorov, V. (2014). Efficient estimation of integrated volatility in presence of infinite variation jumps. Ann. Statist. 42 1029–1069.
  • Jing, B.-Y., Kong, X.-B. and Liu, Z. (2012). Modeling high-frequency financial data by pure jump processes. Ann. Statist. 40 759–784.
  • Jing, B.-Y., Kong, X.-B., Liu, Z. and Mykland, P. (2012). On the jump activity index for semimartingales. J. Econometrics 166 213–223.
  • Kim, D., Kong, X. B., Li, C. and Wang, Y. (2015). Adaptive thresholding for large volatility matrix estimation based on high-frequency financial data. Unpublished manuscript. Available at
  • Kong, X.-B. (2018). Supplement to “On the systematic and idiosyncratic volatility with large panel high-frequency data.” DOI:10.1214/17-AOS1578SUPP.
  • Kong, X.-B., Liu, Z. and Jing, B.-Y. (2015). Testing for pure-jump processes for high-frequency data. Ann. Statist. 43 847–877.
  • Kong, X. B. (2017). On the number of common factors with high-frequency data. Biometrika 104 397–410.
  • Li, J. and Chen, S. X. (2012). Two sample tests for high-dimensional covariance matrices. Ann. Statist. 40 908–940.
  • Mykland, P. A., Shephard, N. and Shephard, K. (2012). Efficient and feasible inference for the components of financial variation using blocked multipower variation. Technical report. Available at
  • Mykland, P. A. and Zhang, L. (2009). Inference for continuous semimartingales observed at high frequency. Econometrica 77 1403–1445.
  • Pelger, M. (2016). Large-dimensional factor modeling based on high-frequency observations. Available at
  • Qiu, Y. and Chen, S. X. (2012). Test for bandedness of high-dimensional covariance matrices and bandwidth estimation. Ann. Statist. 40 1285–1314.
  • Ross, S. A. (1976). The arbitrage theory of capital asset pricing. J. Econom. Theory 13 341–360.
  • Tao, M., Wang, Y. and Chen, X. (2013). Fast convergence rates in estimating large volatility matrices using high-frequency financial data. Econometric Theory 29 838–856.
  • Tao, M., Wang, Y. and Zhou, H. H. (2013). Optimal sparse volatility matrix estimation for high-dimensional Itô processes with measurement errors. Ann. Statist. 41 1816–1864.
  • Todorov, V. and Tauchen, G. (2012a). The realized Laplace transform of volatility. Econometrica 80 1105–1127.
  • Todorov, V. and Tauchen, G. (2012b). Inverse realized Laplace transforms for nonparametric volatility density estimation in jump-diffusions. J. Amer. Statist. Assoc. 107 622–635.
  • Wang, Y. and Zou, J. (2010). Vast volatility matrix estimation for high-frequency financial data. Ann. Statist. 38 943–978.
  • Zhang, L., Mykland, P. A. and Aït-Sahalia, Y. (2005). A tale of two time scales: Determining integrated volatility with noisy high-frequency data. J. Amer. Statist. Assoc. 100 1394–1411.

Supplemental materials

  • Supplement to “On the integrated systematic and idiosyncratic volatility with large panel high-frequency data”. This supplement contains the technical proof of Lemmas 3–5, which is crucial in proving Theorem 1 and Theorem 2.