Electronic Journal of Statistics

A Dual estimator as a tool for solving regression problems

Anatoly Gordinsky

Full-text: Open access


This paper discusses a parameter estimation method that employs an unusual estimator called the Dual estimator. For a linear regression model, we obtain two alternative estimators by subtracting or adding a certain vector to the vector of the Ordinary Least Squares Estimator (OLSE). One of them strictly dominates the latter. Moreover, under the normality assumption this estimator is unbiased, and consistent, and has significantly smaller variance than the OLSE. The use of a priori information is a universal way to choose a better alternative. An important property of the proposed method is the possibility of using the strict inequalities as a priori information. In particular, if the external information is that the L2-norm of the OLS estimate exceeds the same norm of a vector of true coefficients, one can choose a better alternative without additional parameters. If it is known that the parameter is restricted by a linear non-strict inequality, the method has a smaller Mean Squared Error than a Constrained Least Squares technique. Finally, a priori information on two possible parameter values can be successfully used for the experimental confirmation of one of two alternative theories, which is illustrated by a verification of the General Theory of Relativity based upon astronomical data.

Article information

Electron. J. Statist., Volume 7 (2013), 2372-2394.

First available in Project Euclid: 20 September 2013

Permanent link to this document

Digital Object Identifier

Mathematical Reviews number (MathSciNet)

Zentralblatt MATH identifier

Primary: 62G05: Estimation
Secondary: 62J05: Linear regression

Linear regression Dual estimator strictly dominating unbiasedness consistency advantage in variance robustness comparative analysis


Gordinsky, Anatoly. A Dual estimator as a tool for solving regression problems. Electron. J. Statist. 7 (2013), 2372--2394. doi:10.1214/13-EJS848. https://projecteuclid.org/euclid.ejs/1379686548

Export citation


  • [1] Aivazyan, S.A., Yenyukov, I.S., Meshalkin, L.D. (1985). Applied Statistics. Study of Relationships. Finansy i statistika, Moskow (in, Russian).
  • [2] Bateman, H., Erdelyi, A. (1953). Higher Transcendental Functions, Vol. 1, 2, Mc, Grow-Hill.
  • [3] Blaker, H. (1999). A Class of Shrinkage Estimators in Linear Regression. The Canadian Journal of Statistics, 27, 207–220.
  • [4] Carlin, B.P., Louis, T.A. (2008). Bayesian Methods for Data Analysis, 3rd, Ed.
  • [5] Dorugade, A.V., Kashid, D.N. (2010). Alternative Method for Choosing Ridge Parameter for Regression. Applied Mathematical Sciences, 4(9), 447–456.
  • [6] Draper, N.R., Smith, H. (1998). Applied Regression Analysis, 3rd Ed., New York, J. Wiley and Sons, Inc.
  • [7] Dyson, F.W., Eddington, A.S., Davidson, C.R. (1920). A Determination of the Deflection of Light by the Sun’s Gravitational Field, from Observations Made at the Total Eclipse of May 29. Mem. R. Astron. Soc., 220, 291–333.
  • [8] Forbes, C., Evans, M. et al. (2011). Statistical Distributions, 4th Ed., Wiley and Sons, Inc.
  • [9] Gibbons, D.G. (1981). A Simulation Study of Some Ridge Estimators. J. Amer. Statist. Assoc., 76, 131–139.
  • [10] Gnedenko, B. (1978). The Theory of Probability. Translated from Russian, MIR, Moscow.
  • [11] Goldstein, M., Wooff, D. (2007). Bayes Linear Statistics. Theory and Methods, John Wiley and, Sons.
  • [12] Gordinsky, A. et al. (2000). A New Approach to Statistic Processing of Steam Parameter Measurements in the Steam Turbine Path to Diagnose Its Condition. Proc. of the International Joint Power Generation Conference, Miami Beach, FL, July 23–26, 1–5.
  • [13] Gordinsky, A. (2010). Quasi-Estimation as a Basis for Two-Stage Solving of Regression Problem, http://arxiv.org/abs/1010.0959.
  • [14] Hadi, A.S., Ling, R.F. (1998). Some Cautionary Notes on the Use of Principal Components Regression. The American Statistical Association, 52(1).
  • [15] Hastie, T., Tibshirani, R., Friedman, J. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Ed., Springer-Verlag.
  • [16] Hoerl, A.F. (1962). Application of Ridge Analysis to Regression Problems. Chemical Engineering Progress, 58, 54–59.
  • [17] Hoerl, A.E., Kennard, R.W. (1970). Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics, 42(1).
  • [18] Horn, R.A., Johnson, C.R. (1986). Matrix Analysis, Cambridge.
  • [19] James, W., Stein, C. (1961). Estimation with Quadratic Loss. Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I, pp. 361–379, Univ. California Press, Berkeley, Calif.
  • [20] Jiang, Y.H., Smith, P.L. (2002). Understanding and Interpreting Regression Parameter Estimates in Given Contexts: A Monte Carlo Study of Characteristics of Regression and Structural Coefficients, Effect Size R Squared and Significance Level of Predictors. Annual Meeting of the American Educational Research Association (New Orleans, LA, April 1–5, 2002), 25, p.
  • [21] Jolliffe, I.T. (1982). A Note on the Use of Principal Components in Regression. Royal Statistical, Society.
  • [22] Jolliffe, I.T. (2002). Principal Component Analysis: Springer Series in Statistics, 2nd Ed., Springer, NY.
  • [23] Judge, G.G., Takayama, T. (1966). Inequality Restrictions in Regression Analysis. Journal of the American Statistical Association, 61(313), 166–181.
  • [24] Kennefick, D. (2007). Not Only Because of Theory: Dyson, Eddington and the Competing Myths of the 1919 Eclipse Expedition. 2007, http://arxiv.org/abs/0709.0685
  • [25] Knopov, P.S., Korkhin, A.S. (2012). Regression Analysis Under a Priori Parameter Restrictions. Springer, New York, Dordrecht, Heidelberg, London.
  • [26] Li, T.F., Bhoj, S. (1988). A Modified James-Stein Estimator with Application to Multiple Regression Analysis. Scandinavian Journal of Statistics, 15(1), 33–37.
  • [27] McDonald, G.C. (2009). Ridge Regression. John Wiley and Sons, WIREs Comp. Stat., 1, 93–100.
  • [28] Meada, J.L., Renau, R.A. (2010). Least Squares Problems with Inequality Constraints as Quadratic Constraints. Linear Algebra and its Applications, 432(8), 1936–1949
  • [29] Van Nostrand, R.C. (1980). A Critique of Some Ridge Regression Methods (1980): Comment Author(s): Source. Journal of the American Statistical Association, 75(369), 92–94.
  • [30] Petersen, J.H. et al. (2010). Correcting a Statistical Artifact in the Estimation of the Hubble Constant Based on Type IA Supernovae Results in a Change in Estimate of, 1.2
  • [31] Rao, C.R. (1972). Linear Statistical Inference and Its Applications, 2nd Ed., Wiley, New-York.
  • [32] Rao, C.R., Toutenburg, H. (1999). Linear Models: Least Squares and Alternatives, Springer Series in, Statistics.
  • [33] Rothenberg, T.J. (1973). Efficient Estimation with a Priori Information. New Haven and London, Yale University, Press.
  • [34] Samaniego, F.J. (2010). A Comparison of the Bayesian and Frequentist Approaches to Estimation. Springer Series in Statistics, New York, Dordrecht, Heidelberg, London.
  • [35] Schafer, C.M., Stark, P.B. (2003). Using What We Know: Inference with Physical Constraints. Statistical Problems in Particle Physics, Astrophysics, and Cosmology, SLAC, September, 8–11.
  • [36] Seber, G.A.F. (1977). Linear Regression Analysis, John Wiley and, Sons.
  • [37] Tibshirani, R. (1996). Regression Shrinkage and Selection via the Lasso. Journal of the Royal Statistical Society B, 58, 267–288.
  • [38] Yoshioka, S. (1986). Multicollinearity and Avoidance in Regression Analysis. Bihaviormetrika, 19, 103–120.