A Regression Type Problem

Abstract

Let $X_1, \cdots, X_n$ be random vectors that take values in a compact set in $R^d, d = 1, 2$. Let $Y_1, \cdots, Y_n$ be random variables (the responses) which conditionally on $X_1 = x_1, \cdots, X_n = x_n$ are independent with densities $f(y \mid x_i, \theta(x_i)), i = 1, \cdots, n$. Assuming that $\theta$ lies in a sup-norm compact space $\Theta$ of real-valued functions, an $L_1$-consistent estimator (of $\theta$) is constructed via empirical measures. The rate of convergence of the estimator to the true parameter $\theta$ depends on Kolmogorov's entropy of $\Theta$.