In this article, we describe briefly the highlights and various accomplishments in the personal as well as the academic life of Professor Pranab Kumar Sen.

In this paper we provide a comprehensive study of statistical inference in linear and allied models which exhibit some analytic perturbations in their design and covariance matrices. We also indicate a few potential applications. In the theory of perturbations of linear operators it has been known for a long time that the so-called “singular perturbations” can have a big impact on solutions of equations involving these operators even when their size is small. It appears that so far the question of whether such undesirable phenomena can also occur in statistical models and their solutions has not been formally studied. The models considered in this article arise in the context of nonlinear models where a single parameter accounts for the nonlinearity.

We propose here a smooth estimator of the mean residual life function based on randomly censored data. This is derived by smoothing the product-limit estimator using the Chaubey-Sen technique (Chaubey and Sen (1998)). The resulting estimator does not suffer from boundary bias as is the case with standard kernel smoothing. The asymptotic properties of the estimator are investigated. We establish strong uniform consistency and asymptotic normality. This complements the work of Chaubey and Sen (1999) which considered a similar estimation procedure in the case of complete data. It is seen that the properties are similar, though technically more difficult to prove, to those in the complete data case with appropriate modifications due to censoring.

There is a substantial literature on testing for the equality of the cumulative incidence functions associated with one specific cause in a competing risks setting across several populations against specific or all alternatives. In this paper we propose an asymptotically distribution-free test when the alternative is that the incidence functions are linearly ordered, but not equal. The motivation stems from the fact that in many examples such a linear ordering seems reasonable intuitively and is borne out generally from empirical observations. These tests are more powerful when the ordering is justified. We also provide estimators of the incidence functions under this ordering constraint, derive their asymptotic properties for statistical inference purposes, and show improvements over the unrestricted estimators when the order restriction holds.

We present a general methodology for performing statistical inference on the components of a real-valued matrix parameter for which rows and columns are subject to order restrictions. The proposed estimation procedure is based on an iterative algorithm developed by Dykstra and Robertson (1982) for simple order restriction on rows and columns of a matrix. For any order restrictions on rows and columns of a matrix, sufficient conditions are derived for the algorithm to converge in a single application of row and column operations. The new algorithm is applicable to a broad collection of order restrictions. In practice, it is easy to design a study such that the sufficient conditions derived in this paper are satisfied. For instance, the sufficient conditions are satisfied in a balanced design. Using the estimation procedure developed in this article, a bootstrap test for order restrictions on rows and columns of a matrix is proposed. Computer simulations for ordinal data were performed to compare the proposed test with some existing test procedures in terms of size and power. The new methodology is illustrated by applying it to a set of ordinal data obtained from a toxicological study.

We explore conditions for a class of functions defined via an integral representation to be a probability generating function of some positive integer valued random variable. Interest in and research on this question is motivated by an apparently surprising connection between a family of classic shock models due to Esary et. al. (1973) and the negatively aging nonparametric notion of “strongly decreasing failure rate” (SDFR) introduced by Bhattacharjee (2005). A counterexample shows that there exist probability generating functions with our integral representation which are not discrete SDFR, but when used as shock resistance probabilities can give rise to a SDFR survival distribution in continuous time.

Power series distributions form a useful subclass of one-parameter discrete exponential families suitable for modeling count data. A zero-inflated power series distribution is a mixture of a power series distribution and a degenerate distribution at zero, with a mixing probability p for the degenerate distribution. This distribution is useful for modeling count data that may have extra zeros. One question is whether the mixture model can be reduced to the power series portion, corresponding to p=0, or whether there are so many zeros in the data that zero inflation relative to the pure power series distribution must be included in the model i.e., p≥0. The problem is difficult partially because p=0 is a boundary point.

Here, we present a Bayesian test for this problem based on recognizing that the parameter space can be expanded to allow p to be negative. Negative values of p are inconsistent with the interpretation of p as a mixing probability, however, they index distributions that are physically and probabilistically meaningful. We compare our Bayesian solution to two standard frequentist testing procedures and find that using a posterior probability as a test statistic has slightly higher power on the most important ranges of the sample size n and parameter values than the score test and likelihood ratio test in simulations. Our method also performs well on three real data sets.

In recent years, multiple hypothesis testing has come to the forefront of statistical research, ostensibly in relation to applications in genomics and some other emerging fields. The false discovery rate (FDR) and its variants provide very important notions of errors in this context comparable to the role of error probabilities in classical testing problems. Accurate estimation of positive FDR (pFDR), a variant of the FDR, is essential in assessing and controlling this measure. In a recent paper, the authors proposed a model-based nonparametric Bayesian method of estimation of the pFDR function. In particular, the density of p-values was modeled as a mixture of decreasing beta densities and an appropriate Dirichlet process was considered as a prior on the mixing measure. The resulting procedure was shown to work well in simulations. In this paper, we provide some theoretical results in support of the beta mixture model for the density of p-values, and show that, under appropriate conditions, the resulting posterior is consistent as the number of hypotheses grows to infinity.

The paper proposes some robust estimators of the finite population mean. Such estimators are particularly suitable in the presence of some outlying observations. Included as special cases of our general result are robust versions of the ratio estimator and the Horvitz-Thompson estimator. The robust estimators are derived on the basis of certain predictive influence functions.

This paper considers a wide family of semiparametric repeated measures regression models, in which the main interest is on estimating population-level quantities such as mean, variance, probabilities etc. Examples of our framework include generalized linear models for clustered/longitudinal data, among many others. We derive plug-in kernel-based estimators of the population level quantities and derive their asymptotic distribution. An example involving estimation of the survival function of hemoglobin measures in the Kenya hemoglobin study data is presented to demonstrate our methodology.

A rank-based test of the null hypothesis that a regressor has no effect on a response variable is proposed and analyzed. This test is identical in structure to the order selection test but with the raw data replaced by ranks. The test is nonparametric in that it is consistent against virtually any smooth alternative, and is completely distribution free for all sample sizes. The asymptotic distribution of the rank-based order selection statistic is obtained and seen to be the same as that of its raw data counterpart. Exact small sample critical values of the test statistic are provided as well. It is shown that the Pitman-Noether efficiency of the proposed rank test compares very favorably with that of the order selection test. In fact, their asymptotic relative efficiency is identical to that of the Wilcoxon signed rank and t-tests. An example involving microarray data illustrates the usefulness of the rank test in practice.

Nonparametric or distribution-free charts can be useful in statistical process control problems when there is limited or lack of knowledge about the underlying process distribution. In this paper, a phase II Shewhart-type chart is considered for location, based on reference data from phase I analysis and the well-known Mann-Whitney statistic. Control limits are computed using Lugannani-Rice-saddlepoint, Edgeworth, and other approximations along with Monte Carlo estimation. The derivations take account of estimation and the dependence from the use of a reference sample. An illustrative numerical example is presented. The in-control performance of the proposed chart is shown to be much superior to the classical Shewhart X̄ chart. Further comparisons on the basis of some percentiles of the out-of-control conditional run length distribution and the unconditional out-of-control ARL show that the proposed chart is almost as good as the Shewhart X̄ chart for the normal distribution, but is more powerful for a heavy-tailed distribution such as the Laplace, or for a skewed distribution such as the Gamma. Interactive software, enabling a complete implementation of the chart, is made available on a website.

Consider the nonlinear regression model

Y_i=g(x_i, θ)+e_i, i=1, …, n

with x_i∈ℝ^k, θ=(θ₀, θ₁, …, θ_p)^′∈Θ (compact in ℝ^p+1), where g(x, θ)=θ₀+g̃(x, θ₁, …, θ_p) is continuous, twice differentiable in θ and monotone in components of θ. Following Gutenbrunner and Jurečková (1992) and Jurečková and Procházka (1994), we introduce regression rank scores for model (1), and prove their asymptotic properties under some regularity conditions. As an application, we propose some tests in nonlinear regression models with nuisance parameters.

We extend to rank-based tests of multivariate independence the Chernoff-Savage and Hodges-Lehmann classical univariate results. More precisely, we show that the Taskinen, Kankainen and Oja (2004) normal-score rank test for multivariate independence uniformly dominates – in the Pitman sense – the classical Wilks (1935) test, which establishes the Pitman non-admissibility of the latter, and provide, for any fixed space dimensions p, q of the marginals, the lower bound for the asymptotic relative efficiency, still with respect to Wilks’ test, of the Wilcoxon version of the same.

We consider a test for the hypothesis that the within-treatment variance component in a one-way random effects model is null. This test is based on a decomposition of a U-statistic. Its asymptotic null distribution is derived under the mild regularity condition that the second moment of the random effects and the fourth moment of the within-treatment errors are finite. Under the additional assumption that the fourth moment of the random effect is finite, we also derive the distribution of the proposed U-test statistic under a sequence of local alternative hypotheses. We report the results of a simulation study conducted to compare the performance of the U-test with that of the usual F-test. The main conclusions of the simulation study are that (i) under normality or under moderate degrees of imbalance in the design, the F-test behaves well when compared to the U-test, and (ii) when the distribution of the random effects and within-treatment errors are nonnormal, the U-test is preferable even when the number of treatments is small.

In the spirit of modeling inference for microarrays as multiple testing for sparse mixtures, we present a similar approach to a simplified version of quantitative trait loci (QTL) mapping. Unlike in case of microarrays, where the number of tests usually reaches tens of thousands, the number of tests performed in scans for QTL usually does not exceed several hundreds. However, in typical cases, the sparsity p of significant alternatives for QTL mapping is in the same range as for microarrays. For methodological interest, as well as some related applications, we also consider non-sparse mixtures. Using simulations as well as theoretical observations we study false discovery rate (FDR), power and misclassification probability for the Benjamini-Hochberg (BH) procedure and its modifications, as well as for various parametric and nonparametric Bayes and Parametric Empirical Bayes procedures. Our results confirm the observation of Genovese and Wasserman (2002) that for small p the misclassification error of BH is close to optimal in the sense of attaining the Bayes oracle. This property is shared by some of the considered Bayes testing rules, which in general perform better than BH for large or moderate p’s.

The Simes inequality has received considerable attention recently because of its close connection to some important multiple hypothesis testing procedures. We revisit in this article an old result on this inequality to clarify and strengthen it and a recently proposed generalization of it to offer an alternative simpler proof.

While multiple testing procedures have been the focus of much statistical research, an important facet of the problem is how to deal with possible confounding. Procedures have been developed by authors in genetics and statistics. In this chapter, we relate these proposals. We propose two new multiple testing approaches within this framework. The first combines sensitivity analysis methods with false discovery rate estimation procedures. The second involves construction of shrinkage estimators that utilize the mixture model for multiple testing. The procedures are illustrated with applications to a gene expression profiling experiment in prostate cancer.

When conducting a paired 2 × 2 crossover design, each subject is paired with another subject with similar characteristics. The pair is then randomized to the same sequence of two treatments. That is, the two subjects receive the first experimental treatment, and then they cross over and receive the other experimental treatment(s). The paired 2 × 2 crossover design that was used in the Beta Adrenergic Response by GEnotype (BARGE) Study conducted by the National Heart, Lung and Blood Institute’s Asthma Clinical Research Network (ACRN) has been described elsewhere. When the data arising from such a design are balanced and complete – or if at least any missingness that occurs is at random – general linear mixed-effects model methods can be used to analyze the data. In this paper, we present a method based on a pattern-mixture model for analyzing the data arising from a paired 2 × 2 crossover design when some of the data are missing in a non-ignorable fashion. Because of its inherent scientific interest, we focus our particular attention on the estimation of the treatment-by-type of subject interaction term. Finally, we illustrate the pattern-mixture model methods described in this paper on the data arising from the BARGE study.

This paper develops a test for homogeneity in finite mixture models where the mixing proportions are known a priori (taken to be 0.5) and a common nuisance parameter is present. Statistical tests based on the notion of Projected Likelihood Contrasts (PLC) are considered. The PLC is a slight modification of the usual likelihood ratio statistic or the Wilk’s Λ and is similar in spirit to the Rao’s score test. Theoretical investigations have been carried out to understand the large sample statistical properties of these tests. Simulation studies have been carried out to understand the behavior of the null distribution of the PLC statistic in the case of Gaussian mixtures with unknown means (common variance as nuisance parameter) and unknown variances (common mean as nuisance parameter). The results are in conformity with the theoretical results obtained. Power functions of these tests have been evaluated based on simulations from Gaussian mixtures.

The goal of this paper is to study the bootstrap for the Grenander estimator. The first result is a proof of the inconsistency of the nonparametric bootstrap for the Grenander estimator at a given point. The second result is the development and verification of a bootstrap for the L₁ confidence band for the Grenander estimator. As part of this work, kernel estimators are studied as alternatives to the Grenander estimator. We show that when the second derivative of the true density is assumed to be uniformly bounded, there exist kernel estimators with faster convergence rates than the Grenander estimator. We study the implications of this in developing L₁ and uniform confidence bands and discuss some open questions.

We propose new tests to detect a change in the mean of a time series. Like many existing tests, the new ones are based on the CUSUM process. Existing CUSUM tests require an estimator of a scale parameter to make them asymptotically distribution free under the no change null hypothesis. Even if the observations are independent, the estimation of the scale parameter is not simple since the estimator for the scale parameter should be at least consistent under the null as well as under the alternative. The situation is much more complicated in case of dependent data, where the empirical spectral density at 0 is used to scale the CUSUM process. To circumvent these difficulties, new tests are proposed which are ratios of CUSUM functionals. We demonstrate the applicability of our method to detect a change in the mean when the errors are AR(1) and GARCH(1, 1) sequences.

Statistical models incorporating change points are common in practice, especially in the area of biomedicine. This approach is appealing in that a specific parameter is introduced to account for the abrupt change in the response variable relating to a particular independent variable of interest. The statistical challenge one encounters is that the likelihood function is not differentiable with respect to this change point parameter. Consequently, the conventional asymptotic properties for the maximum likelihood estimators fail to hold in this situation. In this paper, we propose an estimating procedure for estimating the change point along with other regression coefficients under the generalized linear model framework. We show that the proposed estimators enjoy the conventional asymptotic properties including consistency and normality. Simulation work we conducted suggests that it performs well for the situations considered. We applied the proposed method to a case-control study aimed to examine the relationship between the risk of myocardial infarction and alcohol intake.

We discuss the use of multivariate kernel smoothing methods to date manuscripts dating from the 11th to the 15th centuries, in the English county of Essex. The dataset consists of some 3300 dated and 5000 undated manuscripts, and the former are used as a training sample for imputing dates for the latter. It is assumed that two manuscripts that are “close”, in a sense that may be defined by a vector of measures of distance for documents, will have close dates. Using this approach, statistical ideas are used to assess “similarity”, by smoothing among distance measures, and thus to estimate dates for the 5000 undated manuscripts by reference to the dated ones.

One of Pranab K. Sen’s major research areas is sequential nonparametrics and semiparametrics and their applications to clinical trials, to which he has made many important contributions. Herein we review a number of these contributions and related developments. We also describe some recent work on nonparametric and semiparametric inference and the associated computational methods in time-sequential clinical trials with survival endpoints.

Nonparametric estimators of the mean total cost have been proposed in a variety of settings. In clinical trials it is generally impractical to follow up patients until all have responded, and therefore censoring of patient outcomes and total cost will occur in practice. We describe a general longitudinal framework in which costs emanate from two streams, during sojourn in health states and in transition from one health state to another. We consider estimation of net present value for expenditures incurred over a finite time horizon from medical cost data that might be incompletely ascertained in some patients. Because patient specific demographic and clinical characteristics would influence total cost, we use a regression model to incorporate covariates. We discuss similarities and differences between our net present value estimator and other widely used estimators of total medical costs. Our model can accommodate heteroscedasticity, skewness and censoring in cost data and provides a flexible approach to analyses of health care cost.

There is increasing interest in the use of diagnostic rules based on microarray data. These rules are formed by considering the expression levels of thousands of genes in tissue samples taken on patients of known classification with respect to a number of classes, representing, say, disease status or treatment strategy. As the final versions of these rules are usually based on a small subset of the available genes, there is a selection bias that has to be corrected for in the estimation of the associated error rates. We consider the problem using cross-validation. In particular, we present explicit formulae that are useful in explaining the layers of validation that have to be performed in order to avoid improperly cross-validated estimates.

An important parameter in the study of population evolution is θ=4Nν, where N is the effective population size and ν is the rate of mutation per locus per generation. Therefore, θ represents the mean number of mutations per site per generation. There are many estimators of θ, one of them being the mean number of pairwise nucleotide differences, which we call T₂. Other estimators are T₁, based on the number of segregating sites and T₃, based on the number of singletons. The concept of selective neutrality can be interpreted as a differentiated nucleotide distribution for mutant sites when compared to the overall nucleotide distribution. Tajima (1989) has proposed the so-called Tajima’s test of selective neutrality based on T₂−T₁. Its complex empirical behavior (Kiihl, 2005) motivates us to propose a test statistic solely based on T₂. We are thus able to prove asymptotic normality under different assumptions on the number of sequences and number of sites via U-statistics theory.

In this article we propose a maximal a posteriori (MAP) criterion for model selection in the motif discovery problem and investigate conditions under which the MAP asymptotically gives a correct prediction of model size. We also investigate robustness of the MAP to prior specification and provide guidelines for choosing prior hyper-parameters for motif models based on sensitivity considerations.