The increased use of screening tests for drug use or antibodies to the HTLV-III (AIDS) virus, as well as pre-employment polygraph testing, has raised concerns about the reliability of the results of these procedures. This paper reviews the mathematical model underlying the analysis of data from screening tests. In addition to the known formulas for the proportion of positive (negative) classifications that are correct, we provide a large sample approximation to their standard errors. The results reinforce the need for confirmatory tests and indicate that moderately large sample sizes should be used to determine the accuracy rates of screening tests that will be applied to the general population in which the prevalence of the disease or trait is low.

Statistical activity can be divided for descriptive and analytical purposes into (a) discovery/imposition of structure, (b) assessment of variation conditional on structure and (c) execution of techniques. Each of these three areas of activity has an associated type of uncertainty, respectively, structural uncertainty, risk and technical uncertainty. In any statistical analysis, an analyst has limited supplies of time, money, know how and computational power and must use these resources to diminish and to characterize better the three main types of uncertainty and the many subtypes that comprise them. No existing school of statistical thinking provides a comprehensive framework for considering the various types of uncertainty and the tradeoffs among them that analysts must make. One result of this is the absence of a system that properly accounts for all of the types of uncertainty. This paper describes the types of uncertainty, catalogues and evaluates current methods as tools for characterizing and diminishing them, considers the types of tradeoffs that analysts must make in applying statistical methods in problems and examines the bias introduced into deliberations by the absence of a proper system of accounting for uncertainty. This paper is an attempt to begin the construction of such a proper system and thus to reduce or eliminate that bias.

An observational study is an attempt to estimate the effects of a treatment when subjects are not randomly assigned to treatment or control. The possibility of using more than one control group has often been briefly mentioned in general discussions of observational studies, and many observational studies have used two control groups. Here, the limited and dispersed literature on this subject is reviewed, and the topic is developed in several directions by using a formal notation for observational studies. The value of a second control group depends on the supplementary information that is available about unobserved biases that are suspected to exist. A second control group provides a test of the assumption that conventional adjustments for observed covariates suffice in estimating treatment effects. Under the best of circumstances, this test is consistent and unbiased, and its power exceeds the probability of falsely detecting a treatment effect. Indeed, under the best of circumstances, two control groups can yield consistent and unbiased estimates of bounds on the treatment effect when conventional adjustments fail. In contrast, however, in the worst of circumstances, a second control group can be of little value.

Testing of precise (point or small interval) hypotheses is reviewed, with special emphasis placed on exploring the dramatic conflict between conditional measures (Bayes factors and posterior probabilities) and the classical P-value (or observed significance level). This conflict is highlighted by finding lower bounds on the conditional measures over wide classes of priors, in normal and binomial situations, lower bounds, which are much larger than the P-value; this leads to the recommendation of several alternatives to P-values. Results are also given concerning the validity of approximating an interval null by a point null. The overall discussion features critical examination of issues such as the probability of objective testing and the possibility of testing from confidence sets.