Based on the duality between tests and confidence sets we introduce a new method to derive one-sided confidence bounds following the rejection of a null hypothesis with two-sided alternatives. This method imputes that the experimenter is only interested in confidence bounds if the null hypothesis is rejected. Furthermore, we suppose that he is only interested in the direction and a lower confidence bound concerning the distance of the true parameter value to the parameter values in the null hypothesis. If the null hypothesis is rejected, the new one-sided confidence bounds are always not worse than the corresponding bounds of the two-sided confidence interval approach. If the true parameter is far away from the null hypothesis, the new bounds tend to be nearly equal to the corresponding one-sided confidence bounds with full confidence level $1 - \alpha$. The new method will be studied and illustrated in more detail in one-parameter exponential families and location families with unimodal Lebesgue densities, and, as an example where conditional tests are available, we consider the comparison of two Poission distributions. In case of the normal distribution with unknown variance we propose among others a modification of a procedure of Hodges and Lehmann. Here it may be surprising that there exist situations where the new method yields confidence bounds exactly matching the classical one-sided confidence bounds.
"Two-Sided Tests and One-Sided Confidence Bounds." Ann. Statist. 22 (3) 1502 - 1516, September, 1994. https://doi.org/10.1214/aos/1176325639