The Jensen effect and functional single index models: Estimating the ecological implications of nonlinear reaction norms

Abstract

This paper develops tools to characterize how species are affected by environmental variability, based on a functional single index model relating a response such as growth rate to environmental conditions. In ecology the curvature of such responses are used, via Jensen’s inequality, to determine whether environmental variability is harmful or beneficial, and differing nonlinear responses to environmental variability can contribute to the coexistence of competing species.

Here, we address estimation and inference for these models with observational data on individual responses to environmental conditions. Because nonparametric estimation of the curvature (second derivative) in a nonparametric functional single index model requires unrealistic sample sizes, we instead focus on directly estimating the effect of the nonlinearity by comparing the average response to a variable environment with the response at the expected environment, which we call the Jensen Effect. We develop a test statistic to assess whether this effect is significantly different from zero. In doing so we reinterpret the SiZer method of Chaudhuri and Marron (J. Amer. Statist. Assoc. 94 (1999) 807–823) by maximizing a test statistic over smoothing parameters. We show that our proposed method works well both in simulations and on real ecological data from the long-term data set described in Drake (Proc. R. Soc. Lond., B Biol. Sci. 272 (2005) 1823–1827).