On Spectral Methods for Variance Based Sensitivity Analysis

Consider a mathematical model with a finite number of random parameters. Variance based sensitivity analysis provides a framework to characterize the contribution of the individual parameters to the total variance of the model response. We consider the spectral methods for variance based sensitivity analysis which utilize representations of square integrable random variables in a generalized polynomial chaos basis. Taking a measure theoretic point of view, we provide a rigorous and at the same time intuitive perspective on the spectral methods for variance based sensitivity analysis. Moreover, we discuss approximation errors incurred by fixing inessential random parameters, when approximating functions with generalized polynomial chaos expansions.


Introduction
The aim of this article is to provide a clear and rigorous understanding of spectral methods for variance based sensitivity analysis which employ generalized polynomial chaos expansions.Our discussion concerns square integrable functions of finitely many independent random variables.The original idea of variance based sensitivity analysis goes back to the work of I.M. Sobol in [22].Other notable subsequent papers in the field include [11,19,23].Unlike local sensitivity analysis, which uses derivative information to assess the sensitivity of a model to parameters, variance based sensitivity analysis measures the contribution of each parameter to the total variance.This is why variance based sensitivity analysis is also referred to as global sensitivity analysis.Specifically, given a function X of d random inputs, parameterized by random variables ξ 1 , . . ., ξ d , a variance based sensitivity analysis aims to quantify the contribution of each ξ i (or subcollections of ξ 1 , . . ., ξ d ) to the variance of X.In [22], the ANOVA decomposition 1 of functions of independent random variables was used to characterize a number of useful sensitivity indices.The classical numerical recipes 52 A. Alexanderian for computation of these indices involve sampling based methods which, in general, require a large number of evaluations of the function X.However, there are many applications where such function evaluations are expensive; for example, evaluating X may require solving a time-dependent partial differential equation.In such cases, sampling based methods tend to become computationally prohibitive.
On the other hand, in the recent years, the spectral methods for quantifying parametric uncertainties, which utilize generalized polynomial chaos expansions, have furnished a host of efficient techniques for analysis of uncertainties in computationally expensive mathematical models; the references [10,28,18,16,15,17,14,21,9,2] represent a small sample of the available literature in this area.The so called generalized polynomial chaos expansions are Fourier expansions in appropriately chosen multivariate orthogonal polynomial bases.The theory of polynomial chaos expansions go back to the seminal work of N. Wiener in [26] and R. Cameron and W. Martin in [7].The practical applications of polynomial chaos expansions often involve a simple special case of the general theory, where one uses a finite number of canonical random variables to parameterize uncertainties in a mathematical model.Once available, these expansions can be used to efficiently characterize the statistical properties of square integrable random variables.In particular, the variance based sensitivity indices can be computed at a negligible computational cost, once such spectral expansions are available.This important point was noted in the papers [8,25] which describe efficient numerical computation of the sensitivity indices with generalized polynomial chaos expansions.
Most of the recent papers discussing the computation of variance based sensitivity indices via generalized polynomial chaos expansions begin by a discussion of the ANOVA (Sobol) functional decomposition followed by the description of variance based sensitivity indices in terms of this decomposition; subsequently, after discussing the relation of ANOVA decompositions to polynomial chaos, they describe the computation of the indices using the polynomial chaos expansion, often through an informal argument.Finally, after all the dust has settled, one arrives at some simple expressions for the sensitivity indices in terms of polynomial chaos expansions.We choose to take a different path and consider the variance based sensitivity indices, which are defined independently of any decomposition, from a measure theoretic point of view.These sensitivity indices are defined in terms of conditional expectations of square integrable random variables.Considering the measure theoretic definition of these indices and noting the probabilistic setup of generalized polynomial chaos expansions reveal a natural mathematical point of view; moreover, this leads to a direct and intuitive way of deriving spectral representations for the conditional expectations involved and subsequently for the variance based sensitivity indices.
The variance based sensitivity indices can be used to identify model parameters that are most responsible for the model variability.Subsequently, the other, inessential, parameters may be fixed at some nominal values to reduce the dimension of the parameter space; the latter can lead to significant reductions in the computational overhead for assessing model uncertainties.Moreover, such simplifications are expected to result in negligible approximation errors.The latter point was noted for example in [24] where some useful error estimates, involving variance based sensitivity indices, were derived.We shall consider this important point and study such error estimates in the case of random variables approximated via generalized polynomial chaos expansions.
The structure of this paper is as follows.In Section 2, we list the basic notation and definitions used throughout the paper.In Section 3, we briefly describe the basics of generalized polynomial chaos expansions in the context of spectral methods for uncertainty analysis.In Section 4, which is devoted to variance based sensitivity analysis, we begin by recalling some fundamental ideas regarding conditional expectation and conditional variance and continue by describing spectral representations of the conditional expectations involved; the discussion in the section then proceeds to definitions of the variance based sensitivity indices and their computation via spectral expansions.Section 5 concerns the approximation errors incurred when inessential variables, characterized as such through a variance based sensitivity analysis, are fixed at nominal values.Finally, in Section 6, we provide some concluding remarks.

Basic notation and definitions
In what follows (Ω, F , µ) denotes a probability space.The set Ω is a sample space, F is an appropriate σ-algebra on Ω, and µ is a probability measure.A real-valued random variable U on (Ω, F , µ) is an F /B(R)-measurable mapping U : (Ω, F , µ) → (R, B(R)), where B(R) denotes the Borel σ-algebra on R. Given a random variable U on Ω we denote its expectation and variance by Denote by L 2 (Ω, F , µ) the Hilbert space of (equivalence classes of) real-valued square integrable random variables on Ω; this space is equipped with the inner product (•, Let {ξ i } i∈I be a collection of random variables on Ω, where I in an index set.We denote by σ({ξ i } i∈I ) the σ-algebra generated by {ξ i } i∈I ; recall that σ({ξ i } i∈I ) is the smallest σ-algebra on Ω with respect to which every ξ i , i ∈ I, is measurable.In the special case where we have a finite collection of random variables, {ξ i } d i=1 , we let ξ = (ξ 1 , . . ., ξ d ) T and use the short-hand notation σ(ξ) to denote the σ-algebra generated by {ξ i } d i=1 .We use F ξ to denote the distribution function [13,27] of a (real-valued) random variable ξ on (Ω, F , µ): Recall that F ξ uniquely characterizes the probability law L ξ = µ•ξ −1 of the random variable ξ.Moreover, for a function g : R → R such that g(ξ) is integrable, we have

Spectral methods for uncertainty assessment
Here we provide a brief account of generalized polynomial chaos expansions, with their practical applications in mind.In particular, we consider the the finitedimensional case involving a finite number of random variables which are used to parameterize uncertainties in a mathematical model.The spectral methods for uncertainty assessment generally utilize the spectral representation of random model observables, which are functions of a finite number of independent random variables, in a polynomial chaos basis.The main motivations to use these spectral representations include efficient sampling, efficient computation of statistical properties (e.g., mean, variance), and, more specific to our discussion, immediate access to variance based sensitivity indices.In this section, we briefly describe the generalized polynomial chaos expansions and some of the related probabilistic setup.For an in-depth coverage of the spectral methods for uncertainty assessment and the related numerical algorithms, we refer to the book [14].

Table 1 Orthogonal polynomial bases corresponding to the choice of distribution
The paper [28], which concerns the practical applications of such spectral representations in scientific computing, discusses choosing ξ i which follow distributions other than standard normal; this is motivated by the need to provide more flexibility in modeling the parametric uncertainties in physical systems.In such cases, when alternate distributions for ξ i are considered, the orthogonal polynomial basis {Ψ k (ξ)} ∞ k=0 should be chosen accordingly to achieve optimal convergence.The authors of [28] then note that the Wiener-Askey system of orthogonal polynomials can be used as a guide to choose appropriate polynomial bases which are orthogonal with respect to the distribution law of ξ i .In the latter case, the expansion in (3.1) is commonly referred to as a generalized polynomial chaos expansion.An important theoretical gap was subsequently filled in [9], where the authors provided rigorous convergence results for the generalized polynomial chaos expansions.The results in [9] cover the more general case of the generalized polynomial chaos expansions of functions in L 2 (Ω, V, µ), where V is a σ-algebra generated by a countable collection of independent random variables.
We list in Table 1 the commonly used distributions for the random variables ξ i and the associated orthogonal polynomial bases [28,14,9].We shall refer to a random variable ξ on Ω for which there exists a orthogonal polynomial basis, {ψ k (ξ)} ∞ k=1 for L 2 (Ω, σ(ξ), µ) as a basic random variable.The random variables following distribution listed in Table 1 are examples of basic random variables.
A d-variate orthogonal polynomial basis is constructed as a tensor product of the univariate orthogonal bases in each coordinate ξ i , i = 1, . . ., d.Note that it is possible to use ξ i that are independent but not necessarily identically distributed, which leads to a mixed generalized polynomial chaos basis.If we denote by ψ k (ξ) the k th order polynomial basis function in ξ then the d-variate basis functions Ψ k are given by, where α k is the multi-index associated with k th basis function Ψ k .Here α k j is a non-negative integer that specifies the order of the univariate basis polynomial in ξ j , for j = 1, . . ., d.This multi-index notation will be used extensively throughout this paper.We shall provide a concrete example of the tensor product basis construction when discussing truncated generalized polynomial chaos expansions below.In this paper, we shall focus on the case where ξ i are continuous random variables whose distributions are chosen from among those listed in Table 1.The case of discrete random variables and their associated bases can be found for example in [28,9].

The image probability space
Let ξ 1 , . . ., ξ d be a collection of independent basic random variables on (Ω, F , µ) as above, and let F ξ denote the joint distribution function of the random d-vector ) where F j is the distribution function corresponding to the j th coordinate.For any random variable U : (Ω, σ(ξ), µ) → R, B(R)), we know by Doob-Dynkin Lemma [13], that there exists a Borel function Thus, instead of working in the abstract probability space (Ω, σ(ξ), µ), it is sometimes more convenient to work in the probability space, (R d , B(R d ), F ξ (dx)).In fact, letting Θ ⊆ R d denote the support of the law of ξ, we may work instead in the image probability space (Θ, B(Θ), F ξ (dx)).
where P depends on the truncation strategy used.There are multiple ways of truncating a multivariate orthogonal polynomial basis.A common approach is truncation based on total polynomial degree.That is, given an expansion order p, one uses a truncated basis {Ψ k (ξ) : |α k | ≤ p}, where α k are the multi-indices associated with the d-variate basis functions Ψ k (ξ) as described in (3.3), and In this case, it is straightforward to show that P in (3.4) is specified by, See [14] for a construction of an indexing scheme for the multi-indices α k , k = 0, . . ., P which is convenient for computer implementations.To illustrate the tensor-product construction and the truncation strategy described above, we show the construction of a third order (p = 3) bivariate (d = 2) basis in Figure 1.We point out that in some applications such isotropic truncations may become impractical, and one needs adaptive truncations which exploit the problem structure and choose optimal polynomial orders in different coordinates.However, for simplicity of presentation, in the present work, we consider the total polynomial degree truncation strategy only.

Variance based sensitivity analysis
This section is devoted to a detailed study of variance based sensitivity analysis.Since the basic mathematical idea behind variance based sensitivity analysis relies on the concept of conditional expectation, we begin by first recalling some fundamentals regarding conditional expectation and conditional variance in Section 4.1.We then proceed by giving a basic result which enables a spectral approximation of the conditional expectation of a square integrable random variable in Section 4.2.Next, in Section 4.3, starting from the definition of variance based sensitivity indices, we provide their spectral representations in terms of generalized polynomial chaos expansions.

Conditional expectation and conditional variance
Consider an integrable random variable U on (Ω, F , µ), and consider a subσ-algebra C of F .The conditional expectation [27] of U with respect to the σ-algebra C, denoted by E [U |C] is a C-measurable function such that for every

An intuitive interpretation of the conditional expectation E [U |C
] is to view it as our best estimate of the random variable U based on the "information content" contained in the σ-algebra C. In the present paper, we consider square integrable random variables, i.e. elements of L 2 (Ω, F , µ).In this case, the Hilbert space structure allows defining the conditional expectation as orthogonal projections onto the space L 2 (Ω, C, µ).That is, given U ∈ L 2 (Ω, F , µ), the conditional expectation E [U |C] is the least-squares best approximation of U in the space L 2 (Ω, C, µ).It is also common to talk about conditional expectation with respect to a random variable.In particular, if U and V are random variables on (Ω, F , µ) we write E [U |V ] to mean E [U |σ(V )]; recall that σ(V ) denotes the σ-algebra generated by V .
We also briefly recall the idea of the conditional variance which is defined based on conditional expectation; namely, consider a random variable U ∈ L 2 (Ω, F , µ) and suppose C ⊆ F is a sub-σ-algebra, the conditional variance Var [U |C] is given by [6]: We also recall the conditional variance formula [6]: As in the case of conditional expectation, it is common to consider Var [U |V ] where V is random variable on (Ω, F , µ); in this case, Var [U |V ] is understood as Var [U |σ(V )].

Spectral approximation of conditional expectations
Consider basic random variables ξ 1 , . . ., ξ d on (Ω, F , µ).We shall, as before, work with random variables U ∈ L 2 (Ω, σ(ξ), µ), which as mentioned before can be written as U = X(ξ) for a Borel function X : R d → R. To make this dependence on ξ explicit, we refer to elements of L 2 (Ω, σ(ξ), µ) in the latter form; that is, we say X(ξ) ∈ L 2 (Ω, σ(ξ), µ) with the understanding that X is real valued Borel random variable on R d .Note that X(ξ) ∈ L 2 (Ω, σ(ξ), µ) can be expanded in the associated generalized polynomial chaos basis: Consider a fixed i ∈ {1, . . ., d} and let y be the conditional expectation y(ω) = E [X(ξ)|ξ i ] (ω).As described above y is the orthogonal projection of X(ξ) onto L 2 (Ω, σ(ξ i ), µ).Hence, in particular, y is measurable with respect to σ(ξ i ) and thus, y(ω) = Y (ξ i (ω)) for a real-valued Borel function Y .Moreover, Y (ξ i ) can be expanded in the complete orthogonal basis Now note that by the tensor product construction of the d-variate basis, the univariate basis {ψ ℓ (ξ i )} ∞ ℓ=0 is a subsequence of the multivariate basis where k(ℓ) ∈ Z * specifies the location of the ℓ th univariate basis function ψ ℓ (ξ i ) in the multivariate basis.Here Z * denotes the set of non-negative integers.Next we note that by the definition of orthogonal projection, we have (Y (ξ i ) − X(ξ), ψ ℓ (ξ i )) = 0, for all ℓ ∈ Z * .Therefore, the expansion coefficients {d ℓ } of Y in (4.3) satisfy, for all ℓ ∈ Z * , where {c k } are the spectral coefficients of X(ξ) in (4.2).That is, the coefficients {d ℓ } of Y (ξ i ) are a subset of coefficients {c k } of X(ξ).Hence, we may write Utilizing the tensor product structure of the d-variate basis (3.3) and the multi-index notation, we note that the set {k(ℓ) : ℓ ∈ Z * } which picks the univariate basis functions {ψ ℓ (ξ i )} from {Ψ k (ξ)} agrees with the set E i defined by Thus, it is possible to write the expansion of Y (ξ i ) as, Moreover, we note that the above developments can be further generalized to consider E [X(ξ)|{ξ i } i∈I ], where I is a subset4 of {1, . . ., d}.Repeating an argument similar to the one above we arrive at: where α k are multi-indices associated with the d-variate generalized polynomial chaos basis {Ψ k (ξ)} ∞ k=0 as described in (3.3).Then, for X(ξ) ∈ L 2 (Ω, σ(ξ), µ) the conditional expectation E[X(ξ)|ξ [I] ] agrees almost surely with, The above result is the main tool used in spectral approximation of variance based sensitivity indices using generalized polynomial chaos expansions.

Variance based sensitivity indices
This section is devoted to the study of the variance based sensitivity indices [22,23,11,19].We begin by the definition of the first order, second order, and total sensitivity indices in Section 4.3.1.Then, in Section 4.3.2,we derive the characterization of these indices using generalized polynomial chaos expansions.

The definition of the variance based sensitivity indices
Let X(ξ) L 2 (Ω, σ(ξ), µ).The first order (or main effect) sensitivity indices measure the effect of the i th coordinate ξ i alone on variance of the random variable X(ξ).For i ∈ {1, . . ., d}, S i is defined as follows, The second order sensitivity indices describe joint effects.Specifically, for i, j ∈ {1, . . ., d}, we define S ij to be the contribution of the interaction between ξ i and ξ j to the total variance.The mathematical definition of S ij is given by, Higher order joint sensitivity indices (e.g., S ijk ) can be defined also, but are rarely used in applications.Instead, we consider the total sensitivity index, which is another useful variance based sensitivity measure.Following [11,19], for i ∈ {1, . . ., d}, we define the total sensitivity index due to ξ i as, where ξ [−i] denotes the random vector ξ = (ξ 1 , . . ., ξ d ) with ξ i removed: in other words, with the notation of Proposition 4.1, ξ [−i] = ξ [I] with I = {1, 2, . . ., d} \ {i}.The computation of the total sensitivity indices is facilitated by the following result: Lemma 4.1.Let X(ξ) ∈ L 2 (Ω, σ(ξ), µ).Then, Proof.By the conditional variance formula (4.1), we have, Remark 4.1.The above result provides an intuitive interpretation of the meaning of a total sensitivity index.Note that the numerator in (4.6) is the total variance minus the variance of the conditional expectation E[X(ξ)|ξ [−i] ], which allows quantifying the portion of the variance due to ξ i .That is, S tot i is the total contribution of ξ i , by itself and through its interactions with other coordinates, to the variance.Remark 4.2.It is also possible to define total sensitivity indices for a subcollection {ξ i } i∈I , I ⊆ {1, . . ., d}, through, where ξ [−I] denotes the random vector ξ with coordinates {ξ i } i∈I removed.

Spectral representation of the sensitivity indices
Here we consider spectral approximation of variance based sensitivity indices introduced in the previous section.In practice, given a random variable in L 2 (Ω, σ(ξ), µ) where ξ is a vector of independent basic random variables, we use its truncated polynomial chaos expansion.Therefore, we state the results of this section for functions X(ξ) ∈ V p ⊆ L 2 (Ω, σ(ξ), µ), where which is a assumed to be a sufficiently rich approximation space.Also, we note that Ψ 0 is a constant term, and we use the convention that Ψ 0 (ξ) ≡ 1.The following result summarizes the rules for computing the variance based sensitivity indices for functions in V p .
Proof.First note that for X(ξ) ∈ V p , we have, X(ξ) = P k=0 c k Ψ k (ξ) and thus, by the orthogonality of the basis (and using the convention Ψ 0 ≡ 1), Now, the proofs of the expressions for S i and S ij follow immediately from their definition (equations (4.4) and (4.5) respectively) and Proposition 4.1 which gives spectral representations for the conditional expectations involved.As for S tot i , first note that by Proposition 4.1, we have and the expression for S tot i follows from (4.6).
Remark 4.3.Note that the index sets E i , J ij , and K i in the above result are determined by the basis of V p alone.Remark 4.5.The above result shows that computing sensitivity indices is of trivial computational cost, when a polynomial chaos expansion is available.We point out that in practical applications of spectral methods for uncertainty assessment, the major portion of the computational cost is incurred when computing the expansion coefficients themselves.This issue, which we shall not delve into in the present work, has generated a great amount of research in the recent years.In practice, there exist efficient methods of computing such expansions, albeit in cases of low to moderate parameter dimension.We refer to [14] for a coverage of various strategies for computing polynomial chaos expansions.
In what follows, we also use the notation V i to denote the total contribution of ξ i to the variance: Also, to emphasize that the quantities S tot i , V i , etc. are computed for a given random variable X(ξ), we will denote these quantities by S tot i (X), V i (X), and so on.

Fixing inessential variables and dimension reduction
Consider a function X(ξ) ∈ L 2 (Ω, σ(ξ), µ) as before.Suppose a variance based sensitivity analysis is conducted and it is found that one of the variables, say ξ i , has a very small contribution to the variance of X(ξ); that is, S tot i is "small".It is reasonable to expect that fixing ξ i at a nominal value will result in a small approximation error.The purpose of this section is to describe estimates of this approximation error.The basic idea behind the developments in this section belongs to [24].The proofs of the results given below follow in similar lines as the arguments given in [24] (where the authors use Sobol functional decompositions to represent a function of finitely many independent uniformly distributed random variables).The results presented here concern the case of random variables that belong to the space spanned by an appropriate generalized (possibly mixed) polynomial chaos basis; that is we work in V p = Span{Ψ 0 (ξ), . . ., Ψ P (ξ)} ⊆ L 2 (Ω, σ(ξ), µ), where as before ξ is a vector of independent basic random variables on Ω.
Note that ϑ can be any number in the support of the law of ξ i .By the tensorproduct construction of the basis of V p , the expansion of X(ξ) in this basis, X(ξ) = Let us note that it is straightforward to generalize Proposition 5.1 and the subsequent results for the case of fixing any subcollection of the coordinates of the random vector (ξ 1 , . . ., ξ d ) T .

Concluding remarks
In this paper, we have considered the spectral representation of variance based sensitivity indices from a measure theoretic point of view.This enables a straightforward presentation, which uses spectral representation of conditional expectations as a means of computing variance based sensitivity indices.Moreover, working in the framework of approximations via generalized polynomial chaos expansions, we considered the approximation errors incurred when fixing inessential parameters in a model.The insight gained from a variance based sensitivity analysis can guide analysis of parametric uncertainties by identifying the parameters most responsible for the variability in a mathematical model.Moreover, such analyses can guide model reduction by fixing inessential parameters.In many physical models, even though the exact value of parameters are difficult to estimate, often there exist widely used values for certain parameters.In such cases, if a variance based sensitivity analysis reveals that such parameters are not influential to model variability, one can consider using the agreed upon nominal values and direct research resources toward more accurate estimation of parameters which are more influential to model variability.
Finally, we mention that practical applications of the idea of variance based sensitivity analysis are abundant in the literature.In addition to the examples given in classical works such as [22,11,19,23,20], we also point to [25] for an application involving finite element model of a foundation, [4] for an application in ocean circulation modeling under uncertainty, and [3] for a sample application to a biochemical model with random reaction rates.In particular, the application in [4] involves observables which vary over space and time.In that case, it was observed that the balance of sensitivities to different model parameters can change significantly in the space-time domain, as dictated by the physics of the problem.

Fig 1 .
Fig 1.A bivariate tensor product basis truncated according to total polynomial degree up to three.The table on the left shows the basis functions and the corresponding multi-indices, and the figure on the right provides a visual illustration of the tensor product basis and the truncation.Note that using (3.5) with p = 3 and d = 2 we have 1+P = (3+2)!/(3!×2!) = 10.

Remark 4 . 4 .
Note that in view of Proposition 4.1 and Remark 4.2, it is straightforward to derive a spectral representation for S tot I , where I ⊆ {1, . . ., d} specifies a subcollection of the random variables ξ 1 , . . ., ξ d .