Approximative solutions of best choice problems

We consider the full information best choice problem from a sequence $X_1,\dots, X_n$ of independent random variables. Under the basic assumption of convergence of the corresponding imbedded point processes in the plane to a Poisson process we establish that the optimal choice problem can be approximated by the optimal choice problem in the limiting Poisson process. This allows to derive approximations to the optimal choice probability and also to determine approximatively optimal stopping times. An extension of this result to the best $m$-choice problem is also given.


Introduction
The best choice problem of a random sequence X 1 , . . ., X n is to find a stopping time τ n to maximize the best choice probability P (X τ = M n ), where M n = max 1 i n X i , under all stopping times τ n.Thus we aim to find an optimal stopping time T n n such that over all stopping times τ n.
Gilbert and Mosteller (1966) [9] found the solution of the full information best choice problem, where X 1 , . . ., X n are iid with known continuous distribution function F .In this case the optimal stopping time is given by where b 0 = 0, i j=1 i j (b −1 i − 1) j j −1 = 1, i = 1, 2, . . .The asymptotic behaviour of b i is described by b i ↑ 1, i(1 − b i ) → c = 0.8043 . . .The optimal probability v n = P (X Tn = M n ) does not depend on F , is strictly decreasing in n and has limiting value v ∞ v n → v ∞ = 0.580164 . . .
A continuous time version of the problem with random number of points given by a homogeneous Poisson process with intensity λ was studied in Sakaguchi (1976) [21].
The approach in Gnedin (1996) [12] was extended in [KR] 1 (2000c) [16] to the best choice problem for (inhomogeneous) discounted sequences X i = c i Y i , where (Y i ) are iid and c i are constants which imply convergence of imbedded normalized point processes ) to some Poisson process N in the plane.The proofs in that paper make use of Gnedin's (1966) [12] result as well as of some general approximation results in [KR] (2000a) [15].The aim of this paper is to extend this approach to general inhomogeneous best choice problems for independent sequences under the basic assumption of convergence of the imbedded point processes N n to some Poisson process N in the plane.Subsequently we also consider an extension to the m-choice problem, where m choices described by stopping times 1 T 1 < • • • < T m n are allowed and the aim is to find m stopping times T m the sup being over all stopping times T 1 < • • • < T m n.
For the corresponding generalized Moser problem of maximizing E X τ resp.E m i=1 X τi a general approximation approach has been developed in [KR] (2000a), [FR] (2011) [7] for m = 1, resp. in [KR] (2002) [18] and [FR] (2011) [6] for m 1; see also Goldstein and Samuel-Cahn (2006) [14].For a detailed history of this problem we refer to Ferguson (2007) [8] for m = 1 resp.to [FR] (2011) [6] in case m 1.Our results for (1.3) are in particular applicable to sequences X i = c i Z i + d i with iid random sequences (Z i ) and with discount and observation factors c i , d i .The corresponding results for the Moser type problems for these sequences can be found in [FR] (2011) [6,7].

Approximative optimal best choice solution
We consider the optimal best choice problem (1.1) for a sequence (X i ) of independent random variables, i.e. to find optimal stopping times T n n such that an is a normalization of X i typically induced from a form of the central limit theorem for maxima where a n > 0, b n ∈ R and c ∈ R ∪ {−∞}.We consider Poisson processes N restricted on M c which may have infinite intensity along the lower boundary [0, 1] × {c}.We assume that the intensity measure µ of N is a Radon measure on M c with the relative topology induced by the usual topology on [0, 1] × R and with µ([0, 1] × {∞}) = 0. Thus any subset of M c bounded away from the lower boundary has only finitely many points of N and N has no points with value ∞.  [15,17] or [FR] (2011) [7].The typical examples from extreme value theory concern the case c = −∞ and c = 0. We generally assume that the intensity measure µ is Lebesgue-continuous with density denoted as f (t, x).Thus the Poisson process does not have multiple points.
We consider also the best choice problem for the limiting continuous time Poisson process N .An N -stopping time T : Ω → [0, 1] is a stopping time w.r.t. the filtration For stopping problems with guarantee value x ≥ c we define the stopping value Y t = x for T = 1 which corresponds to replacing all random variables X n j by X n j ∨ x in the point process N n .
An N -stopping time T is called 'optimal best choice stopping time' for N if the supremum being taken over all N -stopping times S.This definition of optimality implies that only those stopping times are of interest, which stop in some time point τ k of N.
In the following theorem we derive the optimal stopping time T for the continuous time best choice problem for the Poisson process N .Further we show that the best choice problem for X 1 , . . ., X n is approximated by the best choice problem for N .This allows us to get approximations of the best choice probabilities v n = P (X Tn = M n ) and to construct asymptotically optimal best choice stopping sequences Tn .Our approxima- tion result needs the following intensity condition, which is not necessary, when dealing only with the limiting best choice problem (see Section 3).(a) The optimal best choice stopping time T for N is given by
(b) Approximation of the optimal best choice probabilities holds true: Proof.For (t, x) ∈ [0, 1) × [c, ∞) we consider the stopping problem after time tn with guarantee value x.We want to determine optimal stopping times T n (t, x) > tn for X n j , under all stopping times τ > tn.For the proof of convergence properties we consider the stopping behaviour after time tn here (and not only after time zero) in order to make use of the recursive structure of the optimal stopping times.Define for i > tn Then we have to maximize EZ n τ (t, x) = P X n τ = x ∨ max tn<j n X n j .By the classical recursive equation for optimal stopping of finite sequences the optimal stopping times T n (t, x) are given by (see e.g.Proposition 2.1 in [FR] (2011) [6]) T n (t, x) := T >tn n (t, x) where T >n n (t, x) := n and By backward induction in l = n − 1, . . ., tn we obtain for tn < i (2.8) h n is monotonically nondecreasing in (t, x).As consequence we obtain from point process convergence that h n converges uniformly in compact sets in To prove that g n (t, x) also converges uniformly on compact sets in [0, 1] × (c, ∞] we decompose g n into two monotone components. (2.9) By assumption the pointprocess N has only finitely many points in any subset bounded away from the lower boundary c.In consequence point process convergence N n d → N implies that gn (t, x) converges pointwise for all t, x to a limit g∞ (t, x).A detailed argument for this convergence has been given for the related case of maximizing [5,Satz 4.1].This argument transfers to the optimal choice case in a similar way.Using monotonicity properties of gn (t, x) in t, x we next prove that g∞ is continuous which then implies uniform convergence.On one hand side we have for s < t and x > c gn (s, x) sup τ >sn On the other hand gn (s, x) sup τ >sn This implies and thus using that x > c continuity of g∞ (t, x) in t.
To prove continuity of g∞ in x let x < y.Then 0 gn (t, y) − gn (t, x) In consequence g∞ (t, x) is continuous and g n → g ∞ uniformly on compact subsets of [0, 1] × (c, ∞).Point process convergence and the representation in (2.8) imply that The argument above also applies to the modified random variables Defining g ∞ , h ∞ as above we obtain in this case g n (t, x) ↓ g ∞ (t, x) since the discrete stopping problems majorize the continuous time stopping problem.In consequence T (t, x) are the optimal stopping times for N , i.e.
Details of this approximation argument are given in [KR] (2000a) (proof of Theorem 2.5).As a result we get the following estimate with the stopping times Thus the limit g ∞ (t, x) is the same for (X n i ) and for ( Xn i ).
The arguments above in the case x > c can also be extended to the case x = c.Note that for x > c From the intensity assumption (I) and since h ∞ (t, c) = 0 we obtain By assumption (I) We next prove that v is monotonically nonincreasing.Since g ∞ (t, x) − h ∞ (t, x) is monotonically nonincreasing in x for any t it is sufficient to show that for s < t To that aim we get for s < t The first inequality results from definition of g ∞ (s, x), the second inequality is immediate and the third inequality is a consequence from independence of the (Y k ).On the other hand Building the difference of these (in-)equalities and using (2.10) we obtain for x = v(t) the claim in (2.11).

Remark (triangular sequences)
As remarked by a reviewer the proof of Theorem 2.1 extends in the same way to the optimal stopping of triangular sequences (X n i ) and point process convergence as in (2.2).The asymptotically optimal sequence of stopping times in Theorem 2.1, (c) then is given by (2.13) In general the optimal best choice probability s 1 in (2.7) has to be evaluated numerically.Certain classes of intensity functions however allow an explicit evaluation.In consequence v solves the equation where d = 0.8043522 . . . is the unique solution of (2.16) The asymptotic optimal choice probability can be obtained from (2.7) by some calculation as e −y y dy = 0.5801642 . . .This is identical to the asymptotic optimal choice probability in the iid case (see In particular in case of the three extreme value distribution types Λ, Φ α , and Ψ α , one gets for the limiting Poisson processes intensities with densities f (t, x) of the form Thus, these cases fit the form in (2.14).Also the example of a best choice problem for X i = c i Y i for some iid sequence dealt with in [KR] (2000c) fits this condition.

Poisson processes with finite intensities
Gnedin and Sakaguchi (1992) [13] considered the best choice problem for iid sequence (Y k ) with distribution functions F arriving at Poisson distributed time points.For continuous F this can be described by a planar Poisson process N with density f (t, y) = a(t)F (y) where a : [0, 1] → [0, ∞] is continuous integrable.N does not fulfill the infinite intensity condition (I) in Section 2. For the best choice problem in Poisson processes our method of proof can be modified to deal also with the case of finite intensity.
Let N = k = δ τ k ,Y k be a Poisson process on [0, 1] × (c, ∞] with finite Lebesgue continuous intensity measure, i.e. µ satisfies Then the following modifications of the proof of Theorem 2.1 allow to solve this case. Note that under condition (I f ) no longer h ∞ (t, c) = 0 and thus in general no longer one can find to any t ∈ [0, 1) an x > c such that h ∞ (t, x) < g ∞ (t, x).This property holds true only in [0, t 0 ) with This can be seen as follows: For t ∈ [0, t 0 ) and for x close to c holds So far we have obtained optimality of the stopping times T (t, x) for x > c.We next consider the case x = c.Since N has only finitely many points in [0 then implies optimality of T (t, c) and we obtain the following result.Theorem 3.1.Let the Poisson process satisfy the finite intensity condition (I f ).Then the optimal choice stopping time for N is given by v is monotonically nonincreasing and can be chosen right continuous.The optimal choice probability is given by e −µ((r,1]×(y,∞]) µ(dr, dy) e −µ((r,1]×(y,∞]) µ(dr, dy)µ(dr , dy ).
Example 3.2.In case of the finite intensity measure µ with density f (t, y) = a(t)F (y) as in Gnedin and Sakaguchi (1992) [13] let F (c) = 0 and A(t) := Some detailed calculations yield in this case the optimal choice probability s as This coincides with the results obtained in Gnedin and Sakaguchi (1992) [13].

The optimal m-choice problem
In this section we consider the optimal m-choice problem (1.4) for independent sequences (X i ). ) optimal m-choice stopping times.The condition t T The following lemma gives a characterization of optimal m-choice stopping times in the discrete case.
with optimal stopping times given by for 2 l m.
Proof.Let tn < S n − m + 1 be a stopping time and Z max tn<j S X n j be a random variable.Furthermore let for 1 i n, M n i be σ(X n i+1 , . . ., X n n ) measurable with M n i X n i+1 ∨ M n i+1 and P (M n i = x) = 0 for all x > c.In order to maximize P (Z ∨ X n T ∨ M n T = x ∨ max tn<j n X n j ) w.r.t.all stopping times T , S < T n − m + 1 we define Thus we have to maximize EY T = P (Z ∨ X n T ∨ M n T = x ∨ max tn<j n X n j ).The optimal stopping times are given by T n (t, x) = T >tn n (t, x) with For the second equality we use that P (X n i = X n j c) = 0 for i = j and thus X n i = x ∨ max tn<j i X n j > x is strictly larger than Z.Thus Z can not be the maximum.In consequence we get for k S where ĥn (t, x) .
By induction we obtain from (4.3) the representation in (4.2).Define for m = 1: a) The optimal m-choice stopping times for N are given by T m 1 (0, c), . . ., T m m (0, c), where for 2 l m.The thresholds v m (t) are solutions of the equations for 2 l m are approximative optimal m-choice stopping times, i.e.
Proof.The proof of b), c) is similar to the corresponding part in Theorem 2.1.We therefore concentrate on the proof of a).As in the proof of Theorem 2.1 we obtain for x = v m (t).For m = 2 we obtain from (4.7) which implies (4.8).For m 3 (4.8) is equivalent to We prove (4.9) by induction.By induction hypothesis we assume that v m−1 v m−2 . . .v 1 and all of the v i are monotonically nonincreasing.Also we have lim and since v m−1 is monotonically nonincreasing, there exists some t ∈ [s, 1) with v m (t) > v m−1 (t) and v m (t) v m (r) for all r t.We establish (4.9) for x = v m (t).
From the definition of the thresholds follows with the stopping times Thus (4.9) holds true for x = v m (t) and, therefore, v m (t) v m−1 (t), a contradiction.This implies the statement v m v m−1 .
To calculate the optimal thresholds we need to calculate the densities of record stopping times with general threshold v.This allows to calculate the optimal threshold v m .Finally we can calculate g m ∞ (t, x) for x < v m (t) using that h m ∞ (s, y)e −µ((t,s]×(x,∞]) f (t, y)dyds.
Here F m (t,x) is the density of (T m 1 (t, x), Y T m 1 (t, x))χ {T m 1 (t,x)<1} from Lemma 4.3 involving the optimal thresholds v m .For certain densities f (t, x) the optimal stopping curves can be determined directly.
In the following theorem we given an application to the case where the intensity measure µ of the Poisson process N has a density of the form f (t, y) = −a(t)F (y) with a continuous integrable function a : [0, 1] → [0, ∞] not identical zero in any neighbourhood of 1 and with a monotonically nonincreasing function F : [c, ∞] → R, continuous with lim x↓c F (x) = ∞ and F (∞) = 0.
In this situation we get the following result for the optimal m-choice problem for N .
Theorem 4.4.Under the conditions stated above the optimal threshold v m (t ) and the optimal m-choice probability is given by Proof.At first let v(t), t ∈ [0, 1) be a solution of where d > 0 and T (t, x) is the record stopping time associated to v. Let v : [0, 1] → [c, ∞) be chosen right continuous.To calculate F (t,x) the density of (T (t, x), Y T (t,x) ) in case x < v(t) we have to calculate x e µ((t,s]×(x,z]) µ(dr, dz).
With the substitution z = (A(t) − A(s))z this is identical to .
In consequence also in this case for s ∈ (t, 1) and y ∈ In the last equalities we used the substitutions z = A(s)F (y) resp.y = A(s)F (x).Thus for x v m−1 (t) we have h m ∞ (t, x) = H m (A(t)F (x)).Similarly in the case x < v m−1 (t): In the last equalities we used the substitutions z = A(t)F (y) resp.y = A(s) A(t) .Thus for x < v m−1 (t) (and, therefore, A(t)F (x) > d m−1 ) holds h m ∞ (t, x) = H m (A(t)F (x)).
Thus for x v m (t) we have Equalizing this with h m ∞ (t, x) = H m (A(t)F (x)) it follow that the optimal threshold v m (t)  This implies the optimal one-choice probability  Numerical calculation yields the optimal two-choice probability s 2 = H 3 (∞) = 0.8443 . . .
The functions H m , m 4 seem to be too difficult to be calculated explicitly and the corresponding optimal m-choice probability can only be calculated numerically (and even that is a challenge).
Convergence in distribution 'N n d → N on M c ' means convergence in distribution of the point processes restricted on M c .This convergence is defined on the Polish space of locally finite point measures supplied with the topology of vague convergence.Since the Poisson limit N has no points with infinite value this definition prevents that records of N n may disappear to infinity, when N n d → N .For some general conditions to imply this convergence and examples see [KR] (2000a,b)

Theorem 2 . 1 .
Let the imbedded point process N n converge in distribution to the Poisson process N on M c and let the intensity condition (I) hold true.Then we get: an asymptotically optimal sequence of stopping times, i.e. lim n→∞ P (X Tn = M n ) = s 1 .
Let X n i = Xi−bn an denote the normalized version as in Section 2. Similarly we consider the optimal m-choice problem for continuous time Poisson processes in the

Theorem 4 . 2 (
Approximative solution of the best m-choice problem).Let N n d → N on M c = [0, 1] × (c, ∞] and let N satisfy the intensity condition (I).

e x x x 0 H
To that purpose define constants d m and functions H m in the following way: Let for m ∈ N d m be the uniquely determined constant with e −dm dm 0 m (y)dydx = H m (d m ).
is determined by A(t)F (v m (t)) = d m(4.15)    with d m as determined in(4.12).This completes the induction.RemarkAs remarked by the reviewer a nonlinear monotone transformation reduces the product case f (t, y) = −a(t)F (y) to the Poisson process in [0, 1] × [−∞, 0].This reduction allows to reduce the calculations in the proof of Theorem 4.4 and also those in Example 3.2.Since we are convinced that also some examples not in product form can be dealt with by our method we kept the somewhat more involved calculations for use in possible extensions and examples.We next will determine the solution of the optimal 2-choice problem for this type of densities.The constant d 1 is given by d 1 = 0.8043522 . . .,

EJP 17 (
2012), paper 54.Page 19/22 ejp.ejpecp.org The basic assumption in our approach is convergence in distribution of the imbedded planar point process to a Poisson point process N , Kühne and Rüschendorf is abbreviated within this paper with [KR], Faller with [F], Faller and Rüschendorf with [FR].EJP 17 (2012), paper 54.
Thus we get t 0 = 0 and v ≡ c if A(0) d where d is the constant given in (2.16).
If A(0) > d, then t 0 is the smallest point satisfying A(t 0 ) = d, and for t ∈ [0, t o ) v(t) is a solution of the equation (t, x) when these are < 1.We now obtain from (4.7) the inequality x) only stops at time points T m 1 (t, x), . . ., T m m (t, x) but not at time points T m−2 1 (t, x), EJP 17 (2012), paper 54.