On sequential selection and a first passage problem for the Poisson process

This note is motivated by connections between the online and offline problems of selecting a possibly long subsequence from a Poisson-paced sequence of uniform marks under either a monotonicity or a sum constraint. The offline problem with the sum constraint amounts to counting the Poisson arrivals before their total exceeds a certain level. A precise asymptotics for the mean count is obtained by coupling with a nonlinear pure birth process.


Introduction
When a shuttle carrying a large number of hotel guests arrives at the hotel, the passengers start queuing and pass the exit door at times of a Poisson process. The waiting times spent in the queue are added up as the passengers quit. What is the number N(t) of passengers that exit the shuttle before the accumulated waiting time exceeds t?
We shall call this the shuttle exit problem. The exit count N(t) is the maximum number of Poisson times with the total not exceeding t. The total waiting time and the exit count process are important in many models of applied probability. Our interest stems from the connection to the online version of the longest increasing subsequence problem with Poisson arrivals, which we now describe.
Suppose independent, uniform [0,1] marks arrive sequentially at times of a unit rate Poisson process on [0, t]. A prophet with complete overview of the data can use an offline algorithm to select the longest increasing subsequence of length L * (t). A nonclairvoyant gambler learns the data and makes irrevocable decisions in real time using a nonanticipating online selection policy. Let L(t) be the length of increasing subsequence selected under the online policy that achieves the maximum expected length. As t → ∞, (where c 0 = 1.77 . . . is an explicit constant). The limit ratio 2 : √ 2 serves as a rough measure of advantage of the prophet over the gambler. The asymptotics (1) has a long and colourful history, culminating in the work by Baik, Deift and Johansson [3]. See Romik's book [18] for a nice exposition. The leading term of (2) is due to Samuels and Steele [19] who were first to study the online problem, later Bruss and Delbaen [7] identified the logarithmic order of the second term and the full expansion has appeared recently in [13].
Remarkably, the online increasing subsequence problem can be recast as a very different stochastic task, with the monotonicity constraint replaced by the condition that the sum of selected marks should not exceed 1. The latter is commonly interpreted as a bin-packing problem, where gambler's objective is to maximise the expected number of items packed online in a bin of unit capacity [6,9]. By analogy with (1) and (2) it is natural to consider the offline counterpart of L(t) in the bin packing context. Obviously, with full information, the optimal prophet's policy amounts to the smallest first policy that packs the items in the increasing order of size as long as they fit in the bin.
Since the marks sorted into increasing order themselve comprise a homogeneous Poisson process, zooming in the marks scale with factor t and changing the metaphore, it is seen that the number of items packed under the smallest first policy coincides with the exit count N(t) from the shuttle problem we started with.
The first surprise in the online-offline bin packing comparison comes with the fact that the limit prophet-to-gambler ratio is equal to 1. This follows from the asymptotics EN(t) ∼ √ 2t, which in turn can be concluded from a benchmark [6,8,11,21] upper bound EN(t) < √ 2t, the trivial inequality L(t) ≤ N(t) and (2). Therefore to assess the magnitude of prophet's advantage one needs to examine the finer the mean exit count more closely.
In this note we find a formula for EN(t) in terms of the Borel distribution. Though explicit, the formula seem to require substantial analytic work to extract the desired second term of the asymptotic expansion. We circumvent this by resorting to elementary probabilistic tools, with the core of our approach being the observation that N(t), for each fixed t, has the same distribution as the entrance count M(t) appearing in the following dual shuttle entrance problem.
When the shuttle picks up hotel guests at the airport, they enter by the Poisson process. The shuttle departs at the moment when the total waiting time of the driver and all passengers inside the shuttle is t. What is the number M(t) of hotel guests in the shuttle by the departure?
We observe that the process M(t) is a nonlinear pure-birth Markov chain which was considered in Kingman and Volkov [15] in the context of gunfight models. Using the identity in distribution we show that and that the difference is always less than 1. This contrasts sharply with the second terms in (1) and (2). For the difference between the prophet and gambler values we have therefore EN(t) − EL(t) ∼ 1 12 log t.
Bruss and Delbaen [7] showed that L(t) is AN( √ 2t, 1 3 √ 2t) (AN abbreviates 'asymptotically normal'), see also [13]. We argue that the same is true for N(t). The asymptotic coincidence of variances looks unexpected since the underlying selection policies are very different. We remind that in the increasing subsequence problem the types of the limit distribution of L * (t) and L(t) are different, as the distribution of the maximum offline length L * (t) approaches the Tracy-Widom law from the random matrix theory [3,18].
This note is a collection of snapshots around (3). To keep the discussion short, details of routine proofs are only sketched. Related work on sums of consequitive arrivals in the case of inhomogeneous rate appeared in [2], and on the integrated Poisson process in [22].
The rest of the paper is organised as follows. In the next two sections we add insight to what is already known regarding the coupling of online selection problems and the benchmark upper bound. In sections 4 and 5 we scrutinise the exit-entrance duality. In section 6 we record the normal limits. In section 7 we derive a series formula for the mean count. In section 8 we employ the pure birth process to refine the √ 2t asymptotics. In section 9 we depoissonise (3) to improve upon the well known [6,8,9,21] fixed sample asymptotics of the smallest first policy. A large deviation bound needed for our arguments is derived in the last section.
Throughout we shall be using the notation

Coupling of online problems
We first detail the equivalence between the online increasing subsequence and bin packing problems. The question about explicit coupling was emphasized in Section 5 of Steele [21], where problems with fixed number of arrivals n were discussed. The distribution of marks in the increasing subsequence problem does not matter (subject to being continuous), while the bin packing problem is not distribution-free. In the special case of uniform [0, 1] marks and the bin of unit capacity, the equivalence in terms of the optimal policies is commonly argued by comparing the dynamic programming equations for the value function [1,9]. It is also noticed in [9] (p. 455) that the greedy online bin packing policy translates as the increasing sequence of record marks.
The following construction provides a general coupling in our setting with the Poisson arrivals, but it can be readily adjusted to other arrival processes including the discrete time models with fixed or random horizon [1,11,19].
We define an i-selection policy to be a nondecreasing, adapted, cádlág jump process I with I(0) = 0, such that the north-west corners of the graph of I are some atoms (τ k , ξ k ) of Π labeled by increase of the time component. This sequence of atoms spanning the graph is an increasing chain in the partial order in two dimensions.
Similarly, we define a b-selection policy to be a nondecreasing, adapted, cádlág jump process B with B(0) = 1 and values in [0, 1]. We require that each jump be corresponding to an atom (τ k , ξ k ), so that the jump-time is τ k and the increment is ξ k . Thus the range of B is the sequence of partial sums of ξ 1 , ξ 2 , . . . .
For a fixed i-selection policy I, we are going to introduce an invertible random transform φ I of [0, ∞) × [0, 1], which will map I to a b-selection policy with the same path B = I. The construction is iterative.
At each step k we shall have [0, ∞) × [0, 1] and its duplicate obtained by a measurepreserving β k . Start with two identical copies of the strip equipped with Poisson point scatters of Π , and a fixed path of I spanned on some points (τ k , ξ k ). Let β 0 be the identity, and ξ 0 = 0. At step k > 0 only the strip β k ((τ k , ∞) × [ξ k−1 , 1]) undergoes a change which amounts to cutting at height ξ k − ξ k−1 by the horizontal line and placing The mapping β k+1 is the composition of β k and this surgery. With probability one, each point moves under β k 's finitely many times, as the moves may only be associated with (τ k , ξ k )'s to the left of this point. Thus we may define φ I as the composition of all β k 's.
Note that φ I preserves the planar Lebesgue measure and does not alter the time component, so leaving each set (t, ∞) × [0, 1] invariant. Consider the transformed point process Π := φ I (Π ). By the invariance, Π and Π share the same one-dimensional Poisson process of arrival times. Given arrival at time τ , the image of (τ, 1] . But this implies that Π has the same distribution as Π . The transformation φ I sends the sequence (τ k , ξ k ) to a sequence (τ, ξ k − ξ k−1 ) (where ξ 0 = 0), which are now some atoms of Π , and I becomes a b-selection policy spanned on the transformed sequence.
The above concepts of selection policy are much more general than the Markovian threshold policies studied in the literature. For the purpose of optimisation, however, it is sufficient to consider the following family of policies. For ψ : [0, ∞) → [0, 1] thought of as a function controlling the size of acceptance window, and given horizon t, an i-selection policy is defined recursively by the rule: conditionally on arrival occurring at time τ < t and given I(τ −) = x (the last selection so far), the observed mark ξ is selected if and only if In [13] we called such policies self-similar because the performance from each stage on only depends on the mean number of future acceptable arrivals. Thus defined, I is a jump Markov process with transition mechanism determined by ψ. The twin b-selection policy has the acceptance condition given B(τ −) = x (the total of selected items so far). The optimal i-/b-selection policy is of this form with some control ψ * satisfying see [5,7,13]. In [13] we proved that every policy having ψ(z) ∼ 2/z is within O(1) from the optimality, that is achieves the asymptotics (2). The general Markovian policy differs from (4) and (5) in that ψ is replaced by the general function of τ, t and x. Notable other examples are the greedy policy with the function 1 and the stationary policy with the function 2/t ∧ 1.

The upper bound
For the rest of this paper the variable t will have the meaning of either the bin capacity (the offline bin-packing contest) or the total waiting time (the shuttle context). For the time parameter of the Poisson process we shall use the variable x.
Let π 1 < π 2 < . . . be the points of a unit rate Poisson process Π on the positive half-line. The exit count is defined as N(t) := max{n : π 1 + · · · + π n ≤ t}, t ≥ 0, There is a benchmark upper bound for the mean, that appeared in the Poisson setting in [6] (Example 2.4). Similar inequalities for sums of order statistics from the general distribution are found in [8], also see [21] for extended discussion. We relate (6) to an isoperimetric inequality, much in line with the examples from [5,11]. Fix t. The set of Poisson points π n with π 1 + · · · + π n ≤ t is a point subprocess of Π with rate function p t satisfying This suggests a problem from the calculus of variations, The Lagrangian function becomes which for given multiplier θ is maximised by the indicator function q(x) = 1(x ≤ θ). Accounting for the constraint, the overall maximum value of the integral is √ 2t, attained at which gives the upper bound (6) follows.
Remark Solution q * corresponds to a packing policy that picks all items smaller than the threshold 2/t. The policy violates the (almost sure) sum constraint but meets a weaker mean-value constraint. This policy is online implementable and outputs the number of selections with Poisson( √ 2t) distribution, so has the variance about three times higher than under the optimal offline (see below) or the optimal online policy [7,13].
The entrance count process has a simple combinatorial interpretation. Think of an urn with one red and some number of white balls. At times of the Poisson process a ball is randomly chosen and replaced to the urn. If the chosen ball is red, a white ball is added to the urn, otherwise the urn composition is not changed. For the process starting with one red ball, M(t) is the number of white balls in the urn at time t.
The identity (8) only holds for the marginal distributions, and the exit count process (N(t), t ≥ 0) is not even Markovian. The driver's waiting time was included in the total waiting time to avoid a shift in the distributional identity. We note in passing that without appealing to (8) the upper bound EM(t) ≤ √ 2t does not seem at all obvious.

Integrals of the Poisson process
Let The total waiting time accumulated within the real time x is T (x) in the shuttle exit problem, and x + S(x) in the entrance problem, where x is added to account for driver's waiting time. The integration by parts formula becomes By reversibility of Π on [0, x] we have This identity has appeared in [22], where it was concluded analytically from the identity of Laplace transforms. Despite that (9) holds for each fixed x, the processes are very different: T is a jump process with independent increments, while the paths of S are piecewise linear.
Now, given π n+1 , the variables T (π n ) and S(π n+1 ) have the same distribution, and so unconditionally in apparent disagreement with (9). Moreover, S(π n+1 ) d = S(π n ) + π n , which is to be compared with (10) and (11). The latter identity is equivalent to π n+1 (u 1 + · · · + u n ) d = π n (1 + u 1 + · · · + u n−1 ), where the π n 's are independent of the iid uniform u j 's. To prove the last formula directly, one can observe two ways to split T n in independent factors, as π n+1 (T (π n )/π n+1 ) and π n (T (π n )/π n ), then represent the quotients in brackets in terms of the u j 's. See [12] for more involved exponential-uniform identities derived from the planar Poisson process.
Next, we aim to represent the exit and entrance counts as time-changed Poisson process. Let X(t) = min{x : T (x) > t} be the right-continuous inverse of T , with X(0) = π 1 . We can take here min rather than infinum since T jumps at the discrete set of Poisson points. We have then N(t) = Π(X(t)) − 1.
The entrance counting process satisfies The last two formulas give yet another proof that the entrance count is a pure-birth process with the jump rate (n + 1) −1 at state M(t) = n.

Normal limits
Note that T has independent increments. Application of Campbell's formula yields the moments E T (x) = 1 2 x 2 , Var T (x) = 1 3 x 3 , and, more generally, the moment generating function Inverting this, Suyono and van der Weide [22] found the density of T (x) in terms of modified Bessel functions (note that T (x) has mass e −x at zero). Routine application of the law of large numbers and the central limit theorem show that for x → ∞ T (x), S(x) ∼ 1 2 x 2 a.s., and are AN 1 Inverting these asymptotic relations in a way familiar from the renewal theory, using (12), (13) and the asymptotics of Π itself, we obtain for t → ∞ that N(t), M(t) ∼ √ 2t a.s., and are AN √ 2t , 1 The representation M(t) = max{n : ζ n ≤ t} embeds the analysis of the 'renewal function' ν(t) = EM(t) into the general framework of the renewal theory with nonhomogeneous inter-arrival times [20]. Let a n := 1 2 n(n + 1), b n := 1 6 n(n + 1)(2n + 1).
It is routine to see that E ζ n = a n , Var ζ n = b 2 n and that, ζ n ∼ a n a.s., and is AN(a n , b 2 n ).
Inverting this yields another, more straightforward, proof of (14). The normal limit suggests the asymptotics for the variance For a time being we shall take the formula for granted, deferring its justification, by checking the uniform integrability, to the last section of this paper.

Exact formulas
Recall that ζ n = n j=1 jη j (with the η j 's being iid exponential), which has the same distribution as the entrance total waiting time S(π n ) + π n .
The Laplace transform of ζ n is Inverting this yields a formula for the distribution function See [23] for an asymptotic expansion for large t. For the mean of N(t), with a small series work, we obtain an exact formula Intriguingly, (16) can be viewed as a mean over the Borel distribution which is the law for the total offspring in the branching process with the Poisson(1) reproduction. Specifically, 8 Bounds on the mean and the limit constant The transition probability of the entrance count process M is which upon taking the expectation becomes 1 1 Consider the general pure-birth Markov chain M with M (0) = 0, and transition rates β n , n ≥ 0, meeting the regularity condition Applying Jensen's inequality we arrive at a differential inequality which is readily solved by separating variables as ν(t) > √ 2t + 1 − 1, t > 0. So together with (6) we have fairly tight bounds where the gap stays below 1 for all t. The bounds (19) clearly suggest that the gap converges to a constant. Next, we aim at finding the constant perceived from (19). The random variable M(t) + 1 = min{n : ζ n > t} is a stopping time. Doob's optional sampling theorem applied to the martingale ζ n − a n yields a Wald-type identity On the other hand, conditionally on M(t) = n the distribution of ζ M (t)+1 −t is exponential with rate (n + 1) −1 , so unconditionally we can write the identity in distribution where η is a unit exponential random variable, independent of M(t). Thus which together with (20) give 2 Alternatively, (22) can be derived from the k = 2 instance of the formula generalising (18). Expanding the right-hand side, it is seen that all moments EM k (t) can be expressed, recursively, via the first moment ν(s), s ≤ t. Formula (22) allows us to express the variance through the mean as 2 For the general birth process as in the previous footnote, assuming M (0) = 0 the identity is Plugging the lower bound (19) in (23) yields the bound σ 2 (t) < √ 2t + 1 − 1, which for large t is too far from (yet to be justified) (15). But working other way round we substitute (15) with indefinite smaller order remainder in (23), and work out the quadratic equation to extract the value of the sought limit constant: This result contrasts expansions (1) and (2) but brings to mind some analogy with the expansion of the classic renewal function in the setting with uniform interarrival times [4] (Ch. 11, Example 8). Numerical calculations with (16) suggest that the limit is approached monotonically from below. In a private communication, Alex Marynych informed us that he could arrive at the asymptotics √ 2t − 2/3 using Theorem 1 from [16] to approximate the tail of the Borel distribution in (17). Continuing the analogy with [7,13], one can conjecture that the next term of the asymptotic expansion of the mean is of the order of t −1/2 , see also [10] for a similar situation.

The smallest first policy for fixed sample size
We turn to the smallest first policy in the bin packing problem with unit capacity and fixed sample size n. Let 0 < u n1 < · · · < u nn < 1 be the uniform [0, 1] order statistics, and let K n = max{k : u n1 + · · · + u nk ≤ 1.
be the smallest first count, κ n := EK n . An explicit formula for κ n exists [9] (Theorem 7, with c = 1), but is not particularly user-friendly, as involving an alternating double. It is well known that κ n ∼ √ 2n and that √ 2n is also an upper bound [8,9,11,19,21] We assert now a much more precise result: To show this, we first resort to the setting of the Poisson process on [0, 1] with rate t. The exit count N(t) translates as the maximal number of Poisson points whose total is at most 1. Thus we have the poissonisation relation κ n e −t t n n! .
To depoissonise, we check conditions of Theorem 1 from [14]. The function ν(t) given by (16) for complex argument t ∈ C is an entire function, as the series converges everywhere. Some analytic work with the aid of the Stirling formula shows that |ν(t)| < c 1 |t| 1/2 in the sector | arg t| < π/4. Outside the sector, we have an estimate |e t ν(t)| < c 2 |t| 1/2 exp(|t|/ √ 2).
This follows by observing that the maximum of |e t (1 − e −t/j )| for given |t| is achieved at the boundary |arg t| = π/4, and then by approximating the sum (16) by an integral. The cited theorem gives the possonisation error |ν(t) − κ ⌊t⌋ | = O(t −1/2 ), t → ∞, hence (24) implies (25). More generally, suppose the bin has capacity C > 0. Consider the smallest first policy applied to the Poisson process on [0, 1] with rate t and, in parallel, to n items sampled from the uniform [0, 1] distribution. Extending our notation from the case C = 1, let N C (t) and K C,n be the counts of items packed, and let ν C (t), κ C,n be their means, respectively.
Generalising the C = 1 result, we argue that For C ≤ 1, this is straightforward, as ν C (t) = ν(Ct) and we readily conclude (26) from the C = 1 case.
For C > 1 we need to be more careful since the maximum size of item is constrained by 1 and not by C. Assessing the mean in terms of the unit Poisson process on [0, Ct] we have {P(T (π n ) ≤ Ct) − P(T (π n ) ≤ Ct, π n > t)} = ν(Ct) − ∞ n=1 P(T (π n−1 ) ≤ (C − 1)t, π n > t).
As t → ∞, a large deviation estimate for the Poisson process shows that Σ 1 approaches 0 exponentially fast, and the same is shown for Σ 2 using S(π n ) = ζ n and the large deviation estimate (27) in the next section.

A large deviation bound
To justify asymptotics of the variance (15) it remains to verify that the family is uniformly integrable.
For η with the unit exponential distribution, the central moments are estimated as Using this it is easy to check that which verifies the condition for large deviation bounds from [17] (Chapter 3, Theorem 17). Hence we obtain sup z≥0 P(|ζ n − a n | > b n z) < 3 e −z/4 , where both constants are not sharp. Inverting the latter we arrive at similar bound < c e −z 2 /4 , which implies the desired uniform integrability.