for the coupon collector’s

We develop techniques of computing the asymptotics of the moments of the number $T_N$ of coupons that a collector has to buy in order to find all $N$ existing different coupons as $N\rightarrow \infty.$ The probabilities (occurring frequencies) of the coupons can be quite arbitrary. After mentioning the case where the coupon probabilities are equal we consider the general case (of unequal probabilities). For a large class of distributions (after adopting a dichotomy) we arrive at the leading behavior of the moments of $T_N$ as $N\rightarrow \infty.$ We also present various illustrative examples.


Introduction -History of the problem
Consider a population whose members are of N different types (e.g. the population may consist of fish, viruses, English words, or baseball cards).For 1 ≤ j ≤ N we denote by p j the probability that a member of the population is of type j.The members of the population are sampled independently with replacement and their types are recorded.The so-called "Coupon Collector's Problem" (CCP) deals with questions arising in the above procedure.In particular, CCP pertains to the family of urn problems.Other classical such problems are birthday, Dixie cup or occupancy problems, whose origin can be traced back to De Moivre's treatise De Mensura Sortis of 1712 (see, e.g., [16]).CCP (in its simplest form, i.e. the case of equal probabilities) had appeared in W. Feller's classical work [12] and has attracted the attention of various researchers, since it has found many applications in many areas of science (computer science-search algorithms, mathematical programming, optimization, learning processes, engineering, ecology, as well as linguistics, -see, e.g., [5]).Let T N be the number of trials it takes until all N types are detected (at least once).Apart from its distribution some other interesting quantities are the moments (or, equivalently, the rising moments) of the random variable T N .For the case of equal sampling probabilities the first and the second moment of T N are well known and, furthermore, asymptotics as well as limiting results have been obtained by several authors (see for instance [19], [11], [3], [17], [16], [10], [18], and [8]).In particular, in [19] the authors answered the question: how long, on average does it take to obtain m complete sets of N coupons.For unequal probabilities, general asymptotic estimates regarding the first and the second moment, as well as for the variance, have also been obtained by several authors (see, e.g., [7], [13], [8], [9]).
Let r ≥ 1, be an integer.Set T (r) In this paper we consider the r-th rising moment of T N , namely In Section 2 we present general expressions for E T (r) N and exhibit well-known results, mainly for the simplest case of the problem, i.e. the case of equal probabilities.We also describe the general setup of the problem considered in the present paper.In Section 3 we begin by discussing a key feature, namely that the families of the coupon probabilities, i.e. the p j 's, can be divided in two types.The main result for the p j 's of the first type is presented in Theorem 3.5 (the so-called linear case falls in this category).Then, we consider a large class of families of coupon probabilities belonging to the second type.The (leading) asymptotic behavior of the rising moments E T (r) N is given in Theorem 3.7 (the generalized Zipf law falls in this case).Furthermore, Theorem 3.9 helps us obtain asymptotic estimates by comparison with cases for which the asymptotic estimates are known.Section 4 contains various examples.Finally, we mention some possible extensions at the end of the paper.

Preliminaries
For each j ∈ {1, ..., N } it is convenient to introduce the event A k j , that the type j is not detected until trial k (included).Then

By invoking the inclusion-exclusion principle one gets
where the sum extends over all 2 N −1 nonempty subsets J of {1, ..., N }, while |J| denotes the cardinality of J.For z ∈ C, |z| ≥ 1, we introduce the following moment generating function of T N , (the derivation of the second equality is based on Abel's partial summation formula).Consequently, by using (2.1) one arrives at and, hence, by summing the geometric series We proceed by noticing that (2.4) Finally, by comparing (2.3) and (2.5) we get or, equivalently, by substituting x = e −t in the integral, Remark 2.1.An alternative way to derive (2.6)-(2.7) is by adapting the nice approach of [13], where the main ingredient is an appropriate generating function.
Observe that, from which we arrive at the formulas (2.8)

The equally likely case
Naturally, regarding the previous formulas the simplest case occurs when one takes (2.9) Actually, this case apart from its simplicity, has the property that among all sequences, it is the one with the smallest moments of T N .This is a known result (see [5]).For example, (2.7) and (2.8) imply immediately that, for a given z ≥ 1 attains its maximum value, while E T (r) N attain its minimum value, when all p j 's are equal.Under (2.9), one has (2.10) Repeated integration by parts in the last integral yields (2.11) where the α r (N )'s are defined recursively by It seems that formulas for E T (r) N had been first obtained in [15].Foata et al (see [14]), called the numbers α r (N ) hyperharmonic and derived their asymptotics using multivariate generating fuctions.Soon after, Adler et al (see [1]), gave explicit expression for the asymptotics of the hyperharmonic numbers using basic probability arguments.
In particular, (see [14]) (2.12) To give an idea of how higher order asymptotics for E T (r) N look like, let us mention that, e.g., for r = 3 we have either from [14], or by repeated application of Abel partial summation method

E T
(3) where γ is the Euler's constant.
Asymptotics of the moments for the coupon collector's problem

Large N asymptotics for general families of coupon probabilities
When N is large it is not obvious at all what information one can obtain for E T (r) N from formula (2.8).For this reason there is a need to develop efficient ways for deriving asymptotics as N → ∞ (we have already analyzed the very special case of equal probabilities-see formulas (2.12)-(2.13).Let α = {a j } ∞ j=1 be a sequence of strictly positive numbers.Then, for each integer N > 0, one can create a probability measure π N = {p 1 , ..., p N } on the set {1, ..., N } by taking (2.14) Notice that p j depends on α and N , thus, given α, it makes sense to consider the asymp- as N → ∞.This way of producing sequences of probability measures first appeared in [6].
Remark 2.2.Clearly, for given N the p j 's can be assumed monotone in j without loss of generality.As for the sequence {a j } ∞ j=1 , (i) if a j → ∞, then for each k ∈ N there is a j = j(k) ≥ k such that a j ≥ a i , for all i ≤ j.This tells us that, by rearranging the terms a i , where j(k) ≤ i ≤ j(k + 1), {a j } ∞ j=1 can be assumed nondecreasing without loss of generality.
(ii) Similarly, if a j → 0, then {a j } ∞ j=1 can be assumed nonicreasing without loss of generality.

E T (r) N
= A r N H N (α; r). (2.17) As it was noticed in [6] and [8] for as N → ∞, can be treated as two separate problems, namely estimating A r N and estimating H N (α; r).Our analysis focuses on estimating H N (α; r).The estimation of A r N will be considered an external matter which can be handled by existing powerful methods, such as the Euler-Maclaurin Summation formula, the Laplace method for sums (see, e.g., [4]), or even summation by parts.
Remark 3.3.It has been shown in [6], that L 1 (α) < ∞, if and only if there exist a ξ ∈ (0, 1) such that To sum up we have the following dichotomy, simultaneously for all positive integers r: Then (for all positive integers r) by (2.15), (3.1), Fubini-Tonelli's theorem, and repeated integration by parts, we have a −r j . (3.7) ) can serve as an upper bound for the error ∆ r (N ).

The case L r (α) is finite
Let A N and L r (α) be as in (2.14) and (3.1) respectively.We note that, by Theorem 3.1, L r (α) < ∞ implies that lim j a j = ∞ (hence lim N A N = ∞).
Theorem 3.5 states that if L r (α) < ∞, then the asymptotics of E T (r) N are essentially determined by the asymptotics of A N .As was already mentioned, asymptotic estimates of A N can be obtained by various known methods.Alternatively, one can resort to specific features of α.For instance, if α is of the form a j = e jcj , where c j ∞, then it is an easy exercise to see that, as N → ∞, a j ∼ a N . (3.10) To verify (3.10), we use (3.9) and sum a geometric series to get where M = 1/(e c1 − 1).Since, the result follows.In words, if a sequence satisfies (3.9), then in the sum of (3.10), the last term dominates all the previous terms.Examples of such sequences are a j = e j r with r > 1, a j = j j and a j = j! (see Example 4.5).
We now continue with a much more challenging case.x aj = ∞, for all x ∈ (0, 1).For our further analysis, we follow [6], and write a j in the form , where f (x) > 0, (3.11) and assume that f (x) possesses two derivatives satisfying the following conditions as x → ∞: → 0. (3.12) Conditions (3.12) are satisfied by a variety of commonly used functions.For example, as well as various convex combinations of products of such functions.
Remark 3.6.From condition (ii) of (3.12), one has This can be justified by considering the function g(x) = ln(f (x)) and applying the Mean Value Theorem.
Hence, in view of (2.16) one can write (2.15) as: It has been established in [6] that, and also that These two results came out under conditions (3.12).Applying the Bounded Convergence Theorem for the first integral on (3.16) yields (in view of (3.17)) Next, we want to estimate the integral which appears in (3.19).We begin by noticing that by the Dominated Convergence Theorem (since In view of (3.18) the above formula implies that lim Since f is increasing, we have However, by (3.18) lim Hence, by taking limits in (3.22) and using (3.20) and (3.13), we get Finally, by the definition of F (•) and the Taylor expansion for the logarithm, namely ln(1 and the proof is completed. Remark 3.8.Using Theorem 3.7 in (2.17) we get, as N → ∞, where the last equality follows from (2.14).

Asymptotic estimates for the rising moments of T N by comparison with known sequences
In this subsubsection we will present a theorem that helps us obtain asymptotic estimates by comparison with sequences α for which the asymptotic estimates of H N (α; r) are known (for instance, via Theorem 3.7).A similar theorem concerning the special case of r = 1, can be found in [6].First, we recall the following notation.Suppose that {s j } ∞ j=1 and {t j } ∞ j=1 are two sequences of nonnegative terms.The symbol s j t j means that there are two constants C 1 > C 2 > 0 and an integer j 0 > 0 such that i.e. s j = O(t j ) and t j = O(s j ).(i) If there exists an j 0 such that a j = b j , for all j ≥ j 0 , then if we let c N = e pN , then {b j : 0 ≤ j ≤ N } = {c N a j : 0 ≤ j ≤ N }, for each N , i.e. the elements of the two truncated sequences are proportional to each other.Hence, the sequences β and α produce the same coupon probabilities.In this way we get cheaply a counterexample for Theorem 3.7, in case where f (•) does not satisfy all conditions of (3.12).
Example 4.4.a j = 1/j p , p > 0. This is the so-called generalized Zipf law.In this case Theorem 3.1 implies L r (α) = ∞.If f (x) = x p , then f satisfies (3.12) and hence Theorem 3.7 apply.It is now straightforward (say, form the Euler-Maclaurin Summation formula-see, e.g., [2]) to estimate A r N and get where ζ(•) denotes the Riemann zeta function.Hence Theorem 3.7 gives

Concluding remarks
The main topic of this paper was the asymptotics of E[ T N ], namely the r-th rising moment of T N , as N → ∞.We have already mentioned the work of H.J. Godwin [15], in the case of uniform coupon probabilities.We are not aware of any previous work on asymptotics of higher rising moments (r ≥ 3) in the case of unequal coupon probabilities.Of course, in the existing literature there are many works on the asymptotics of E[ T N ] and, also, few works regarding E[ T 2 N ] and V [ T N ], namely the variance of the random variable T N .
Let us discuss briefly few representative works.The first and the second moment of T N were studied in [7].In this article R.K. Brayton (Ph.D. thesis under N. Levinson) derived an asymptotic formula for V [ T N ] under very restrictive assumptions on α.In particular, the probabilities p j considered in [7] must satisfy: General asymptotic estimates, for the case r = 1 were found in [6], for the families of coupon probabilities which we study in the present paper.Our results here are in accordance with [6].
The case r = 1, 2 was considered in [8].The authors adopted the dichotomy of [6] and obtained the leading behavior of the variance V [ T N ].Moreover, for a large class of families of coupon probabilities they obtained detailed asymptotics of E[ T N ] and E[ T N (T N + 1) ] (up to the fifth and sixth term respectively).Notice that their results complement the results of [7], since they concern quite general sequences for which the ratio λ(N ) of (5.1) is not bounded (e.g.linear and Zipf).
Recently, J. Du Boisberranger, D. Gardy, and Y. Ponty, [9] considered the word collector problem, i.e. the expected number of calls to a random weighted generator before all the words of a given length in a language are generated.The main ingredient of this instance of the non-uniform coupon collector lies in the, potentially large, multiplicity of the words (coupons) of a given probability (composition).They obtained a general theorem that gives an asymptotic equivalent for the expected waiting time of a general version of the Coupon Collector (case r = 1).This theorem is especially well-suited for classes of coupons featuring high multiplicities.Their results and [6] are complementary.
Finally, let us mention that it was pointed out to us that an important case in the applications is when there is a subcollection of coupons that continues to grow (as N grows) and all of the coupons in the subcollection have the same probability; it could be called "uniform subcollection".This can be modeled by a sequence {a j } ∞ j=1 which is the "union" of two subsequences one of which is constant (this corresponds to the uniform subcollection of coupons), while the other is of the form discussed in this subsection.In this case, we conjecture that Theorem 3.7 is still valid (under an appropriate renaming of the index) provided the "density" of the constant subsequence is sufficiently small.In other words, the uniform subcollection does not affect the asymptotic distribution.Furthermore, if {a j } ∞ j=1 is the "union" of two vanishing subsequences one of which decays much faster than the other, then we conjecture that the faster one prevails in the asymptotics, provided its density is not very small.