A device has two arms with unknown deterministic payoffs and the aim is to asymptotically identify the best one without spending too much time on the other. The Narendra algorithm offers a stochastic procedure to this end. We show under weak ergodic assumptions on these deterministic payoffs that the procedure eventually chooses the best arm (i.e., with greatest Cesaro limit) with probability one for appropriate step sequences of the algorithm. In the case of i.i.d. payoffs, this implies a “quenched” version of the “annealed” result of Lamberton, Pagès and Tarrès [Ann. Appl. Probab. 14 (2004) 1424–1454] by the law of iterated logarithm, thus generalizing it.

More precisely, if (η_ℓ,i)_i∈ℕ ∈ {0, 1}^ℕ, ℓ ∈ {A, B}, are the deterministic reward sequences we would get if we played at time i, we obtain infallibility with the same assumption on nonincreasing step sequences on the payoffs as in Lamberton, Pagès and Tarrès [Ann. Appl. Probab. 14 (2004) 1424–1454], replacing the i.i.d. assumption by the hypothesis that the empirical averages ∑_i=1ⁿη_A,i / n and ∑_i=1ⁿη_B,i / n converge, as n tends to infinity, respectively, to θ_A and θ_B, with rate at least 1/(log n)^1+ε, for some ε > 0.

We also show a fallibility result, that is, convergence with positive probability to the choice of the wrong arm, which implies the corresponding result of Lamberton, Pagès and Tarrès [Ann. Appl. Probab. 14 (2004) 1424–1454] in the i.i.d. case.

A many-server queueing system is considered in which customers arrive according to a renewal process and have service and patience times that are drawn from two independent sequences of independent, identically distributed random variables. Customers enter service in the order of arrival and are assumed to abandon the queue if the waiting time in queue exceeds the patience time. The state of the system with N servers is represented by a four-component process that consists of the forward recurrence time of the arrival process, a pair of measure-valued processes, one that keeps track of the waiting times of customers in queue and the other that keeps track of the amounts of time customers present in the system have been in service and a real-valued process that represents the total number of customers in the system. Under general assumptions, it is shown that the state process is a Feller process, admits a stationary distribution and is ergodic. It is also shown that the associated sequence of scaled stationary distributions is tight, and that any subsequence converges to an invariant state for the fluid limit. In particular, this implies that when the associated fluid limit has a unique invariant state, then the sequence of stationary distributions converges, as N → ∞, to the invariant state. In addition, a simple example is given to illustrate that, both in the presence and absence of abandonments, the N → ∞ and t → ∞ limits cannot always be interchanged.

This note investigates cover levels of finite sets in the random interlacements model introduced in [Ann. of Math. (2) 171 (2010) 2039–2087], that is, the least level such that the set is completely contained in the random interlacement at that level. It proves that as the cardinality of a set goes to infinity, the rescaled and recentered cover level tends in distribution to the Gumbel distribution with cumulative distribution function exp(−exp(−z)).

We consider an SIR epidemic model propagating on a configuration model network, where the degree distribution of the vertices is given and where the edges are randomly matched. The evolution of the epidemic is summed up into three measure-valued equations that describe the degrees of the susceptible individuals and the number of edges from an infectious or removed individual to the set of susceptibles. These three degree distributions are sufficient to describe the course of the disease. The limit in large population is investigated. As a corollary, this provides a rigorous proof of the equations obtained by Volz [Mathematical Biology 56 (2008) 293–310].

For population genetics models with recombination, obtaining an exact, analytic sampling distribution has remained a challenging open problem for several decades. Recently, a new perspective based on asymptotic series has been introduced to make progress on this problem. Specifically, closed-form expressions have been derived for the first few terms in an asymptotic expansion of the two-locus sampling distribution when the recombination rate ρ is moderate to large. In this paper, a new computational technique is developed for finding the asymptotic expansion to an arbitrary order. Computation in this new approach can be automated easily. Furthermore, it is proved here that only a finite number of terms in the asymptotic expansion is needed to recover (via the method of Padé approximants) the exact two-locus sampling distribution as an analytic function of ρ; this function is exact for all values of ρ ∈ [0, ∞). It is also shown that the new computational framework presented here is flexible enough to incorporate natural selection.

We consider minimizing the probability of falling below a target growth rate of the wealth process up to a time horizon T in an incomplete market model, and then study the asymptotic behavior of minimizing probability as T → ∞. This problem can be closely related to an ergodic risk-sensitive stochastic control problem in the risk-averse case. Indeed, in our main theorem, we relate the former problem concerning the asymptotics for risk minimization to the latter as its dual. As a result, we obtain an expression of the limit value of the probability as the Legendre transform of the value of the control problem, which is characterized as the solution to an H-J-B equation of ergodic type, in the case of a Markovian incomplete market model.

We use interacting particle systems to investigate survival and extinction of a species with colonies located on each site of ℤ^d. In each of the four models studied, an individual in a local population can reproduce, die or migrate to neighboring sites.

We prove that an increase of the death rate when the local population density is small (the Allee effect) may be critical for survival, and that the migration of large flocks of individuals is a possible solution to avoid extinction when the Allee effect is strong. We use attractiveness and comparison with oriented percolation, either to prove the extinction of the species, or to construct nontrivial invariant measures for each model.

We consider a stochastic directed graph on the integers whereby a directed edge between i and a larger integer j exists with probability p_j−i depending solely on the distance between the two integers. Under broad conditions, we identify a regenerative structure that enables us to prove limit theorems for the maximal path length in a long chunk of the graph. The model is an extension of a special case of graphs studied in [Markov Process. Related Fields 9 (2003) 413–468]. We then consider a similar type of graph but on the “slab” ℤ × I, where I is a finite partially ordered set. We extend the techniques introduced in the first part of the paper to obtain a central limit theorem for the longest path. When I is linearly ordered, the limiting distribution can be seen to be that of the largest eigenvalue of a |I| × |I| random matrix in the Gaussian unitary ensemble (GUE).

Scheduling control problems for a family of unitary networks under heavy traffic with general interarrival and service times, probabilistic routing and an infinite horizon discounted linear holding cost are studied. Diffusion control problems, that have been proposed as approximate models for the study of these critically loaded controlled stochastic networks, can be regarded as formal scaling limits of such stochastic systems. However, to date, a rigorous limit theory that justifies the use of such approximations for a general family of controlled networks has been lacking. It is shown that, under broad conditions, the value function of the suitably scaled network control problem converges to that of the associated diffusion control problem. This scaling limit result, in addition to giving a precise mathematical basis for the above approximation approach, suggests a general strategy for constructing near optimal controls for the physical stochastic networks by solving the associated diffusion control problem.

If (C_n) is a Markov chain on a discrete state space $\mathcal{S}$, a Markov chain (C_n, M_n) on the product space $\mathcal{S}\times\mathcal{S}$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov chain behaves like the original Markov chain and the second component changes only when both coordinates are equal. The asymptotic properties of this Markov chain are investigated. A representation of its invariant measure is, in particular, obtained. When the state space is infinite it is shown that this Markov chain is in fact null recurrent if the initial Markov chain (C_n) is positive recurrent and reversible. In this context, the scaling properties of the location of the second component, the mouse, are investigated in various situations: simple random walks in ℤ and ℤ² reflected a simple random walk in ℕ and also in a continuous time setting. For several of these processes, a time scaling with rapid growth gives an interesting asymptotic behavior related to limiting results for occupation times and rare events of Markov processes.

Cell polarity refers to the spatial asymmetry of molecules on the cell membrane. Altschuler, Angenent, Wang and Wu have proposed a stochastic model for studying the emergence of polarity in the presence of feedback between molecules. We analyze their model further by representing it as a model of an evolving population with interacting individuals. Under a suitable scaling of parameters, we show that in the infinite population limit we get a Fleming–Viot process. Using well-known results for such processes, we establish that cell polarity is exhibited by the model and also study its dependence on the biological parameters of the model.

This article is concerned with the Axelrod model, a stochastic process which similarly to the voter model includes social influence, but unlike the voter model also accounts for homophily. Each vertex of the network of interactions is characterized by a set of F cultural features, each of which can assume q states. Pairs of adjacent vertices interact at a rate proportional to the number of features they share, which results in the interacting pair having one more cultural feature in common. The Axelrod model has been extensively studied during the past ten years, based on numerical simulations and simple mean-field treatments, while there is a total lack of analytical results for the spatial model itself. Simulation results for the one-dimensional system led physicists to formulate the following conjectures. When the number of features F and the number of states q both equal two, or when the number of features exceeds the number of states, the system converges to a monocultural equilibrium in the sense that the number of cultural domains rescaled by the population size converges to zero as the population goes to infinity. In contrast, when the number of states exceeds the number of features, the system freezes in a highly fragmented configuration in which the ultimate number of cultural domains scales like the population size. In this article, we prove analytically for the one-dimensional system convergence to a monocultural equilibrium in terms of clustering when F = q = 2, as well as fixation to a highly fragmented configuration when the number of states is sufficiently larger than the number of features. Our first result also implies clustering of the one-dimensional constrained voter model.