A noninvertible function of a first-order Markov process or of a nearest-neighbor Markov random field is called a hidden Markov model. Hidden Markov models are generally not Markovian. In fact, they may have complex and long range interactions, which is largely the reason for their utility. Applications include signal and image processing, speech recognition and biological modeling. We show that hidden Markov models are dense among essentially all finite-state discrete-time stationary processes and finite-state lattice-based stationary random fields. This leads to a nearly universal parameterization of stationary processes and stationary random fields, and to a consistent nonparametric estimator. We show the results of attempts to fit simple speech and texture patterns.

The purpose of this paper is to show the strong consistency of the maximum pseudo-likelihood estimator for continuous state space stationary Gibbsian processes under fairly general conditions. Besides the maximum pseudo-likelihood estimator of Besag, we consider its extension, the maximum pseudo-likelihood of second order. The framework of our study is Ruelle's theory of superstable potential functions.

We prove survival for a class of discrete time Markov processes whose states are finite sets of integers. As applications, we obtain upper bounds for the critical values of various two-dimensional oriented percolation models. The technique of proof is based generally on that used by Holley and Liggett to prove survival of the one-dimensional basic contact process. However, the fact that our processes evolve in discrete time requires that we make substantial changes in the way this technique is used. When applied to oriented percolation on the two-dimensional square lattice, our result gives the following bounds: $p_c \leq 2/3$ for bond percolation and $p_c \leq 3/4$ for site percolation.

This paper studies the fluid approximation (also known as the functional strong law of large numbers) and the stability (positive Harris recurrence) for a multiclass queueing network. Both of these are related to the stabilities of a linear fluid model, constructed from the first-order parameters (i.e., long-run average arrivals, services and routings) of the queueing network. It is proved that the fluid approximation for the queueing network exists if the corresponding linear fluid model is weakly stable, and that the queueing network is stable if the corresponding linear fluid model is (strongly) stable. Sufficient conditions are found for the stabilities of a linear fluid model.

We consider a network in which a call holds a given number of uniformly chosen links and releases them simultaneously. We show pathwise propagation of chaos and convergence of the process of empirical fluctuations to a Gaussian Ornstein-Uhlenbeck process. The limiting martingale problem is obtained by closing a hierarchy. The drift term is given by a simple factorization technique related to mean-field interaction, but the Doob-Meyer bracket contains special terms coming from the strong interaction due to simultaneous release. This is treated by closing another hierarchy pertaining to a measure-valued process related to calls routed through couples of links, and the factorization is again related to mean-field interaction. Fine estimates obtained by pathwise interaction graph constructions are used for tightness purposes and are thus shown to be of optimal order.

In polling systems, $M \geq 2$ queues are visited by a single server in cyclic order. These systems model such diverse applications as token-ring communication networks and cyclic production systems. We study polling systems with exhaustive service and zero switchover (walk) times. Under standard heavy-traffic assumptions and scalings, the total unfinished work converges to a one-dimensional reflected Brownian motion, whereas the workloads of individual queues change at a rate that becomes infinite in the limit. Although it is impossible to obtain a multidimensional limit process in the usual sense, we obtain an "averaging principle" for the individual workloads. To illustrate the use of this principle, we calculate a heavy-traffic estimate of waiting times.

The interactions between macroparasites and their hosts, in terms of parasite-induced acquired immunity and additional mortality of the host, are of considerable interest and importance. In this paper, a simple nonlinear stochastic model for the parasite load within a single host over the lifetime of the host is investigated. By concentrating on a model incorporating only parasite-induced excess host mortality, exact algebraic results are possible, which provide insight into the effects of this interaction mechanism. A method of approximating the moments of parasite load is explored under a range of parametric assumptions. Extensions of the model to allow for heterogeneity between hosts and to incorporate various types of acquired immunity within the host are discussed briefly.

In this paper explicit formulas are given for the distribution function, the density function and the moments of the local time of the reflecting Brownian motion process.

An explanation of a remarkable identity in law, due to A. Dassios, concerning the $\alpha$-quantile of Brownian motion with drift is given with the help of Bertoin's rearrangement of positive and negative excursions for Brownian motion with drift.

A model of commodity trading consists of $n$ traders, each bringing to the market his own individual good and each having his own preference for the goods on the market. The trade results in a so-called core allocation, that is, an exchange of goods which cannot be destabilized by a coalition of traders. Shapley and Scarf, who proposed the model, proved the existence of such an exchange by means of an algorithm invented by Gale. The algorithm determines sequentially a cyclic decomposition of the set of traders into trading groups with equally priced goods that satisfies the stability requirement. In this paper the work of the algorithm is studied under an assumption that the traders' individual preferences are independent and uniform. It is shown that the decreasing sequence of the market sizes has the same distribution as a Markov chain $\{\nu_i\}$ on integers in which the next state $\nu'$ is obtained from the current state $\nu$ by randomly mapping $\lbrack\nu\rbrack$ into $\lbrack\nu\rbrack$ and deleting all the cycles. The number of steps of the algorithm is proved to be asymptotically normal with mean and variance both of order $n^{1/2}$.

We consider a general single-server multiclass queueing system that incurs a delay cost $C_k(\tau_k)$ for each class $k$ job that resides $\tau_k$ units of time in the system. This paper derives a scheduling policy that minimizes the total cumulative delay cost when the system operates during a finite time horizon. Denote the marginal delay cost function and the (possibly nonstationary) average processing time of class $k$ by $c_k = C'_k$ and $1/\mu_k$, respectively, and let $a_k(t)$ be the "age" or time that the oldest class $k$ job has been waiting at time $t$. We call the scheduling policy that at time $t$ serves the oldest waiting job of that class $k$ with the highest index $\mu_k(t)c_k(a_k(t))$, the generalized $c\mu$ rule. As a dynamic priority rule that depends on very little data, the generalized $c\mu$ rule is attractive to implement. We show that, with nondecreasing convex delay costs, the generalized $c\mu$ rule is asymptotically optimal if the system operates in heavy traffic and give explicit expressions for the associated performance characteristics: the delay (throughput time) process and the minimum cumulative delay cost. The optimality result is robust in that it holds for a countable number of classes and several homogeneous servers in a nonstationary, deterministic or stochastic environment where arrival and service processes can be general and interdependent.

We consider the following distributed optimization problem: Given a set $X_1,\ldots,X_n$ of pairwise independent random variables and a target value $T$, a subset of the $X_i$'s must be selected whose sum is close to $T$. However, no cooperation is permitted in determining the set; each variable must be "programmed" in advance, joining or not joining according to its own value. Such conditions may arise, for example, when supply of some commodity is controlled at several random sources. Under these general conditions we show that the mean square error in approximating $T$ is always minimized by programming each $X_i$ to join if its value is between 0 and $\theta_i$, for appropriate choice of thresholds $\theta_i$. When in addition the variables are identically distributed, we give conditions under which the $\theta_i$'s must be equal. The case of uniform distribution on [0, 1] (in which our conditions fail) is analyzed in detail, showing the rather bizarre behavior of the $\theta_i$'s which may take place in general as the target value is gradually changed. We analyze also the problem in which the variables are permitted to contribute any part of themselves to the sum; here it turns out that in an optimal strategy, each program will be of the form "contribute the minimum of $X_i$ and $\gamma_i$," with all the $\gamma_i$'s equal in the i.i.d. case. Finally, we show how the original target shooting problem can be generalized to a kind of load balancing, where variables are assigned to different buckets, each with its own target, and the penalty is a weighted sum of squared errors. The surprising result here is that when the weights are equal, an optimal solution assigns variables only according to their signs.

Recursive methods are described for computing the frequency and distribution functions of trimmed sums of independent and identically distributed nonnegative integer-valued random variables. Surprisingly, for fixed arguments, these can be evaluated with just a finite number of arithmetic operations (and whatever else it takes to evaluate the common frequency function of the original summands). These methods give rise to very accurate computational algorithms that permit a delicate numerical investigation, herein described, of Feller's weak law of large numbers and its trimmed version for repeated St. Petersburg games. The performance of Stigler's theorem for the asymptotic distribution of trimmed sums is also investigated on the same example.