Geometric ergodicity and hybrid markov chains by gareth o. Variance bounding and geometric ergodicity of markov chain monte carlo kernels for approximate bayesian computation. The latter condition is verified for the timediscretised twocentre open jackson network. We argue that the spectral theory of nonreversible markov chains may often be more effectively cast within the framework of the naturally associated weightedl. We study the question of geometric ergodicity in a class of markov chains on the state space of nonnegative integers for which, apart from a finite number. Multiplicative ergodicity and large deviations for an. It intro duces a new condition called strong convergence. Extremal indices, geometric ergodicity of markov chains. Geometric ergodicity in a class of denumerable markov chains. Spectral theory and limit theorems for geometrically ergodic. If the doeblin condition is satisfied, then for the constants in 2 the relation holds. We also characterize the geometric ergodicity of xn in terms of hitting time.
It establishes the conditions under which a markov chain can be analyzed to determine its steady state behavior. We study the question of geometric ergodicity in a class of markov chains on the state space of nonnegative integers for which, apart from a finite number of boundary rows and columns, the elements pjk of the onestep transition matrix are of the form c kj where c k is a probability distribution on the set of integers. The hypotheses above imply that there is a unique stationary measure for the markov chain. In the paper, we will study perturbation theory and uniform ergodicity for discretetime general markov chains. Quasistationarity of discretetime markov chains with.
We consider the poisson equations for denumerable markov chains with unbounded cost functions. Thus, this papers assumptions are not uniformly weaker nor stronger than those in jmmd15. Geometric ergodicity along with a moment condition results in the existence of a markov chain central limit theorem for. We consider both regular and singular perturbations. Geometric ergodicity of random scan gibbs samplers for. Perturbation theory and uniform ergodicity for discrete. Representation theory for a class of denumerable markov chains. It is named after the russian mathematician andrey markov markov chains have many applications as statistical models of realworld processes, such as studying cruise. Geometric ergodicity is a useful stability property for mcmc samplers jones and hobert, 2001. A markov chain is called an ergodic or irreducible markov chain if it is possible to eventually get from every state to every other state with positive probability. It introduces a new condition called strong convergence. However, this does not necessarily hold for the same weight function.
Before moving on, let us introduce the basic setup, the readers are urged to refer 4,21. Approximations of geometrically ergodic markov chains je rey negrea and je rey s. At the heart of the paper is the optimal scaling analysis, in which we show that, for inhomogeneous product target distribution at stationary, the mana proposal variance scale like o n for the average acceptance rate to be bounded away from zero, as the dimension napproaches in. This formula says that the number of visits to i is a geometric 1. Geometric ergodicity of a randomwalk metorpolis algorithm. Tweedie ergodicity and recurrence of marko v chains theory problem, and by a brief note on conditions for ergodicity and recurrence previously known in the case of countable x. In this dissertation, we establish verifiable conditions under which geometric ergodicity is guaranteed for gibbs samplers in a general model setting. We argue that the spectral theory of nonreversible markov chains may often be more effectively cast within the framework of the naturally associated. Geometric ergodicity in denumerable markov chains, the quarterly journal of mathematics, volume, issue 1, 1 january 1962, pages 728, s. Geometric ergodicity in denumerable markov chains the.
We then restrict ourselves to markov chains which take values in a nite state space. Geometric ergodicity through variable transformation in. We also characterize the geometric ergodicity of x, in terms of the hitting time distributions of the socalled small sets. The paper is concluded by are application of the results to a queueing 388 r. And as in the main stream markov chain theory, they use the term geometric ergodicity for convergence involving the total variation norm l1 norm. The measure is always ergodic for the shift map if the associated markov chain is irreducible any state can be reached with positive probability form any other state in a finite number of steps. The most commonly discussed coefficient of ergodicity is the hajnal coefficient, tipsup f ipispjsl2. Three kinds of geometric convergence for markov chains. Our approach is different and more direct, and enables us to establish geometric ergodicity for arbitrary n and p. It follows as a corollary that, if a markov chain is aperiodic, positiverecurrent, and geometrically ergodic, we can again give the parameters pti a common value. A markov chain is geometrically ergodic if it converges to its invariant distribution at a geometric rate in total variation norm. We study geometric ergodicity of deterministic and random scan versions of the twovariable gibbs sampler. In exchange, our results require that the approximating kernel be close in the operator norm induced by l 2.
This is especially true because most instances of the metropolishastingsgreen algorithm are reversible or can be made to be reversible geyer, 2011, sections 1. Jun 29, 2009 geometric ergodicity and the spectral gap of nonreversible markov chains article pdf available in probability theory and related fields 1541 june 2009 with 76 reads how we measure reads. The first objective of this paper is to develop an analysis for average optimality without assuming a special markov chain structure. Variance bounding and geometric ergodicity of markov chain. Error bounds for approximations of geometrically ergodic. This paper studies two properties of the set of markov chains induced by the deterministic policies in a markov decision chain. Under minor additional assumptions the solution is.
Geometric ergodicity is important because rwma is reversible markov chain and theorem roberts and rosenthal, 1997 a reversible markov chain has a central limit theorem for all squareintegrable functionals if and only if it is geometrically ergodic. Roberts and rosenthal, 2004 in that it ensures rapid convergence of the markov chain since t markov processes 7 is the. Ergodicity theorem the foundation of markov chain theory is the ergodicity theorem. Geometric ergodicity refers to the rate of convergence to the invariant distribution. For countable state space chains in discretetime, multiplicative results. If a markov chain is geometrically ergodic, then the markov chain is also geometrically recurrent see theorem 1. When the weight function is bounded, then w geometric ergodicity is equivalent tow geometric.
By the latter we mean that transition probabilities of a markov chain, with several ergodic classes, are perturbed such that rare transitions among the different ergodic classes of the unperturbed chain are allowed. Solutions to the poisson equations exist in the banach space of bounded realvalued functions with respect to a weighted supremum norm such that the markov chain is geometrically ergodic. Markov chains discretetime markov processes on discrete state spaces keywords convergence rates irreducible markov processes geometric ergodicity lyapunov functions drift conditions resolvents generators. Theorem 6 that under suitable conditions, hybrid chains will \inherit the geometric ergodicity of their constituent chains. Processes which are ergodic but not geometrically ergodic often require a more subtle treatment than we provide here. This suggests the possibility of establishing the geometric ergodicity of large and complicated markov chain algorithms, simply by verifying the geometric ergodicity of the simpler chains which give rise to them. The mathematical tools needed for the task are developed in the first chapter.
In the second chapter the solidarity nature of geometric ergodicity within an irreducible class is. Part i studies geometric ergodicity of denumerable, multichain markov chains. The key theorem of this paper states a geometric ergodicity to be equivalent to weak a geometric recurrence for denumerable, aperiodic markov chains with a multichain structure. Miller, geometric ergodicity in a class of denumerable markov chains, z. Ergodicity of stochastic processes and the markov chain. The kendal theorem states that under some mild conditions imposed on a probability distribution on positive integers i. A markov chain is called an ergodic chain if it is possible to go from every state to every state not necessarily in one move. The extremal index can be expressed in terms of this random walk. On the geometric ergodicity of twovariable gibbs samplers. Transition functions and markov processes 7 is the. In this setting, we use a di erent method, as in, to prove a central limit theorem for functions of ergodic markov chains, where we have to impose. We will encounter this situation in section 5 for markov chains arising from the random walk metropolis algorithm. We show that many common markov chain monte carlo kernels used to facilitate inference in this setting can fail to be variance bounding and hence geometrically ergodic, which can have.
A geometrically ergodic chain converges to its target distribution at a geometric rate. Subgeometric rates of convergence in wasserstein distance. Convergence rate, geometric drift condition, markov chain. A class of denumerable markov chains 503 next consider y x. Rosenthal october 23, 2019 abstract a common tool in the practice of markov chain monte carlo is to use approximating transition kernels to speed up computation when the desired kernel is slow to evaluate or intractable. The kendal theorem states that under mild conditions imposed on a probability distribution on. Approximations of geometrically ergodic markov chains. On the uniqueness of solutions to the poisson equations. Such a process may be described as a general random walk on the non. Nummelin, a splitting technique for harris recurrent markov chains, z. We give a su cient condition which simultaneously guarantees both versions are geometrically ergodic. Each x i takes values in the state space, xand we will use bto represent the borel. Geometric ergodicity for some spacetime maxstable markov.
Approximate bayesian computation has emerged as a standard computational tool when dealing with intractable likelihood functions in bayesian inference. Renewal theory and computable convergence rates for. Millergeometric ergodicity in a class of denumerable markov chains. Strong ergodicity and geometric ergodicity in the case of an irreducible homogeneous markov chain, a strongly ergodic chain is one satisfying sup c pi. As a corollary, we obtain a tintegrated form of geiii for geometrically ergodic chains. On ergodicity and recurrence properties of a markov chain. Geometric ergodicity and the spectral gap of nonreversible markov chains i. Geometric ergodicity and the spectral gap of nonreversible markov chains article pdf available in probability theory and related fields 1541 june 2009 with 76 reads how we measure reads. A markov chain is transient if all of its states are transient. Markov chains, wasserstein distance, subgeometric ergodicity, markov chain monte carlo in in. We give a test function criterion for geometric ergodicity and apply it to random walks on the. A markov chain is a stochastic model describing a sequence of possible events in which the probability of each event depends only on the state attained in the previous event.
Large deviations principles for functionals of a markov chain and the empirical measures are derived in section 6. Thus it generalises the classical theorem by doeblin, yosida and kakutani on strong ergodicity to more general matrix norms for denumerable markov chains cf. Geometric ergodicity and hybrid markov chains roberts. Various notions of geometric ergodicity for markov chains on general.
Hence, the corresponding twodimensional markov chain is. Geometric ergodicity of harris recurrent marcov chains. Verejones, geometric ergodicity in denumerable markov chains, quart. Meyny june 27, 2009 abstract we argue that the spectral theory of nonreversible markov chains may often be more e ectively cast within the framework of the naturally associated weightedl 1 space lv, instead of the usual hilbert space l 2 l. On ergodicity and recurrence properties of a markov chain by. We then apply these results to a collection of chains commonly used in markov chain monte carlo simulation algorithms, the socalled hybrid chains. There are central limit theorems that dont use geometric. Representation theory for a class of denumerable markov. Introduction convergence of general statespace markov chains in total variation distance or vtotal variation has been studied by many authors. This is an electronic reprint of the original article published by the institute of mathematical statistics in the annals of applied probability, 2005, vol. In this paper we consider a discretetime markov decision chain with a denumerable state space and compact action sets and we assume that for all states the rewards and transition probabilities depend continuously on the actions. Pdf geometric ergodicity and the spectral gap of non.
This observation is, in part, based on the following results. It can be interpreted as a strong version of stability, as it implies that the markov chains generated by the deterministic stationary policies are uniformly stable. A markov chain can be characterized by the properties of its states. On the uniqueness of solutions to the poisson equations for. Nummelin, strong ratio limit theorems for cprecurrent markov chains, ann. On the relation between recurrence and ergodicity properties. Variable transformation to obtain geometric ergodicity in. Approximate bayesian computation has emerged as a standard computational tool when dealing with intractable likelihood functions in bayesian inferenc.
Average, sensitive and blackwell optimal policies in. This condition requires the existence of a vector on the state space, with components greater or equal to 1, such that the nstep transition probability matrix converges geometrically fast to the stationary matrix. Geometric ergodicity of harris recurrent marcov chains with. Thus geometric ergodicity is a very desirable property for a markov chain to have. Geometric ergodicity and the spectral gap of nonreversible markov chains article pdf available in probability theory and related fields 1541. Geometric ergodicity of the randomwalkbased metropolis algorithm on r k has previously been studied by roberts and tweedie 1996. Geometric ergodicity, renewal theory, reversible markov chain, markov chain monte carlo, metropolishastings algorithm, spectral gap. A question of increasing importance in the markov chain monte carlo literature gelfand and smith, 1990. Verejones, on quasistationary distributions in discretetime markov chains with a denumerable infinity of states, j. Perturbation analysis for denumerable markov chains 841 2. A sufficient condition for geometric ergodicity of an ergodic markov chain is the doeblin condition see, for example, which for a discrete finite or countable markov chain may be stated as follows. Geometric ergodicity of metropolis algorithms sciencedirect. Is ergodic markov chain both irreducible and aperiodic or.
We study the parametric perturbation of markov chains with denumerable state spaces. The wandering mathematician in previous example is an ergodic markov chain. Pdf perturbation analysis for denumerable markov chains. Tweedie, some ergodic properties of the feller minimal process, quart. The first condition generalises geometric as well as strong ergodicity. A markov chain is called a regular chain if some power of the transition matrix has only positive elements. The kendalls theorem and its application to the geometric. The markov chain formed by strategy x achieves weak ergodicity if lim 5 t. In continuoustime, it is known as a markov process. Ergodicity for countable inhomogeneous markov chains. Assuming only that the markov chain is geometrically ergodic and. Moreover, it is shown how a certain sequence of isoperimetric constants can be used to obtain bounds for the rates of geometric convergence, and prove that these bounds are sharp in some cases. Geometric ergodicity in the context of markov chains, ergodic means aperiodic. Geometric ergodicity of gibbs samplers in bayesian.
Smith and roberts, 1993 is the issue of geometric ergodicity of markov chains tierney, 1994, section 3. Various notions of geometric ergodicity for markov chains on general state spaces exist. The main result of the present paper is the geometric ergodicity of the just mentioned markov chains. Geometric ergodicity and the spectral gap of nonreversible.