Tagged Questions

Stochastic processes (with either discrete or continuous time dependence) on a discrete (finite or countably infinite) state space in which the distribution of the next state depends only on the current state. For Markov processes on continuous state spaces please use (markov-process) instead.

A mosquito is walking at random on the nonnegative number line. She starts at $1$. When she is at $0$, she always takes a step $1$ unit to the right, but, from any positive position on the line, she ...

What is a good reference for general state space Markov chains? Is there a reference which assumes only familiarity with finite/countable state space Markov chains and then extends the results (e.g., ...

Arrivals occur at rate $\lambda$ according to a Poisson process the
service time have an exponential distribution with parameter $1/\mu$ in an M/M/1 queue, where $\mu$ is the mean service rate where ...

Suppose I have two (independent) discrete-time and space, preferably non-homogeneous Markov chains $\Gamma^{(i)}=\{\gamma_1^{(i)},\gamma_2^{(i)},...\}, \ i=1,2$ and I want to find a way to check when ...

Let $P_{ij}$ a transition matrix, a class $C$ is closed if given two different states $i$ and $j$ $$i\in C, i\rightarrow j\Rightarrow j\in C$$
If a Markov Chain is irreducible the transition matrix ...

I have two Markov Chains, and they exhibit some correlation between them. For instance, when Chain A moves to state i, there is a high likelihood that Chain B moves to state j. How would I go about ...

Consider a vertex-transitive nonameanable graph. Consider a site $x$ having a graph distance $d$ from the origin and let $X(n)$ be a random walk starting from $x$. Is there a general upper bound as a ...

Let $X_n$ be a random variable taking on one of three values $a,b$ or $c$ over time. That is, for each $n \in \mathbb{N}$, we have $X_n \in \{a,b,c\}$.
Also, for each $n \in \mathbb{N}$, let $F_n$ be ...

Problem: Suppose the employment situation in a country evolves in the following manner: from all the people that are unemployed in some year, $1/16$ of them finds a job next year. Furthermore, from ...

Suppose I have an infinite number of time steps $X_0,\ldots,X_i,\ldots$, where each $X_i$ is an infinite dimensional random vector consisting of 0's and 1's.
I now specify $P(X_i|X_{(i-1)})$ and an ...

I have a $M/M/1$ queueing system that is described below:
There are two types of customers in the system with different arrival
rates, $\lambda_{sg}$ and $\lambda_{sb}$.
Service rate is $\mu$.
Type ...

I can represent stochastically-articulated sequences of states using a transition matrix M where a given entry in cell (i,j) corresponds to the probability of state j given that the current (or, most ...

I have an absorbing Markov Chain that has 5 states, that can be envisioned as 5 nodes in a straight line. The left and right most nodes are the absorbing states. Everything starts at the middle node ...

Let M be a Markov matrix with rows summing to 1.
The interpretation of the left eigenvectors of M is clear. For instance, the first left eigenvector is the stationary distribution of M. And the left ...

I am struggling with how to calculate the values of a Markov matrix which has multiple states.
For example,
Imagine an unfair 6 sided dice. The chance of rolling a 1,2,3,4,5 or 6 is
0.3, 0.25, 0.2, ...

Let $(X_n)_{n \geq 0}$ be a irreducible, positive recurrent Markov chain. We have a theorem that states that the unique stationary distribution is then given by $$\pi(x)= \frac{1}{E_x[H_x]},$$
where ...

I am trying to solve a standard ETA on a birth-death process with $n$ states $\in \{0,\cdots,n-1\}$ where state $n-1$ is absorbing. Also $\mu_i$ is the expected time to absorption starting at state ...

Forgive me for my weak statistic background, hopefully what I'm asking makes sense. So some quick background, I have one markov chain from a data set and many additional chains that I'm producing from ...