Fdtxjr

Which situations would cause a company to ground or recall a aircraft series?

How does Ehrenfest's theorem apply to the quantum harmonic oscillator?

Called into a meeting and told we are being made redundant (laid off) and "not to share outside". Can I tell my partner?

Can one live in the U.S. and not use a credit card?

Does Christianity allow for believing on someone else's behalf?

How do spaceships determine each other's mass in space?

Would an aboleth's Phantasmal Force lair action be affected by Counterspell, Dispel Magic, and/or Slow?

Does an unused member variable take up memory?

Trig Subsitution When There's No Square Root

Which classes are needed to have access to every spell in the PHB?

Damage bonus for different weapons

Getting the || sign while using Kurier

What is this diamond of every day?

Is it possible to avoid unpacking when merging Association?

The meaning of ‘otherwise’

Why do phishing e-mails use faked e-mail addresses instead of the real one?

In the late 1940’s to early 1950’s what technology was available that could melt ice?

How exactly does an Ethernet collision happen in the cable, since nodes use different circuits for Tx and Rx?

Having the player face themselves after the mid-game

What do *foreign films* mean for an American?

What is the generally accepted pronunciation of “topoi”?

What materials can be used to make a humanoid skin warm?

I can't die. Who am I?

How to write a chaotic neutral protagonist and prevent my readers from thinking they are evil?

How to prove a Markov chain is irreducible in general?

Irreducible Markov chainMarkov chain monte carloWhat is the difference between a reversible markov chain and a reversible in equilibrium markov chain?How to prove irreducible and aperiodic Markov chain is regularIs $X_n$ a Markov chain?Does product Markov chain hit diagonal?Is it true that a irreducible, recurrent markov chain with nonzero stationary distribution tends to stay in the same state?Are the stationary probabilities of an irreducible Markov chain zero, infinite or something else?Harmonic functions on an irreducible recurrent Markov chain are constantLet $Y_n=X_3n.$ Show that $Y_0,Y_1,…$ is a Markov chain.

1

$begingroup$

Consider the following example:

Given a random variable $X_n$ is $Markov(pi,P)$, where $P$ is irreducible and has an stationary distribution $pi$.

Then I can show that the time reversal of $X_n$, $Y_n=X_N-n$ is also a Markov chain. with transition probability $$P(Y_n+1=j|Y_n=i)=hatp_ij=fracpi_jp_jipi_i$$

However, is the Markov chain for $Y_n$ also irreducible?

My reasoningss:

From definition, In order to prove irreducibility I need to show that

$$P(Y_n=j | Y_0=i)=hatp^(n)_ij>0$$ for some $n>0$

The only thing I can think of now is

beginalign
P(Y_n=j | Y_0=i)&=hatp_ik_1hatp_k_1k_2hatp_k_2k_3...hatp_k_n-1j\& = fracpi_k_1p_k_1ipi_ifracpi_k_2p_k_2k_1pi_k_1fracpi_k_3p_k_3k_2pi_k_2...fracpi_jp_jk_n-1pi_k_n-1\
&=fracpi_jp^(n)_jipi_iendalign

Since the Markov chain for $X_n$ is irreducible, we know that $p^(n)_ji>0$ for some $n$. However, I cannot argue $pi_i$ and $pi_j$ are always greater than $0$, so we are not 100% sure that $P(Y_n=j | Y_0=i)=hatp^(n)_ij>0$, since it is possible that $pi_i=0$ or $pi_j=0$ for some $i$ and $j$.

So, how do we show that a Markov chain is irreducible in general? What am I missing here?

proof-verification proof-writing stochastic-processes markov-chains

share|cite|improve this question

edited yesterday

Raven Cheuk

asked yesterday

Raven CheukRaven Cheuk

1186

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  "However, I cannot argue $π_i$ and $π_j$ are always greater than $0$" - actually you can, because $X_n$ is irreducible. If $pi$ is an stationary distribution, then a state $i$ is transient if and only if $pi_i=0$. An irreducible Markov chain either has all recurrent states or all transient states.
  $endgroup$
  – Math1000
  yesterday
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Just to double confirm your last statement. Since "if the chain is irreducible, then either all states are recurrent or all are transient." If all states are transient, then the stationary distribution wouldn't exist. So, it should be an irreducible stationary Markov chain has no transient states, am I right?
  $endgroup$
  – Raven Cheuk
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  You are correct.
  $endgroup$
  – Math1000
  yesterday
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  There's so many concepts in Markov chain. If I don't know which concept to use, I can get stuck for hours. However, once someone points out which concept to use, it will suddenly become so obviously. Any tips to be so clear about Markov chain? btw, I like your profile pic, it's a Markov chain, isn't it?
  $endgroup$
  – Raven Cheuk
  yesterday
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Yes, I got the picture from Wikipedia :)
  $endgroup$
  – Math1000
  yesterday
  

  
  
  
  

add a comment | 

1

$begingroup$

Consider the following example:

Given a random variable $X_n$ is $Markov(pi,P)$, where $P$ is irreducible and has an stationary distribution $pi$.

Then I can show that the time reversal of $X_n$, $Y_n=X_N-n$ is also a Markov chain. with transition probability $$P(Y_n+1=j|Y_n=i)=hatp_ij=fracpi_jp_jipi_i$$

However, is the Markov chain for $Y_n$ also irreducible?

My reasoningss:

From definition, In order to prove irreducibility I need to show that

$$P(Y_n=j | Y_0=i)=hatp^(n)_ij>0$$ for some $n>0$

The only thing I can think of now is

beginalign
P(Y_n=j | Y_0=i)&=hatp_ik_1hatp_k_1k_2hatp_k_2k_3...hatp_k_n-1j\& = fracpi_k_1p_k_1ipi_ifracpi_k_2p_k_2k_1pi_k_1fracpi_k_3p_k_3k_2pi_k_2...fracpi_jp_jk_n-1pi_k_n-1\
&=fracpi_jp^(n)_jipi_iendalign

Since the Markov chain for $X_n$ is irreducible, we know that $p^(n)_ji>0$ for some $n$. However, I cannot argue $pi_i$ and $pi_j$ are always greater than $0$, so we are not 100% sure that $P(Y_n=j | Y_0=i)=hatp^(n)_ij>0$, since it is possible that $pi_i=0$ or $pi_j=0$ for some $i$ and $j$.

So, how do we show that a Markov chain is irreducible in general? What am I missing here?

proof-verification proof-writing stochastic-processes markov-chains

share|cite|improve this question

edited yesterday

Raven Cheuk

asked yesterday

Raven CheukRaven Cheuk

1186

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  "However, I cannot argue $π_i$ and $π_j$ are always greater than $0$" - actually you can, because $X_n$ is irreducible. If $pi$ is an stationary distribution, then a state $i$ is transient if and only if $pi_i=0$. An irreducible Markov chain either has all recurrent states or all transient states.
  $endgroup$
  – Math1000
  yesterday
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Just to double confirm your last statement. Since "if the chain is irreducible, then either all states are recurrent or all are transient." If all states are transient, then the stationary distribution wouldn't exist. So, it should be an irreducible stationary Markov chain has no transient states, am I right?
  $endgroup$
  – Raven Cheuk
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  You are correct.
  $endgroup$
  – Math1000
  yesterday
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  There's so many concepts in Markov chain. If I don't know which concept to use, I can get stuck for hours. However, once someone points out which concept to use, it will suddenly become so obviously. Any tips to be so clear about Markov chain? btw, I like your profile pic, it's a Markov chain, isn't it?
  $endgroup$
  – Raven Cheuk
  yesterday
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Yes, I got the picture from Wikipedia :)
  $endgroup$
  – Math1000
  yesterday
  

  
  
  
  

add a comment | 

$begingroup$

Consider the following example:

Given a random variable $X_n$ is $Markov(pi,P)$, where $P$ is irreducible and has an stationary distribution $pi$.

Then I can show that the time reversal of $X_n$, $Y_n=X_N-n$ is also a Markov chain. with transition probability $$P(Y_n+1=j|Y_n=i)=hatp_ij=fracpi_jp_jipi_i$$

However, is the Markov chain for $Y_n$ also irreducible?

My reasoningss:

From definition, In order to prove irreducibility I need to show that

$$P(Y_n=j | Y_0=i)=hatp^(n)_ij>0$$ for some $n>0$

The only thing I can think of now is

beginalign
P(Y_n=j | Y_0=i)&=hatp_ik_1hatp_k_1k_2hatp_k_2k_3...hatp_k_n-1j\& = fracpi_k_1p_k_1ipi_ifracpi_k_2p_k_2k_1pi_k_1fracpi_k_3p_k_3k_2pi_k_2...fracpi_jp_jk_n-1pi_k_n-1\
&=fracpi_jp^(n)_jipi_iendalign

Since the Markov chain for $X_n$ is irreducible, we know that $p^(n)_ji>0$ for some $n$. However, I cannot argue $pi_i$ and $pi_j$ are always greater than $0$, so we are not 100% sure that $P(Y_n=j | Y_0=i)=hatp^(n)_ij>0$, since it is possible that $pi_i=0$ or $pi_j=0$ for some $i$ and $j$.

So, how do we show that a Markov chain is irreducible in general? What am I missing here?

proof-verification proof-writing stochastic-processes markov-chains

share|cite|improve this question

edited yesterday

Raven Cheuk

asked yesterday

Raven CheukRaven Cheuk

1186

$endgroup$

share|cite|improve this question

edited yesterday

Raven Cheuk

asked yesterday

Raven CheukRaven Cheuk

1186

share|cite|improve this question

edited yesterday

Raven Cheuk

asked yesterday

Raven CheukRaven Cheuk

1186

asked yesterday

Raven CheukRaven Cheuk

1186

asked yesterday

Raven CheukRaven Cheuk

1186