Fdtxjr

Let $(X_n)_ninmathbbN $ be a sequence of randon variables defined by
$$X_n:=mboxmaximum score obtained after n mbox throws of a fair dice.$$

I need show that $(X_n)_ninmathbbN $ is a Markov Chain. In this sense, we have to show that
$$P(X_n+1=i_n+1|X_1=i_1,X_2=i_2,ldots,X_n=i_n)=P(X_n+1=i_n+1|X_n=i_n)tag1$$
I know that if we consider the random variable $Y_k$ which are the result of $k$th throw of our fair dice, then $(Y_k)_kinmathbbN$ is iid with uniform distribution over $left1,2,3,4,5,6right$, so, we have $X_n=maxleftY_1,ldots,Y_nright$. Furthermore, the cumulative distribution of $X_n$ is
$$F_X_n(k)=left{beginarrayll
0 & mathrmif: k<1 \
fraci^n6^n & mathrmif: ileq k <i+1 :: mathrmfor: i=1,2,3,4,5.\
1 & mathrmif:: kgeq 6
endarrayright.$$

Therefore, (1) is equivalent to
$$P(maxleftY_1,ldots,Y_n,Y_n+1right=i_n+1|Y_1=i_1,maxleftY_1,Y_2right=i_2,ldots,maxleftY_1,ldots,Y_nright=i_n)=P(maxleftY_1,ldots,Y_n,Y_n+1right=i_n+1|maxleftY_1,ldots,Y_nright=i_n)$$
I have not been able to prove this last, I need help in this matter.

Here is the Markov transition graph for your problem, where the states are labeled by the current maximum and $0$ represents the initial state:

All you need do is calculate the probabilities for the transitions and see that the final state (the sole attractor) is (in probability) $6$. Each transition depends solely on the current state, not the sequence that led to the current state.

Note that $$X_n+1=max Y_1,dots,Y_n+1 = max maxY_1,dots,Y_n,Y_n+1 = max X_n,Y_n+1.$$

So $X_n+1$ only depends on $X_n$ and none of the $X_i$ for $i<n$.

Markov Matrix for one-step transition probability would be

beginbmatrix
frac16 & frac16 & frac16 & frac16 & frac16 & frac16 \
0 & frac26 & frac16 & frac16 & frac16 & frac16 \
0 & 0 & frac36 & frac16 & frac16 & frac16 \
0 & 0 & 0 & frac46 & frac16 & frac16 \
0 & 0 & 0 & 0 & frac56 & frac16 \
0 & 0 & 0 & 0 & 0 & 1
endbmatrix

The long term limiting probabilities would be as followed
beginalign
pi_6 &= 1 & pi_1 &= pi_2 = pi_3 = pi_4 = pi_5 = 0
endalign