Expected number of visits to a state in a Markov ChainExpected value of visits in a state of a discrete Markov chainFinite state space Markov chainMarkov chain and computing the expected number for fixed absorbing stateRelation between the expected number of visits to a state and reachability in a Markov chainMarkov Chain expected number of visitsExpected payoff of a 2-State Markov ChainExpected number of visits to state in Markov chainMarkov chain expected stepsExpected time of visits to a transient state in a Markov chainThe number of visits made by a Markov chain
India just shot down a satellite from the ground. At what altitude range is the resulting debris field?
What Brexit proposals are on the table in the indicative votes on the 27th of March 2019?
Balance Issues for a Custom Sorcerer Variant
What happens if you roll doubles 3 times then land on "Go to jail?"
Why escape if the_content isnt?
Is the destination of a commercial flight important for the pilot?
How do we know the LHC results are robust?
Trouble understanding the speech of overseas colleagues
Is expanding the research of a group into machine learning as a PhD student risky?
What does "I’d sit this one out, Cap," imply or mean in the context?
How can we prove that any integral in the set of non-elementary integrals cannot be expressed in the form of elementary functions?
Class Action - which options I have?
You cannot touch me, but I can touch you, who am I?
Is `x >> pure y` equivalent to `liftM (const y) x`
How does buying out courses with grant money work?
Is there a problem with hiding "forgot password" until it's needed?
Why Were Madagascar and New Zealand Discovered So Late?
Do the temporary hit points from Reckless Abandon stack if I make multiple attacks on my turn?
How does the UK government determine the size of a mandate?
Sequence of Tenses: Translating the subjunctive
Short story about space worker geeks who zone out by 'listening' to radiation from stars
How to Reset Passwords on Multiple Websites Easily?
Applicability of Single Responsibility Principle
Customer Requests (Sometimes) Drive Me Bonkers!
Expected number of visits to a state in a Markov Chain
Expected value of visits in a state of a discrete Markov chainFinite state space Markov chainMarkov chain and computing the expected number for fixed absorbing stateRelation between the expected number of visits to a state and reachability in a Markov chainMarkov Chain expected number of visitsExpected payoff of a 2-State Markov ChainExpected number of visits to state in Markov chainMarkov chain expected stepsExpected time of visits to a transient state in a Markov chainThe number of visits made by a Markov chain
$begingroup$
Attached above is an image from an exercice I have worked on, I have answered all questions but particularily struggled with question 4. When we were given the solutions, the correction simply said: $E_3(N(4)) = fracrho_341- rho44$ which equals infinity because the denominator is zero, but I struggle to understand how the expression was achieved. Is there a general formula for this? I couldn't find it in my class notes. How can we get the expected number of visits to class 3 starting at 2 , for instance?
stochastic-processes
asked Mar 17 at 20:43
bluemusebluemuse
976
$endgroup$
add a comment |
$begingroup$
Attached above is an image from an exercice I have worked on, I have answered all questions but particularily struggled with question 4. When we were given the solutions, the correction simply said: $E_3(N(4)) = fracrho_341- rho44$ which equals infinity because the denominator is zero, but I struggle to understand how the expression was achieved. Is there a general formula for this? I couldn't find it in my class notes. How can we get the expected number of visits to class 3 starting at 2 , for instance?
stochastic-processes
asked Mar 17 at 20:43
bluemusebluemuse
976
$endgroup$
add a comment |
$begingroup$
Attached above is an image from an exercice I have worked on, I have answered all questions but particularily struggled with question 4. When we were given the solutions, the correction simply said: $E_3(N(4)) = fracrho_341- rho44$ which equals infinity because the denominator is zero, but I struggle to understand how the expression was achieved. Is there a general formula for this? I couldn't find it in my class notes. How can we get the expected number of visits to class 3 starting at 2 , for instance?
stochastic-processes
asked Mar 17 at 20:43
bluemusebluemuse
976
$endgroup$
Attached above is an image from an exercice I have worked on, I have answered all questions but particularily struggled with question 4. When we were given the solutions, the correction simply said: $E_3(N(4)) = fracrho_341- rho44$ which equals infinity because the denominator is zero, but I struggle to understand how the expression was achieved. Is there a general formula for this? I couldn't find it in my class notes. How can we get the expected number of visits to class 3 starting at 2 , for instance?
stochastic-processes
stochastic-processes
asked Mar 17 at 20:43
bluemusebluemuse
976
asked Mar 17 at 20:43
bluemusebluemuse
976
asked Mar 17 at 20:43
bluemusebluemuse
976
asked Mar 17 at 20:43
bluemusebluemuse
976
asked Mar 17 at 20:43
bluemusebluemuse
976
976
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
If you start in state 3, then you enter state 4 with positive probability, and once you enter state 4, you stay there indefinitely. Therefore, you're expected to visit state 4 infinitely many times, if you start in state 3.
answered Mar 17 at 21:19
AlexandrosAlexandros
1,0111413
$endgroup$
$begingroup$
I do understand the intuition behind the infinite result, I just don't understand the equation given. And how can this result be achieved if state 4 was not an absorbing state, for instance?
$endgroup$
– bluemuse
Mar 17 at 21:25
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3152056%2fexpected-number-of-visits-to-a-state-in-a-markov-chain%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If you start in state 3, then you enter state 4 with positive probability, and once you enter state 4, you stay there indefinitely. Therefore, you're expected to visit state 4 infinitely many times, if you start in state 3.
answered Mar 17 at 21:19
AlexandrosAlexandros
1,0111413
$endgroup$
$begingroup$
I do understand the intuition behind the infinite result, I just don't understand the equation given. And how can this result be achieved if state 4 was not an absorbing state, for instance?
$endgroup$
– bluemuse
Mar 17 at 21:25
add a comment |
$begingroup$
If you start in state 3, then you enter state 4 with positive probability, and once you enter state 4, you stay there indefinitely. Therefore, you're expected to visit state 4 infinitely many times, if you start in state 3.
answered Mar 17 at 21:19
AlexandrosAlexandros
1,0111413
$endgroup$
$begingroup$
I do understand the intuition behind the infinite result, I just don't understand the equation given. And how can this result be achieved if state 4 was not an absorbing state, for instance?
$endgroup$
– bluemuse
Mar 17 at 21:25
add a comment |
$begingroup$
If you start in state 3, then you enter state 4 with positive probability, and once you enter state 4, you stay there indefinitely. Therefore, you're expected to visit state 4 infinitely many times, if you start in state 3.
answered Mar 17 at 21:19
AlexandrosAlexandros
1,0111413
$endgroup$
If you start in state 3, then you enter state 4 with positive probability, and once you enter state 4, you stay there indefinitely. Therefore, you're expected to visit state 4 infinitely many times, if you start in state 3.
answered Mar 17 at 21:19
AlexandrosAlexandros
1,0111413
answered Mar 17 at 21:19
AlexandrosAlexandros
1,0111413
answered Mar 17 at 21:19
AlexandrosAlexandros
1,0111413
answered Mar 17 at 21:19
AlexandrosAlexandros
1,0111413
1,0111413
$begingroup$
I do understand the intuition behind the infinite result, I just don't understand the equation given. And how can this result be achieved if state 4 was not an absorbing state, for instance?
$endgroup$
– bluemuse
Mar 17 at 21:25
add a comment |
$begingroup$
I do understand the intuition behind the infinite result, I just don't understand the equation given. And how can this result be achieved if state 4 was not an absorbing state, for instance?
$endgroup$
– bluemuse
Mar 17 at 21:25
$begingroup$
I do understand the intuition behind the infinite result, I just don't understand the equation given. And how can this result be achieved if state 4 was not an absorbing state, for instance?
$endgroup$
– bluemuse
Mar 17 at 21:25
$begingroup$
I do understand the intuition behind the infinite result, I just don't understand the equation given. And how can this result be achieved if state 4 was not an absorbing state, for instance?
$endgroup$
– bluemuse
Mar 17 at 21:25
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3152056%2fexpected-number-of-visits-to-a-state-in-a-markov-chain%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
