Fdtxjr

How do spaceships determine each other's mass in space?

Factor Rings over Finite Fields

Traveling to heavily polluted city, what practical measures can I take to minimize impact?

Did Amazon pay $0 in taxes last year?

I can't die. Who am I?

Is there a math expression equivalent to the conditional ternary operator?

Can I negotiate a patent idea for a raise, under French law?

Is it possible to clone a polymorphic object without manually adding overridden clone method into each derived class in C++?

Use Mercury as quenching liquid for swords?

Is it a Cyclops number? "Nobody" knows!

Converting from "matrix" data into "coordinate" data

Smooth vector fields on a surface modulo diffeomorphisms

Computation logic of Partway in TikZ

Should we avoid writing fiction about historical events without extensive research?

Idiom for feeling after taking risk and someone else being rewarded

Leveling the sagging side of the home

Called into a meeting and told we are being made redundant (laid off) and "not to share outside". Can I tell my partner?

What will happen if my luggage gets delayed?

Create chunks from an array

How is it possible to drive VGA displays at such high pixel clock frequencies?

Too soon for a plot twist?

What is the purpose of a disclaimer like "this is not legal advice"?

Can one live in the U.S. and not use a credit card?

Having the player face themselves after the mid-game

Book on Markov Decision Processes with many worked examples

What is the difference between all types of Markov Chains?Potential theory: discrete-time Markov processesReferences for basics of Piecewise-Deterministic Markov ProcessesReference request for stochastic process and applicationsReference request for this topicsGeneral state Markov Chains - referencesSoft Question - book recommendation - Stochastic ProcessesContinuous time Markov processes on general state spacesGood book for Integer/Non-Linear/Stochastic/Dynamic programing [Operations Research]Nonhomogenous Chains

2

$begingroup$

I am looking for a book (or online article(s)) on Markov decision processes that contains lots of worked examples or problems with solutions. The purpose of the book is to grind my teeth on some problems during long commutes.

The book must...

• have many examples using dynamic programming and the Bellman equation in discrete space and discrete time;


• touch on policy and value iteration, and perhaps their computational complexity and implementation,

and ideally would...

• use techniques from convex optimization, Lagrange multipliers, or combine with other computational techniques, such as sorting algorithms;


• cover more modern examples besides the usual queueing / inventory problems, such as reinforcement learning;


• contain lots of neat tricks and calculations.

An ideal book would not...

• be a Theorem-Proof bible, aiming to identify the weakest conditions for optimality results


• consider controlled Markov processes and viscosity solutions

I own Sheldon Ross's Applied probability models with optimization applications, in which there are several worked examples, a fair bit of good problems, but no solutions. I have been looking at Puterman's classic textbook Markov Decision Processes: Discrete Stochastic Dynamic Programming, but it is over 600 pages long and a bit on the "bible" side.

I'm looking for something more like Markov Chains and Mixing Times by Levin, Wilmer and Peres, but for MDPs. They have bite-sized chapters and a fair bit of explicit calculation. I like Norris's Markov Chains, which has some nice introductory exposition on potential theory, as well as the Applications chapter in David Williams's Probability with martingales. I do not mind if this "workbook" I am looking for is at an "advanced undergraduate" level, or directed at engineers or computer scientists.

reference-request markov-chains

share|cite|improve this question

asked yesterday

snarsnar

4,2841219

$endgroup$

add a comment | 

2

$begingroup$

I am looking for a book (or online article(s)) on Markov decision processes that contains lots of worked examples or problems with solutions. The purpose of the book is to grind my teeth on some problems during long commutes.

The book must...

• have many examples using dynamic programming and the Bellman equation in discrete space and discrete time;


• touch on policy and value iteration, and perhaps their computational complexity and implementation,

and ideally would...

• use techniques from convex optimization, Lagrange multipliers, or combine with other computational techniques, such as sorting algorithms;


• cover more modern examples besides the usual queueing / inventory problems, such as reinforcement learning;


• contain lots of neat tricks and calculations.

An ideal book would not...

• be a Theorem-Proof bible, aiming to identify the weakest conditions for optimality results


• consider controlled Markov processes and viscosity solutions

I own Sheldon Ross's Applied probability models with optimization applications, in which there are several worked examples, a fair bit of good problems, but no solutions. I have been looking at Puterman's classic textbook Markov Decision Processes: Discrete Stochastic Dynamic Programming, but it is over 600 pages long and a bit on the "bible" side.

I'm looking for something more like Markov Chains and Mixing Times by Levin, Wilmer and Peres, but for MDPs. They have bite-sized chapters and a fair bit of explicit calculation. I like Norris's Markov Chains, which has some nice introductory exposition on potential theory, as well as the Applications chapter in David Williams's Probability with martingales. I do not mind if this "workbook" I am looking for is at an "advanced undergraduate" level, or directed at engineers or computer scientists.

reference-request markov-chains

share|cite|improve this question

asked yesterday

snarsnar

4,2841219

$endgroup$

add a comment | 

$begingroup$

I am looking for a book (or online article(s)) on Markov decision processes that contains lots of worked examples or problems with solutions. The purpose of the book is to grind my teeth on some problems during long commutes.

The book must...

• have many examples using dynamic programming and the Bellman equation in discrete space and discrete time;


• touch on policy and value iteration, and perhaps their computational complexity and implementation,

and ideally would...

• use techniques from convex optimization, Lagrange multipliers, or combine with other computational techniques, such as sorting algorithms;


• cover more modern examples besides the usual queueing / inventory problems, such as reinforcement learning;


• contain lots of neat tricks and calculations.

An ideal book would not...

• be a Theorem-Proof bible, aiming to identify the weakest conditions for optimality results


• consider controlled Markov processes and viscosity solutions

I own Sheldon Ross's Applied probability models with optimization applications, in which there are several worked examples, a fair bit of good problems, but no solutions. I have been looking at Puterman's classic textbook Markov Decision Processes: Discrete Stochastic Dynamic Programming, but it is over 600 pages long and a bit on the "bible" side.

I'm looking for something more like Markov Chains and Mixing Times by Levin, Wilmer and Peres, but for MDPs. They have bite-sized chapters and a fair bit of explicit calculation. I like Norris's Markov Chains, which has some nice introductory exposition on potential theory, as well as the Applications chapter in David Williams's Probability with martingales. I do not mind if this "workbook" I am looking for is at an "advanced undergraduate" level, or directed at engineers or computer scientists.

reference-request markov-chains

share|cite|improve this question

asked yesterday

snarsnar

4,2841219

$endgroup$

share|cite|improve this question

asked yesterday

snarsnar

4,2841219

share|cite|improve this question

asked yesterday

snarsnar

4,2841219

asked yesterday

snarsnar

4,2841219

asked yesterday

snarsnar

4,2841219