Fdtxjr

How will losing mobility of one hand affect my career as a programmer?

How can a jailer prevent the Forge Cleric's Artisan's Blessing from being used?

Products and sum of cubes in Fibonacci

What is the oldest known work of fiction?

Why does John Bercow say “unlock” after reading out the results of a vote?

Finding all intervals that match predicate in vector

I'm in charge of equipment buying but no one's ever happy with what I choose. How to fix this?

Why "be dealt cards" rather than "be dealing cards"?

Why did Kant, Hegel, and Adorno leave some words and phrases in the Greek alphabet?

Can I Retrieve Email Addresses from BCC?

Why Were Madagascar and New Zealand Discovered So Late?

Teaching indefinite integrals that require special-casing

Ways to speed up user implemented RK4

How does it work when somebody invests in my business?

Should my PhD thesis be submitted under my legal name?

Can somebody explain Brexit in a few child-proof sentences?

What would be the benefits of having both a state and local currencies?

Displaying the order of the columns of a table

Valid Badminton Score?

Lay out the Carpet

Is there an Impartial Brexit Deal comparison site?

How can I replace every global instance of "x[2]" with "x_2"

How could Frankenstein get the parts for his _second_ creature?

There is only s̶i̶x̶t̶y one place he can be

The Bayes predictor of the square loss is $Bbb E_P[Ymid X=x]$?

What is the derivative of the cross entropy loss when extending an arbitrary predictor for multi class classification?Mean Square Error Minimization Conditioned On Multivariate Normal Random VariablesNotation in the derivative of the hinge loss functionsquare loss function in classificationProving that the Bayes optimal predictor is in fact optimalLet X : Ω → R be a random variable on a probability space that is normally distributed.The Bayes optimal predictor is optimalhinge loss vs. square of hinge loss componentsBayes (optimal) classifier for binary classification with asymmetric loss functionInequality for Log Concave Distributions

0

$begingroup$

Let $(X,Y) in Bbb X times Bbb Y$ be jointly distributed according
to distribution $P$. Let $h: Bbb X rightarrow tilde Bbb Y$,
where $tilde Bbb Y$ is a predicted output. $ $Let $L(h,P) equiv
Bbb E_P[l(Y, h(X))]$ where $l$ is some loss function.

Show that $f = arg min_h L(h,P) = Bbb E_p[Y mid X = x]$ if $l$ is the
square loss function: $l(Y, h(X)) = (y - h(x))^2$

I figured I show this by showing any other $h$ leads to a larger $L(h,P)$ than $Bbb E_P[Ymid X=x]$.

I start with $$Bbb E_P[(y - Bbb E_p[Ymid X=x])^2] le Bbb E_P[(y - h(x))^2]$$

Then expanding we have:

$$Bbb E_P[y^2-2yBbb E_p[Y|X=x] + Bbb E_P[Ymid X=x]^2] le Bbb E_P[y^2 - 2yh(x) + h(x)^2]$$

And simplifying:

$$-2Bbb E_P[y]Bbb E_p[Ymid X=x] + Bbb E_P[Ymid X=x]^2 le -2Bbb E_P[yh(x)] + Bbb E_P[h(x)^2]$$

But from here I'm a little stuck as to how to continue.

Does anyone have any ideas?

probability machine-learning

share|cite|improve this question

edited Mar 17 at 12:39

Bernard

123k741117

asked Mar 17 at 12:33

Oliver GOliver G

1,3651632

$endgroup$

This question has an open bounty worth +100
reputation from Oliver G ending ending at 2019-04-01 18:27:16Z">in 6 days.

Looking for an answer drawing from credible and/or official sources.

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  You want to show that the conditional expectation minimises the square loss. You can find discussion of this here: stats.stackexchange.com/questions/71863/….
  $endgroup$
  – Minus One-Twelfth
  Mar 17 at 14:04
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  "I start with" You don't start with your desired conclusion. You start with what you know.
  $endgroup$
  – leonbloy
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  The link given by MinusOne-Twelfth has effectively answered the question in details. Is there anything else you'd like to know?
  $endgroup$
  – Saad
  yesterday
  

  
  
  
  

add a comment | 

0

$begingroup$

Let $(X,Y) in Bbb X times Bbb Y$ be jointly distributed according
to distribution $P$. Let $h: Bbb X rightarrow tilde Bbb Y$,
where $tilde Bbb Y$ is a predicted output. $ $Let $L(h,P) equiv
Bbb E_P[l(Y, h(X))]$ where $l$ is some loss function.

Show that $f = arg min_h L(h,P) = Bbb E_p[Y mid X = x]$ if $l$ is the
square loss function: $l(Y, h(X)) = (y - h(x))^2$

I figured I show this by showing any other $h$ leads to a larger $L(h,P)$ than $Bbb E_P[Ymid X=x]$.

I start with $$Bbb E_P[(y - Bbb E_p[Ymid X=x])^2] le Bbb E_P[(y - h(x))^2]$$

Then expanding we have:

$$Bbb E_P[y^2-2yBbb E_p[Y|X=x] + Bbb E_P[Ymid X=x]^2] le Bbb E_P[y^2 - 2yh(x) + h(x)^2]$$

And simplifying:

$$-2Bbb E_P[y]Bbb E_p[Ymid X=x] + Bbb E_P[Ymid X=x]^2 le -2Bbb E_P[yh(x)] + Bbb E_P[h(x)^2]$$

But from here I'm a little stuck as to how to continue.

Does anyone have any ideas?

probability machine-learning

share|cite|improve this question

edited Mar 17 at 12:39

Bernard

123k741117

asked Mar 17 at 12:33

Oliver GOliver G

1,3651632

$endgroup$

This question has an open bounty worth +100
reputation from Oliver G ending ending at 2019-04-01 18:27:16Z">in 6 days.

Looking for an answer drawing from credible and/or official sources.

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  You want to show that the conditional expectation minimises the square loss. You can find discussion of this here: stats.stackexchange.com/questions/71863/….
  $endgroup$
  – Minus One-Twelfth
  Mar 17 at 14:04
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  "I start with" You don't start with your desired conclusion. You start with what you know.
  $endgroup$
  – leonbloy
  yesterday
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  The link given by MinusOne-Twelfth has effectively answered the question in details. Is there anything else you'd like to know?
  $endgroup$
  – Saad
  yesterday
  

  
  
  
  

add a comment | 

$begingroup$

Let $(X,Y) in Bbb X times Bbb Y$ be jointly distributed according
to distribution $P$. Let $h: Bbb X rightarrow tilde Bbb Y$,
where $tilde Bbb Y$ is a predicted output. $ $Let $L(h,P) equiv
Bbb E_P[l(Y, h(X))]$ where $l$ is some loss function.

Show that $f = arg min_h L(h,P) = Bbb E_p[Y mid X = x]$ if $l$ is the
square loss function: $l(Y, h(X)) = (y - h(x))^2$

I figured I show this by showing any other $h$ leads to a larger $L(h,P)$ than $Bbb E_P[Ymid X=x]$.

I start with $$Bbb E_P[(y - Bbb E_p[Ymid X=x])^2] le Bbb E_P[(y - h(x))^2]$$

Then expanding we have:

$$Bbb E_P[y^2-2yBbb E_p[Y|X=x] + Bbb E_P[Ymid X=x]^2] le Bbb E_P[y^2 - 2yh(x) + h(x)^2]$$

And simplifying:

$$-2Bbb E_P[y]Bbb E_p[Ymid X=x] + Bbb E_P[Ymid X=x]^2 le -2Bbb E_P[yh(x)] + Bbb E_P[h(x)^2]$$

But from here I'm a little stuck as to how to continue.

Does anyone have any ideas?

probability machine-learning

share|cite|improve this question

edited Mar 17 at 12:39

Bernard

123k741117

asked Mar 17 at 12:33

Oliver GOliver G

1,3651632

$endgroup$

share|cite|improve this question

edited Mar 17 at 12:39

Bernard

123k741117

asked Mar 17 at 12:33

Oliver GOliver G

1,3651632

share|cite|improve this question

edited Mar 17 at 12:39

Bernard

123k741117

asked Mar 17 at 12:33

Oliver GOliver G

1,3651632

edited Mar 17 at 12:39

Bernard

123k741117

edited Mar 17 at 12:39

Bernard

123k741117

asked Mar 17 at 12:33

Oliver GOliver G

1,3651632

asked Mar 17 at 12:33

Oliver GOliver G

1,3651632