Relationship between the optimal solution and the empirical solution with finite samples in linear least square regressiondistinguishing two random distributionsPolynomial Regression and MulticollinearityLeast Squares in a Matrix FormExistence of a vector in a given basis of a vector space with increasing coordinatesHow can I prove that this matrix is nonsingular?Does this operation exist? What's its name?Prove that $[x_1,x_2,…,x_n]=[y_1,y_2,…,y_k]$.Jacobian matrix vs. Transformation matrixDetermine independence of functions of r.v.'s and compute joint distributionChange of basis in two dimensions, use order of transformation matrices?
Will the concrete slab in a partially heated shed conduct a lot of heat to the unconditioned area?
Is it appropriate to ask a former professor to order a library book for me through ILL?
Is it a Cyclops number? "Nobody" knows!
Short story about cities being connected by a conveyor belt
What is better: yes / no radio, or simple checkbox?
How to install "rounded" brake pads
How can I portion out frozen cookie dough?
The (Easy) Road to Code
Too soon for a plot twist?
Giving a talk in my old university, how prominently should I tell students my salary?
Are small insurances worth it?
Paper published similar to PhD thesis
An Undercover Army
Interpretation of linear regression interaction term plot
How does learning spells work when leveling a multiclass character?
What do you call someone who likes to pick fights?
Was this cameo in Captain Marvel computer generated?
What is Tony Stark injecting into himself in Iron Man 3?
Should I apply for my boss's promotion?
I am the light that shines in the dark
Is this Paypal Github SDK reference really a dangerous site?
Should I file my taxes? No income, unemployed, but paid 2k in student loan interest
Book where society has been split into 2 with a wall down the middle where one side embraced high tech whereas other side were totally against tech
How to distinguish easily different soldier of ww2?
Relationship between the optimal solution and the empirical solution with finite samples in linear least square regression
distinguishing two random distributionsPolynomial Regression and MulticollinearityLeast Squares in a Matrix FormExistence of a vector in a given basis of a vector space with increasing coordinatesHow can I prove that this matrix is nonsingular?Does this operation exist? What's its name?Prove that $[x_1,x_2,…,x_n]=[y_1,y_2,…,y_k]$.Jacobian matrix vs. Transformation matrixDetermine independence of functions of r.v.'s and compute joint distributionChange of basis in two dimensions, use order of transformation matrices?
$begingroup$
Consider a linear system $y_i=Wx_i+v_i$, where $x_iin R^dtimes1 sim N(0,Sigma_x)$, $v_iin R^ptimes1 sim N(0,Sigma_v)$, $y_iin R^ptimes1$ and $Win R^ptimes d$.
Now we consider a linear estimator $hatx_i=beta_1y_i+beta_0$. We know that the optimal theoretical least square solution of is given by:
$$beta^ast_1=Sigma_xW^top(WSigma_xW^top+Sigma_v)^-1$$
$$beta^ast_0=0$$
On the other hand, suppose we have a set consisting of $N$ ovservations pairs $(x_1,y_1),(x_2,y_2),..,(x_N,y_N)$. We denote $Y_N=[y_1,y_2,...,y_N]$, $X_N=[x_1,x_2,...,x_N]$ and $V_N=[v_1,v_2,...,v_N]$. the abover system and the linear estimator can be written as
$$Y_N=WX_N+V_N$$
$$hatX_N=[beta_1, beta_0]beginbmatrixY_N\1endbmatrix$$.
Therefore, the least square solution is given by :
$$[beta^astast_1, beta^astast_0]=X_N[Y^top_N, 1^top]left(beginbmatrixY_N\1endbmatrix[Y^top_N, 1^top]right)^-1$$
where $beta^astast_1$ and $beta^astast_0$ can be considered as random variables.
We know that when $Nto infty$, $beta^astast_1to beta^ast_1$.
My question is: when N is finite, can we describe the relationship between $beta^astast_1$ and $beta^ast_1$, e.g., the pdf of $beta^astast_1-beta^ast_1$, for a given $N$?
linear-algebra random-variables
New contributor
$endgroup$
add a comment |
$begingroup$
Consider a linear system $y_i=Wx_i+v_i$, where $x_iin R^dtimes1 sim N(0,Sigma_x)$, $v_iin R^ptimes1 sim N(0,Sigma_v)$, $y_iin R^ptimes1$ and $Win R^ptimes d$.
Now we consider a linear estimator $hatx_i=beta_1y_i+beta_0$. We know that the optimal theoretical least square solution of is given by:
$$beta^ast_1=Sigma_xW^top(WSigma_xW^top+Sigma_v)^-1$$
$$beta^ast_0=0$$
On the other hand, suppose we have a set consisting of $N$ ovservations pairs $(x_1,y_1),(x_2,y_2),..,(x_N,y_N)$. We denote $Y_N=[y_1,y_2,...,y_N]$, $X_N=[x_1,x_2,...,x_N]$ and $V_N=[v_1,v_2,...,v_N]$. the abover system and the linear estimator can be written as
$$Y_N=WX_N+V_N$$
$$hatX_N=[beta_1, beta_0]beginbmatrixY_N\1endbmatrix$$.
Therefore, the least square solution is given by :
$$[beta^astast_1, beta^astast_0]=X_N[Y^top_N, 1^top]left(beginbmatrixY_N\1endbmatrix[Y^top_N, 1^top]right)^-1$$
where $beta^astast_1$ and $beta^astast_0$ can be considered as random variables.
We know that when $Nto infty$, $beta^astast_1to beta^ast_1$.
My question is: when N is finite, can we describe the relationship between $beta^astast_1$ and $beta^ast_1$, e.g., the pdf of $beta^astast_1-beta^ast_1$, for a given $N$?
linear-algebra random-variables
New contributor
$endgroup$
add a comment |
$begingroup$
Consider a linear system $y_i=Wx_i+v_i$, where $x_iin R^dtimes1 sim N(0,Sigma_x)$, $v_iin R^ptimes1 sim N(0,Sigma_v)$, $y_iin R^ptimes1$ and $Win R^ptimes d$.
Now we consider a linear estimator $hatx_i=beta_1y_i+beta_0$. We know that the optimal theoretical least square solution of is given by:
$$beta^ast_1=Sigma_xW^top(WSigma_xW^top+Sigma_v)^-1$$
$$beta^ast_0=0$$
On the other hand, suppose we have a set consisting of $N$ ovservations pairs $(x_1,y_1),(x_2,y_2),..,(x_N,y_N)$. We denote $Y_N=[y_1,y_2,...,y_N]$, $X_N=[x_1,x_2,...,x_N]$ and $V_N=[v_1,v_2,...,v_N]$. the abover system and the linear estimator can be written as
$$Y_N=WX_N+V_N$$
$$hatX_N=[beta_1, beta_0]beginbmatrixY_N\1endbmatrix$$.
Therefore, the least square solution is given by :
$$[beta^astast_1, beta^astast_0]=X_N[Y^top_N, 1^top]left(beginbmatrixY_N\1endbmatrix[Y^top_N, 1^top]right)^-1$$
where $beta^astast_1$ and $beta^astast_0$ can be considered as random variables.
We know that when $Nto infty$, $beta^astast_1to beta^ast_1$.
My question is: when N is finite, can we describe the relationship between $beta^astast_1$ and $beta^ast_1$, e.g., the pdf of $beta^astast_1-beta^ast_1$, for a given $N$?
linear-algebra random-variables
New contributor
$endgroup$
Consider a linear system $y_i=Wx_i+v_i$, where $x_iin R^dtimes1 sim N(0,Sigma_x)$, $v_iin R^ptimes1 sim N(0,Sigma_v)$, $y_iin R^ptimes1$ and $Win R^ptimes d$.
Now we consider a linear estimator $hatx_i=beta_1y_i+beta_0$. We know that the optimal theoretical least square solution of is given by:
$$beta^ast_1=Sigma_xW^top(WSigma_xW^top+Sigma_v)^-1$$
$$beta^ast_0=0$$
On the other hand, suppose we have a set consisting of $N$ ovservations pairs $(x_1,y_1),(x_2,y_2),..,(x_N,y_N)$. We denote $Y_N=[y_1,y_2,...,y_N]$, $X_N=[x_1,x_2,...,x_N]$ and $V_N=[v_1,v_2,...,v_N]$. the abover system and the linear estimator can be written as
$$Y_N=WX_N+V_N$$
$$hatX_N=[beta_1, beta_0]beginbmatrixY_N\1endbmatrix$$.
Therefore, the least square solution is given by :
$$[beta^astast_1, beta^astast_0]=X_N[Y^top_N, 1^top]left(beginbmatrixY_N\1endbmatrix[Y^top_N, 1^top]right)^-1$$
where $beta^astast_1$ and $beta^astast_0$ can be considered as random variables.
We know that when $Nto infty$, $beta^astast_1to beta^ast_1$.
My question is: when N is finite, can we describe the relationship between $beta^astast_1$ and $beta^ast_1$, e.g., the pdf of $beta^astast_1-beta^ast_1$, for a given $N$?
linear-algebra random-variables
linear-algebra random-variables
New contributor
New contributor
edited 15 hours ago
czk32611
New contributor
asked 2 days ago
czk32611czk32611
11
11
New contributor
New contributor
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
czk32611 is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3137598%2frelationship-between-the-optimal-solution-and-the-empirical-solution-with-finite%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
czk32611 is a new contributor. Be nice, and check out our Code of Conduct.
czk32611 is a new contributor. Be nice, and check out our Code of Conduct.
czk32611 is a new contributor. Be nice, and check out our Code of Conduct.
czk32611 is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3137598%2frelationship-between-the-optimal-solution-and-the-empirical-solution-with-finite%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown