Fdtxjr

Term for the "extreme-extension" version of a straw man fallacy?

Was a professor correct to chastise me for writing "Dear Prof. X" rather than "Dear Professor X"?

How do I go from 300 unfinished/half written blog posts, to published posts?

Can the discrete variable be a negative number?

Class Action - which options I have?

Flow chart document symbol

Did Dumbledore lie to Harry about how long he had James Potter's invisibility cloak when he was examining it? If so, why?

Hostile work environment after whistle-blowing on coworker and our boss. What do I do?

What does 算不上 mean in 算不上太美好的日子?

How to check is there any negative term in a large list?

Inappropriate reference requests from Journal reviewers

How do scammers retract money, while you can’t?

Avoiding estate tax by giving multiple gifts

Pole-zeros of a real-valued causal FIR system

Two monoidal structures and copowering

A Rare Riley Riddle

Sort a list by elements of another list

What is paid subscription needed for in Mortal Kombat 11?

A problem in Probability theory

Customer Requests (Sometimes) Drive Me Bonkers!

Is this apparent Class Action settlement a spam message?

Is there a korbon needed for conversion?

Is exact Kanji stroke length important?

Short story about space worker geeks who zone out by 'listening' to radiation from stars

Covariance matrix and projection

Covariance matrix of data projected onto eigenvectors is diagonal.Covariance matrix of Y when we have the covariance matrix of XHow can the variance of data be represented as this product of the principal direction and the covariance matrix?Covariance matrix as linear transformationI - P projection matrixEigenvectors of covariance matrix vs Eigenvectors of transformation matrix and PCACovariance matrix multiplied by eigenvectorHow to compute projection matrix for gradient projection algorithm?Can off-diagonals be nonzero for covariance matrix after PCA?Mean vector and covariance matrix

1

$begingroup$

I have troubles understanding a geometrical meaning of a covariance matrix.

Let's say we have a data set containing two points (-1,1), (-1,2) and write them in to the matrix

$$D = beginbmatrix
-1 & -1\
1 & 2
endbmatrix.
$$

Then we'll choose the unit-vector $v = left( frac1sqrt2, frac1sqrt2 right)^T$.

And now, in this article is the following statement:

So, if we would like to represent the covariance matrix with a vector
and its magnitude, we should simply try to find the vector that points
into the direction of the largest spread of the data, and whose
magnitude equals the spread (variance) in this direction.

If we define this vector as $v$, then the projection of our data $D$
onto this vector is obtained as $v^TD$, and the
variance of the projected data is $v^TSigma v$.

The covariance matrix $Sigma$ looks like this:

$$
Sigma = beginbmatrix
0 & 0\
0 & frac12
endbmatrix
$$

Also, the projection of $D$ onto $v$ is the following vector:

$$
v^TD = beginbmatrix
0\
sqrt2 - frac1sqrt2
endbmatrix
$$

So, in our case

$$
v^TSigma v = left[ frac1sqrt2, frac1sqrt2 right] beginbmatrix
0 & 0\
0 & frac12
endbmatrix
beginbmatrix
frac1sqrt2\
frac1sqrt2
endbmatrix = frac14.
$$

So, could you, please, explain, how can $v^TSigma v$ be a variance of values projected on $v$? I would really appreciate "the proof", that $v^TSigma v$ equals variance, as I don't see, why it is true.

linear-algebra variance covariance projection

share|cite|improve this question

edited Mar 18 at 12:32

Eenoku

asked Mar 18 at 0:10

EenokuEenoku

410316

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  The variance is $v^TSigmav$ not $v^TDv$
  $endgroup$
  – David M.
  Mar 18 at 1:18
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @DavidM. Thank you for the correction!
  $endgroup$
  – Eenoku
  Mar 18 at 1:45
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Also, $v^TDneq Dv$, as $D$ is not symmetric!
  $endgroup$
  – David M.
  Mar 18 at 1:49
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @DavidM. Thank you again!
  $endgroup$
  – Eenoku
  Mar 18 at 1:59
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @DavidM. They weren't pedantic, but necessary :) Something like $beginbmatrix 0&0\0&frac12 endbmatrix$ ? I'm afraid I exchanged columns with rows before in hurry...
  $endgroup$
  – Eenoku
  Mar 18 at 2:20
  
  

  
  
  
  

 | 
show 2 more comments

1

$begingroup$

I have troubles understanding a geometrical meaning of a covariance matrix.

Let's say we have a data set containing two points (-1,1), (-1,2) and write them in to the matrix

$$D = beginbmatrix
-1 & -1\
1 & 2
endbmatrix.
$$

Then we'll choose the unit-vector $v = left( frac1sqrt2, frac1sqrt2 right)^T$.

And now, in this article is the following statement:

So, if we would like to represent the covariance matrix with a vector
and its magnitude, we should simply try to find the vector that points
into the direction of the largest spread of the data, and whose
magnitude equals the spread (variance) in this direction.

If we define this vector as $v$, then the projection of our data $D$
onto this vector is obtained as $v^TD$, and the
variance of the projected data is $v^TSigma v$.

The covariance matrix $Sigma$ looks like this:

$$
Sigma = beginbmatrix
0 & 0\
0 & frac12
endbmatrix
$$

Also, the projection of $D$ onto $v$ is the following vector:

$$
v^TD = beginbmatrix
0\
sqrt2 - frac1sqrt2
endbmatrix
$$

So, in our case

$$
v^TSigma v = left[ frac1sqrt2, frac1sqrt2 right] beginbmatrix
0 & 0\
0 & frac12
endbmatrix
beginbmatrix
frac1sqrt2\
frac1sqrt2
endbmatrix = frac14.
$$

So, could you, please, explain, how can $v^TSigma v$ be a variance of values projected on $v$? I would really appreciate "the proof", that $v^TSigma v$ equals variance, as I don't see, why it is true.

linear-algebra variance covariance projection

share|cite|improve this question

edited Mar 18 at 12:32

Eenoku

asked Mar 18 at 0:10

EenokuEenoku

410316

$endgroup$

• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  The variance is $v^TSigmav$ not $v^TDv$
  $endgroup$
  – David M.
  Mar 18 at 1:18
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @DavidM. Thank you for the correction!
  $endgroup$
  – Eenoku
  Mar 18 at 1:45
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  Also, $v^TDneq Dv$, as $D$ is not symmetric!
  $endgroup$
  – David M.
  Mar 18 at 1:49
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @DavidM. Thank you again!
  $endgroup$
  – Eenoku
  Mar 18 at 1:59
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  @DavidM. They weren't pedantic, but necessary :) Something like $beginbmatrix 0&0\0&frac12 endbmatrix$ ? I'm afraid I exchanged columns with rows before in hurry...
  $endgroup$
  – Eenoku
  Mar 18 at 2:20
  
  

  
  
  
  

 | 
show 2 more comments

$begingroup$

I have troubles understanding a geometrical meaning of a covariance matrix.

Let's say we have a data set containing two points (-1,1), (-1,2) and write them in to the matrix

$$D = beginbmatrix
-1 & -1\
1 & 2
endbmatrix.
$$

Then we'll choose the unit-vector $v = left( frac1sqrt2, frac1sqrt2 right)^T$.

And now, in this article is the following statement:

So, if we would like to represent the covariance matrix with a vector
and its magnitude, we should simply try to find the vector that points
into the direction of the largest spread of the data, and whose
magnitude equals the spread (variance) in this direction.

If we define this vector as $v$, then the projection of our data $D$
onto this vector is obtained as $v^TD$, and the
variance of the projected data is $v^TSigma v$.

The covariance matrix $Sigma$ looks like this:

$$
Sigma = beginbmatrix
0 & 0\
0 & frac12
endbmatrix
$$

Also, the projection of $D$ onto $v$ is the following vector:

$$
v^TD = beginbmatrix
0\
sqrt2 - frac1sqrt2
endbmatrix
$$

So, in our case

$$
v^TSigma v = left[ frac1sqrt2, frac1sqrt2 right] beginbmatrix
0 & 0\
0 & frac12
endbmatrix
beginbmatrix
frac1sqrt2\
frac1sqrt2
endbmatrix = frac14.
$$

So, could you, please, explain, how can $v^TSigma v$ be a variance of values projected on $v$? I would really appreciate "the proof", that $v^TSigma v$ equals variance, as I don't see, why it is true.

linear-algebra variance covariance projection

share|cite|improve this question

edited Mar 18 at 12:32

Eenoku

asked Mar 18 at 0:10

EenokuEenoku

410316

$endgroup$

share|cite|improve this question

edited Mar 18 at 12:32

Eenoku

asked Mar 18 at 0:10

EenokuEenoku

410316

share|cite|improve this question

edited Mar 18 at 12:32

Eenoku

asked Mar 18 at 0:10

EenokuEenoku

410316

asked Mar 18 at 0:10

EenokuEenoku

410316

asked Mar 18 at 0:10

EenokuEenoku

410316