Gradient of Quadratic Form with Inverse of Complex MatricesDerivative of Nested Matrix Quadratic FormJordan Normal form — Complex matricesderivative of gradient involving inverse of matricesDerivative of quadratic form of complex valued matricesMinimization with complex gradient descentGradient descend for quadratic function with constraintsChange of basis in gradient descent of quadratic formgradient of hermitian quadratic formgradient of quadratic formExplanation of gradient descent on convex quadraticProjected gradient descent with matrices
Brexit - No Deal Rejection
What is the significance behind "40 days" that often appears in the Bible?
Recruiter wants very extensive technical details about all of my previous work
Can I use USB data pins as a power source?
et qui - how do you really understand that kind of phraseology?
ERC721: How to get the owned tokens of an address
Employee lack of ownership
What is the meaning of まっちろけ?
Is a party consisting of only a bard, a cleric, and a warlock functional long-term?
How can we have a quark condensate without a quark potential?
How to prove the triangle inequality for this metric space
While on vacation my taxi took a longer route, possibly to scam me out of money. How can I deal with this?
Four married couples attend a party. Each person shakes hands with every other person, except their own spouse, exactly once. How many handshakes?
Examples of transfinite towers
Do the common programs (for example: "ls", "cat") in Linux and BSD come from the same source code?
How to explain that I do not want to visit a country due to personal safety concern?
Is there a hypothetical scenario that would make Earth uninhabitable for humans, but not for (the majority of) other animals?
Why do newer 737s use two different styles of split winglets?
Custom alignment for GeoMarkers
What is the relationship between relativity and the Doppler effect?
Is Manda another name for Saturn (Shani)?
Why does overlay work only on the first tcolorbox?
Encrypting then Base64 Encoding
Does .bashrc contain syntax errors?
Gradient of Quadratic Form with Inverse of Complex Matrices
Derivative of Nested Matrix Quadratic FormJordan Normal form — Complex matricesderivative of gradient involving inverse of matricesDerivative of quadratic form of complex valued matricesMinimization with complex gradient descentGradient descend for quadratic function with constraintsChange of basis in gradient descent of quadratic formgradient of hermitian quadratic formgradient of quadratic formExplanation of gradient descent on convex quadraticProjected gradient descent with matrices
$begingroup$
I want to calculate the gradient of
$$ w^H H F (F^H F)^-1 F^H H^H w $$
with respect to $ F $, which is complex.
I am basing on this previous answer Derivative of Nested Matrix Quadratic Form that uses differentials to compute the derivative of a similar expression with real matrices. However, I have difficulties in computing the differential when $ (.)^H $ is involved.
For instance, I make these changes: $ x = F^H H^H w $ and $ Z = F^H F $. Then, I obtain $ dx = 0 $ and $ dZ = F^H dF $.
Is it correct that $ dx = 0 $ or should I consider approaching the problem from a different perspective? Thank you!
calculus linear-algebra matrices derivatives gradient-descent
asked Mar 12 at 11:52
Fer NandoFer Nando
214
$endgroup$
add a comment |
$begingroup$
I want to calculate the gradient of
$$ w^H H F (F^H F)^-1 F^H H^H w $$
with respect to $ F $, which is complex.
I am basing on this previous answer Derivative of Nested Matrix Quadratic Form that uses differentials to compute the derivative of a similar expression with real matrices. However, I have difficulties in computing the differential when $ (.)^H $ is involved.
For instance, I make these changes: $ x = F^H H^H w $ and $ Z = F^H F $. Then, I obtain $ dx = 0 $ and $ dZ = F^H dF $.
Is it correct that $ dx = 0 $ or should I consider approaching the problem from a different perspective? Thank you!
calculus linear-algebra matrices derivatives gradient-descent
asked Mar 12 at 11:52
Fer NandoFer Nando
214
$endgroup$
add a comment |
$begingroup$
I want to calculate the gradient of
$$ w^H H F (F^H F)^-1 F^H H^H w $$
with respect to $ F $, which is complex.
I am basing on this previous answer Derivative of Nested Matrix Quadratic Form that uses differentials to compute the derivative of a similar expression with real matrices. However, I have difficulties in computing the differential when $ (.)^H $ is involved.
For instance, I make these changes: $ x = F^H H^H w $ and $ Z = F^H F $. Then, I obtain $ dx = 0 $ and $ dZ = F^H dF $.
Is it correct that $ dx = 0 $ or should I consider approaching the problem from a different perspective? Thank you!
calculus linear-algebra matrices derivatives gradient-descent
asked Mar 12 at 11:52
Fer NandoFer Nando
214
$endgroup$
I want to calculate the gradient of
$$ w^H H F (F^H F)^-1 F^H H^H w $$
with respect to $ F $, which is complex.
I am basing on this previous answer Derivative of Nested Matrix Quadratic Form that uses differentials to compute the derivative of a similar expression with real matrices. However, I have difficulties in computing the differential when $ (.)^H $ is involved.
For instance, I make these changes: $ x = F^H H^H w $ and $ Z = F^H F $. Then, I obtain $ dx = 0 $ and $ dZ = F^H dF $.
Is it correct that $ dx = 0 $ or should I consider approaching the problem from a different perspective? Thank you!
calculus linear-algebra matrices derivatives gradient-descent
calculus linear-algebra matrices derivatives gradient-descent
asked Mar 12 at 11:52
Fer NandoFer Nando
214
asked Mar 12 at 11:52
Fer NandoFer Nando
214
asked Mar 12 at 11:52
Fer NandoFer Nando
214
asked Mar 12 at 11:52
Fer NandoFer Nando
214
asked Mar 12 at 11:52
Fer NandoFer Nando
214
214
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
As you suggested, define the variables
$$eqalign
x &= F^HH^Hw &implies x^H = w^HHF cr
Z &= F^HF &implies Z^-1F^H = F^+ rm ,,(pseudoinverse)cr
$$
and yes, in the context of Wirtinger derivatives $,dx=0$.
Write the function in terms of these new variables. Then find its differential and gradient.
$$eqalign
phi &= x^HZ^-1x cr
dphi
&= dx^HZ^-1x + x^HdZ^-1x cr
&= dx^HZ^-1x - x^HZ^-1dZ,Z^-1x cr
&= (w^HH,dF)Z^-1x - x^HZ^-1(F^HdF),Z^-1x cr
&= Big(Z^-1xw^HH - Z^-1xx^HZ^-1F^HBig)^T:dF cr
&= Big(Z^-1F^HH^Hww^HH - Z^-1F^HH^Hww^HHFZ^-1F^HBig)^T:dF cr
&= Big(F^+H^Hww^HH - F^+H^Hww^HHFF^+Big)^T:dF cr
&= Big((F^+H^Hww^HH),(I - FF^+)Big)^T:dF cr
fracpartialphipartial F &= (I - FF^+)^T (F^+H^Hww^HH)^T cr
$$
where a colon was used in some steps as a convenient product notation for the trace, i.e.
$$A:B = rm Tr(A^TB)$$
answered Mar 12 at 17:47
greggreg
8,8401824
$endgroup$
$begingroup$
Thank you greg!
$endgroup$
– Fer Nando
Mar 13 at 21:44
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3145005%2fgradient-of-quadratic-form-with-inverse-of-complex-matrices%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
As you suggested, define the variables
$$eqalign
x &= F^HH^Hw &implies x^H = w^HHF cr
Z &= F^HF &implies Z^-1F^H = F^+ rm ,,(pseudoinverse)cr
$$
and yes, in the context of Wirtinger derivatives $,dx=0$.
Write the function in terms of these new variables. Then find its differential and gradient.
$$eqalign
phi &= x^HZ^-1x cr
dphi
&= dx^HZ^-1x + x^HdZ^-1x cr
&= dx^HZ^-1x - x^HZ^-1dZ,Z^-1x cr
&= (w^HH,dF)Z^-1x - x^HZ^-1(F^HdF),Z^-1x cr
&= Big(Z^-1xw^HH - Z^-1xx^HZ^-1F^HBig)^T:dF cr
&= Big(Z^-1F^HH^Hww^HH - Z^-1F^HH^Hww^HHFZ^-1F^HBig)^T:dF cr
&= Big(F^+H^Hww^HH - F^+H^Hww^HHFF^+Big)^T:dF cr
&= Big((F^+H^Hww^HH),(I - FF^+)Big)^T:dF cr
fracpartialphipartial F &= (I - FF^+)^T (F^+H^Hww^HH)^T cr
$$
where a colon was used in some steps as a convenient product notation for the trace, i.e.
$$A:B = rm Tr(A^TB)$$
answered Mar 12 at 17:47
greggreg
8,8401824
$endgroup$
$begingroup$
Thank you greg!
$endgroup$
– Fer Nando
Mar 13 at 21:44
add a comment |
$begingroup$
As you suggested, define the variables
$$eqalign
x &= F^HH^Hw &implies x^H = w^HHF cr
Z &= F^HF &implies Z^-1F^H = F^+ rm ,,(pseudoinverse)cr
$$
and yes, in the context of Wirtinger derivatives $,dx=0$.
Write the function in terms of these new variables. Then find its differential and gradient.
$$eqalign
phi &= x^HZ^-1x cr
dphi
&= dx^HZ^-1x + x^HdZ^-1x cr
&= dx^HZ^-1x - x^HZ^-1dZ,Z^-1x cr
&= (w^HH,dF)Z^-1x - x^HZ^-1(F^HdF),Z^-1x cr
&= Big(Z^-1xw^HH - Z^-1xx^HZ^-1F^HBig)^T:dF cr
&= Big(Z^-1F^HH^Hww^HH - Z^-1F^HH^Hww^HHFZ^-1F^HBig)^T:dF cr
&= Big(F^+H^Hww^HH - F^+H^Hww^HHFF^+Big)^T:dF cr
&= Big((F^+H^Hww^HH),(I - FF^+)Big)^T:dF cr
fracpartialphipartial F &= (I - FF^+)^T (F^+H^Hww^HH)^T cr
$$
where a colon was used in some steps as a convenient product notation for the trace, i.e.
$$A:B = rm Tr(A^TB)$$
answered Mar 12 at 17:47
greggreg
8,8401824
$endgroup$
$begingroup$
Thank you greg!
$endgroup$
– Fer Nando
Mar 13 at 21:44
add a comment |
$begingroup$
As you suggested, define the variables
$$eqalign
x &= F^HH^Hw &implies x^H = w^HHF cr
Z &= F^HF &implies Z^-1F^H = F^+ rm ,,(pseudoinverse)cr
$$
and yes, in the context of Wirtinger derivatives $,dx=0$.
Write the function in terms of these new variables. Then find its differential and gradient.
$$eqalign
phi &= x^HZ^-1x cr
dphi
&= dx^HZ^-1x + x^HdZ^-1x cr
&= dx^HZ^-1x - x^HZ^-1dZ,Z^-1x cr
&= (w^HH,dF)Z^-1x - x^HZ^-1(F^HdF),Z^-1x cr
&= Big(Z^-1xw^HH - Z^-1xx^HZ^-1F^HBig)^T:dF cr
&= Big(Z^-1F^HH^Hww^HH - Z^-1F^HH^Hww^HHFZ^-1F^HBig)^T:dF cr
&= Big(F^+H^Hww^HH - F^+H^Hww^HHFF^+Big)^T:dF cr
&= Big((F^+H^Hww^HH),(I - FF^+)Big)^T:dF cr
fracpartialphipartial F &= (I - FF^+)^T (F^+H^Hww^HH)^T cr
$$
where a colon was used in some steps as a convenient product notation for the trace, i.e.
$$A:B = rm Tr(A^TB)$$
answered Mar 12 at 17:47
greggreg
8,8401824
$endgroup$
As you suggested, define the variables
$$eqalign
x &= F^HH^Hw &implies x^H = w^HHF cr
Z &= F^HF &implies Z^-1F^H = F^+ rm ,,(pseudoinverse)cr
$$
and yes, in the context of Wirtinger derivatives $,dx=0$.
Write the function in terms of these new variables. Then find its differential and gradient.
$$eqalign
phi &= x^HZ^-1x cr
dphi
&= dx^HZ^-1x + x^HdZ^-1x cr
&= dx^HZ^-1x - x^HZ^-1dZ,Z^-1x cr
&= (w^HH,dF)Z^-1x - x^HZ^-1(F^HdF),Z^-1x cr
&= Big(Z^-1xw^HH - Z^-1xx^HZ^-1F^HBig)^T:dF cr
&= Big(Z^-1F^HH^Hww^HH - Z^-1F^HH^Hww^HHFZ^-1F^HBig)^T:dF cr
&= Big(F^+H^Hww^HH - F^+H^Hww^HHFF^+Big)^T:dF cr
&= Big((F^+H^Hww^HH),(I - FF^+)Big)^T:dF cr
fracpartialphipartial F &= (I - FF^+)^T (F^+H^Hww^HH)^T cr
$$
where a colon was used in some steps as a convenient product notation for the trace, i.e.
$$A:B = rm Tr(A^TB)$$
answered Mar 12 at 17:47
greggreg
8,8401824
edited Mar 12 at 17:52
answered Mar 12 at 17:47
greggreg
8,8401824
answered Mar 12 at 17:47
greggreg
8,8401824
answered Mar 12 at 17:47
greggreg
8,8401824
8,8401824
$begingroup$
Thank you greg!
$endgroup$
– Fer Nando
Mar 13 at 21:44
add a comment |
$begingroup$
Thank you greg!
$endgroup$
– Fer Nando
Mar 13 at 21:44
$begingroup$
Thank you greg!
$endgroup$
– Fer Nando
Mar 13 at 21:44
$begingroup$
Thank you greg!
$endgroup$
– Fer Nando
Mar 13 at 21:44
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3145005%2fgradient-of-quadratic-form-with-inverse-of-complex-matrices%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
