Bound on hessian when Lipschitz gradient is bounded Announcing the arrival of Valued Associate #679: Cesar Manara Planned maintenance scheduled April 23, 2019 at 00:00UTC (8:00pm US/Eastern)Step size in gradient descentRudin Theorem $1.11$Is there Lipschitz property for subdifferential?Any example of strongly convex functions whose gradients are Lipschitz continuous in $mathbbR^N$Lipschitz Number in Gradient DescentHow is Lipschitz continuity for Fréchet derivatives defined?Lipschitz-constant gradient implies bounded eigenvalues on HessianLocally Lipschitz functions which is approximation of a continuous functionSub-gradient and super-gradient are bounded implies globally Lipschitz.Best constant for Lipschitz continuous gradient function$C^1$ function $f$ with bounded gradient such that $fracf(x)$ is unbounded
Co-worker has annoying ringtone
"Lost his faith in humanity in the trenches of Verdun" — last line of an SF story
How do I find out the mythology and history of my Fortress?
Importance of からだ in this sentence
How often does castling occur in grandmaster games?
Converted a Scalar function to a TVF function for parallel execution-Still running in Serial mode
How would a mousetrap for use in space work?
QGIS virtual layer functionality does not seem to support memory layers
Time to Settle Down!
Did any compiler fully use 80-bit floating point?
Project Euler #1 in C++
Why weren't discrete x86 CPUs ever used in game hardware?
What is Adi Shankara referring to when he says "He has Vajra marks on his feet"?
Do I really need to have a message in a novel to appeal to readers?
Crossing US/Canada Border for less than 24 hours
What is the difference between globalisation and imperialism?
Put R under double integral
Did Mueller's report provide an evidentiary basis for the claim of Russian govt election interference via social media?
Is there a kind of relay that only consumes power when switching?
Is there hard evidence that the grant peer review system performs significantly better than random?
Why limits give us the exact value of the slope of the tangent line?
What's the meaning of "fortified infraction restraint"?
If Windows 7 doesn't support WSL, then what does Linux subsystem option mean?
Would it be easier to apply for a UK visa if there is a host family to sponsor for you in going there?
Bound on hessian when Lipschitz gradient is bounded
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 23, 2019 at 00:00UTC (8:00pm US/Eastern)Step size in gradient descentRudin Theorem $1.11$Is there Lipschitz property for subdifferential?Any example of strongly convex functions whose gradients are Lipschitz continuous in $mathbbR^N$Lipschitz Number in Gradient DescentHow is Lipschitz continuity for Fréchet derivatives defined?Lipschitz-constant gradient implies bounded eigenvalues on HessianLocally Lipschitz functions which is approximation of a continuous functionSub-gradient and super-gradient are bounded implies globally Lipschitz.Best constant for Lipschitz continuous gradient function$C^1$ function $f$ with bounded gradient such that $fracf(x)x$ is unbounded
$begingroup$
I know that if we have a Lipschitz gradient
$$|nabla f(x) - nabla f(y)|leq L|x-y|,, forall x,y, $$
we can say that $nabla^2fpreceq LI.$ I have a problem where difference of gradient is bounded as below
$$|nabla f(x) - nabla f(y)|leq L|x-y|^n, forall x,y, $$
where $n$ is an integer greater than or equal to 2. Is there some theorem which says about bounds on $nabla^2f$ in the form $nabla^2fpreceq LI |x-y|$. I am trying to bound $nabla^2f$, but I am not sure how to do it. Any direction in this question would be really helpful. Here is example of $f(X) = |A - XYY^TX^T|_F^2$ where the gradient of $f$ follows above bound where $A,X,Y in R^n times n$
convex-analysis lipschitz-functions upper-lower-bounds
asked Mar 27 at 20:06
Dushyant SahooDushyant Sahoo
818
$endgroup$
add a comment |
$begingroup$
I know that if we have a Lipschitz gradient
$$|nabla f(x) - nabla f(y)|leq L|x-y|,, forall x,y, $$
we can say that $nabla^2fpreceq LI.$ I have a problem where difference of gradient is bounded as below
$$|nabla f(x) - nabla f(y)|leq L|x-y|^n, forall x,y, $$
where $n$ is an integer greater than or equal to 2. Is there some theorem which says about bounds on $nabla^2f$ in the form $nabla^2fpreceq LI |x-y|$. I am trying to bound $nabla^2f$, but I am not sure how to do it. Any direction in this question would be really helpful. Here is example of $f(X) = |A - XYY^TX^T|_F^2$ where the gradient of $f$ follows above bound where $A,X,Y in R^n times n$
convex-analysis lipschitz-functions upper-lower-bounds
asked Mar 27 at 20:06
Dushyant SahooDushyant Sahoo
818
$endgroup$
1
$begingroup$
Your target inequality doesn't make much sense. And where is $f$ defined? Compact set? $mathbb R^n$?
$endgroup$
– amsmath
Mar 27 at 20:30
add a comment |
$begingroup$
I know that if we have a Lipschitz gradient
$$|nabla f(x) - nabla f(y)|leq L|x-y|,, forall x,y, $$
we can say that $nabla^2fpreceq LI.$ I have a problem where difference of gradient is bounded as below
$$|nabla f(x) - nabla f(y)|leq L|x-y|^n, forall x,y, $$
where $n$ is an integer greater than or equal to 2. Is there some theorem which says about bounds on $nabla^2f$ in the form $nabla^2fpreceq LI |x-y|$. I am trying to bound $nabla^2f$, but I am not sure how to do it. Any direction in this question would be really helpful. Here is example of $f(X) = |A - XYY^TX^T|_F^2$ where the gradient of $f$ follows above bound where $A,X,Y in R^n times n$
convex-analysis lipschitz-functions upper-lower-bounds
asked Mar 27 at 20:06
Dushyant SahooDushyant Sahoo
818
$endgroup$
I know that if we have a Lipschitz gradient
$$|nabla f(x) - nabla f(y)|leq L|x-y|,, forall x,y, $$
we can say that $nabla^2fpreceq LI.$ I have a problem where difference of gradient is bounded as below
$$|nabla f(x) - nabla f(y)|leq L|x-y|^n, forall x,y, $$
where $n$ is an integer greater than or equal to 2. Is there some theorem which says about bounds on $nabla^2f$ in the form $nabla^2fpreceq LI |x-y|$. I am trying to bound $nabla^2f$, but I am not sure how to do it. Any direction in this question would be really helpful. Here is example of $f(X) = |A - XYY^TX^T|_F^2$ where the gradient of $f$ follows above bound where $A,X,Y in R^n times n$
convex-analysis lipschitz-functions upper-lower-bounds
convex-analysis lipschitz-functions upper-lower-bounds
asked Mar 27 at 20:06
Dushyant SahooDushyant Sahoo
818
asked Mar 27 at 20:06
Dushyant SahooDushyant Sahoo
818
edited Mar 27 at 20:46
Dushyant Sahoo
asked Mar 27 at 20:06
Dushyant SahooDushyant Sahoo
818
asked Mar 27 at 20:06
Dushyant SahooDushyant Sahoo
818
asked Mar 27 at 20:06
Dushyant SahooDushyant Sahoo
818
818
1
$begingroup$
Your target inequality doesn't make much sense. And where is $f$ defined? Compact set? $mathbb R^n$?
$endgroup$
– amsmath
Mar 27 at 20:30
add a comment |
1
$begingroup$
Your target inequality doesn't make much sense. And where is $f$ defined? Compact set? $mathbb R^n$?
$endgroup$
– amsmath
Mar 27 at 20:30
1
1
$begingroup$
Your target inequality doesn't make much sense. And where is $f$ defined? Compact set? $mathbb R^n$?
$endgroup$
– amsmath
Mar 27 at 20:30
$begingroup$
Your target inequality doesn't make much sense. And where is $f$ defined? Compact set? $mathbb R^n$?
$endgroup$
– amsmath
Mar 27 at 20:30
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
If $n>1$ then the inequality implies that $nabla f$ is constant, and $nabla^2f$ is zero: divide by $|x-y|$ and let $xto y$.
answered Mar 27 at 20:32
dawdaw
25.2k1745
$endgroup$
$begingroup$
I had one more question. As the Hessian is zero, how does this impact convergence result of the gradient descent? So I am trying to run gradient descent and wanted to bound the hessian to get good convergence result.
$endgroup$
– Dushyant Sahoo
Apr 1 at 6:03
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3165062%2fbound-on-hessian-when-lipschitz-gradient-is-bounded%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If $n>1$ then the inequality implies that $nabla f$ is constant, and $nabla^2f$ is zero: divide by $|x-y|$ and let $xto y$.
answered Mar 27 at 20:32
dawdaw
25.2k1745
$endgroup$
$begingroup$
I had one more question. As the Hessian is zero, how does this impact convergence result of the gradient descent? So I am trying to run gradient descent and wanted to bound the hessian to get good convergence result.
$endgroup$
– Dushyant Sahoo
Apr 1 at 6:03
add a comment |
$begingroup$
If $n>1$ then the inequality implies that $nabla f$ is constant, and $nabla^2f$ is zero: divide by $|x-y|$ and let $xto y$.
answered Mar 27 at 20:32
dawdaw
25.2k1745
$endgroup$
$begingroup$
I had one more question. As the Hessian is zero, how does this impact convergence result of the gradient descent? So I am trying to run gradient descent and wanted to bound the hessian to get good convergence result.
$endgroup$
– Dushyant Sahoo
Apr 1 at 6:03
add a comment |
$begingroup$
If $n>1$ then the inequality implies that $nabla f$ is constant, and $nabla^2f$ is zero: divide by $|x-y|$ and let $xto y$.
answered Mar 27 at 20:32
dawdaw
25.2k1745
$endgroup$
If $n>1$ then the inequality implies that $nabla f$ is constant, and $nabla^2f$ is zero: divide by $|x-y|$ and let $xto y$.
answered Mar 27 at 20:32
dawdaw
25.2k1745
answered Mar 27 at 20:32
dawdaw
25.2k1745
answered Mar 27 at 20:32
dawdaw
25.2k1745
answered Mar 27 at 20:32
dawdaw
25.2k1745
25.2k1745
$begingroup$
I had one more question. As the Hessian is zero, how does this impact convergence result of the gradient descent? So I am trying to run gradient descent and wanted to bound the hessian to get good convergence result.
$endgroup$
– Dushyant Sahoo
Apr 1 at 6:03
add a comment |
$begingroup$
I had one more question. As the Hessian is zero, how does this impact convergence result of the gradient descent? So I am trying to run gradient descent and wanted to bound the hessian to get good convergence result.
$endgroup$
– Dushyant Sahoo
Apr 1 at 6:03
$begingroup$
I had one more question. As the Hessian is zero, how does this impact convergence result of the gradient descent? So I am trying to run gradient descent and wanted to bound the hessian to get good convergence result.
$endgroup$
– Dushyant Sahoo
Apr 1 at 6:03
$begingroup$
I had one more question. As the Hessian is zero, how does this impact convergence result of the gradient descent? So I am trying to run gradient descent and wanted to bound the hessian to get good convergence result.
$endgroup$
– Dushyant Sahoo
Apr 1 at 6:03
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3165062%2fbound-on-hessian-when-lipschitz-gradient-is-bounded%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
Your target inequality doesn't make much sense. And where is $f$ defined? Compact set? $mathbb R^n$?
$endgroup$
– amsmath
Mar 27 at 20:30