Fdtxjr

How to find all the available tools in macOS terminal?

"Seemed to had" is it correct?

What are 'alternative tunings' of a guitar and why would you use them? Doesn't it make it more difficult to play?

When to stop saving and start investing?

How much radiation do nuclear physics experiments expose researchers to nowadays?

How to recreate this effect in Photoshop?

Is a manifold-with-boundary with given interior and non-empty boundary essentially unique?

Why is "Captain Marvel" translated as male in Portugal?

Is it ethical to give a final exam after the professor has quit before teaching the remaining chapters of the course?

How can I make names more distinctive without making them longer?

If Jon Snow became King of the Seven Kingdoms what would his regnal number be?

Withdrew £2800, but only £2000 shows as withdrawn on online banking; what are my obligations?

How discoverable are IPv6 addresses and AAAA names by potential attackers?

Should I discuss the type of campaign with my players?

What is this single-engine low-wing propeller plane?

Right-skewed distribution with mean equals to mode?

Is 1 ppb equal to 1 μg/kg?

Java 8 stream max() function argument type Comparator vs Comparable

When is phishing education going too far?

Do I really need recursive chmod to restrict access to a folder?

Letter Boxed validator

G-Code for resetting to 100% speed

Is there a documented rationale why the House Ways and Means chairman can demand tax info?

Is there a "higher Segal conjecture"?

How to minimize the $0$-“norm” with quadratic constraint?

1

$begingroup$

I have a vector

$$y = Ax + n$$

where vectors $x, y, n$ are $25 times 1$, matrix $A$ is $25 times 25$ and near-orthogonal (actually, it's a part of the DFT matrix). Also, $x$ is sparse and has only $5$ non-zero elements. $A$ and $y$ are known, $n$ is the Gaussian random variable, and I need to recover $x$ as accurately as possible.

How to solve the problem? We don't care about the complexity, the only consideration is the accuracy. Thanks in advance.

linear-algebra matrices statistics convex-optimization least-squares

share|cite|improve this question

edited Mar 26 at 9:12

Rodrigo de Azevedo

13.2k41961

asked Mar 25 at 16:08

LinTInaLinTIna

315

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  $25 choose 5$ is rather small, you can enumerate all sparsity patterns and determine which combination gives the highest likelihood for $n$
  $endgroup$
  – LinAlg
  Mar 25 at 16:56
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @LinAlg I think it’s a good idea but can you kindly tell the specific process as I’m not sure what to do, such as the computing of x after I enumerating it’s non-zero elements’ index.
  $endgroup$
  – LinTIna
  Mar 25 at 17:08
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  $begingroup$
  What does the title have to do with the actual question?
  $endgroup$
  – Rodrigo de Azevedo
  Mar 26 at 9:13
  

  
  
  
  

add a comment | 

1

$begingroup$

I have a vector

$$y = Ax + n$$

where vectors $x, y, n$ are $25 times 1$, matrix $A$ is $25 times 25$ and near-orthogonal (actually, it's a part of the DFT matrix). Also, $x$ is sparse and has only $5$ non-zero elements. $A$ and $y$ are known, $n$ is the Gaussian random variable, and I need to recover $x$ as accurately as possible.

How to solve the problem? We don't care about the complexity, the only consideration is the accuracy. Thanks in advance.

linear-algebra matrices statistics convex-optimization least-squares

share|cite|improve this question

edited Mar 26 at 9:12

Rodrigo de Azevedo

13.2k41961

asked Mar 25 at 16:08

LinTInaLinTIna

315

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  $25 choose 5$ is rather small, you can enumerate all sparsity patterns and determine which combination gives the highest likelihood for $n$
  $endgroup$
  – LinAlg
  Mar 25 at 16:56
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @LinAlg I think it’s a good idea but can you kindly tell the specific process as I’m not sure what to do, such as the computing of x after I enumerating it’s non-zero elements’ index.
  $endgroup$
  – LinTIna
  Mar 25 at 17:08
  

  
  
  
  


• 
  
  
  

  
  2
  

  
  
  
  
  
  

  
  $begingroup$
  What does the title have to do with the actual question?
  $endgroup$
  – Rodrigo de Azevedo
  Mar 26 at 9:13
  

  
  
  
  

add a comment | 

$begingroup$

I have a vector

$$y = Ax + n$$

where vectors $x, y, n$ are $25 times 1$, matrix $A$ is $25 times 25$ and near-orthogonal (actually, it's a part of the DFT matrix). Also, $x$ is sparse and has only $5$ non-zero elements. $A$ and $y$ are known, $n$ is the Gaussian random variable, and I need to recover $x$ as accurately as possible.

How to solve the problem? We don't care about the complexity, the only consideration is the accuracy. Thanks in advance.

linear-algebra matrices statistics convex-optimization least-squares

share|cite|improve this question

edited Mar 26 at 9:12

Rodrigo de Azevedo

13.2k41961

asked Mar 25 at 16:08

LinTInaLinTIna

315

$endgroup$

share|cite|improve this question

edited Mar 26 at 9:12

Rodrigo de Azevedo

13.2k41961

asked Mar 25 at 16:08

LinTInaLinTIna

315

share|cite|improve this question

edited Mar 26 at 9:12

Rodrigo de Azevedo

13.2k41961

asked Mar 25 at 16:08

LinTInaLinTIna

315

edited Mar 26 at 9:12

Rodrigo de Azevedo

13.2k41961

edited Mar 26 at 9:12

Rodrigo de Azevedo

13.2k41961

asked Mar 25 at 16:08

LinTInaLinTIna

315

asked Mar 25 at 16:08

LinTInaLinTIna

315

1 Answer
1

active

oldest

votes

0

$begingroup$

Suppose you know the sparsity pattern of $x$ (enumeration works here), then the problem reduces to $y=Ax+n$ where $x$ is $5 times 1$, $y$ and $n$ are $25times 1$ and $A$ is $25times 5$. My assertion is that the best $x$ is the one for that maximizes the maximum likelihood of $n$:
$$max_n,x left (2pi)^-frack2(detSigma)^-frac12 e^-frac12left(n-muright)Sigma^-1left(n-muright) : y = Ax+n right,$$
or that minimizes the negative log likelihood:
$$min_n,x left frack2log(2pi) + frac12logdetSigma + frac12left(y-Ax-muright)Sigma^-1left(y-Ax-muright) right.$$
Assuming that $nsim N(0,I)$, this simplifies to:
$$min_x leftAx-y.$$
So, just do least squares estimation of the linear model $y=Ax$, and select the sparsity pattern for which $||Ax-y||$ is as small as possible.

share|cite|improve this answer

answered Mar 25 at 18:36

LinAlgLinAlg

10.1k1521

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  thank you. By the way, if the number of non-zero elemsnts of x is uncertain, for example, can be 5, 6 or 7. Then is there any good idea?
  $endgroup$
  – LinTIna
  Mar 26 at 5:03
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @LinTIna you have to formulate a criterion to decide between 5, 6 or 7. For example, you can look at the difference in likelihood.
  $endgroup$
  – LinAlg
  Mar 26 at 12:17
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  do you mean I need to enumerate 5,6 and 7. Then look at the result (maybe the norm of y-Ax) to decide it?
  $endgroup$
  – LinTIna
  Mar 26 at 12:52
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  yes, that is one way
  $endgroup$
  – LinAlg
  Mar 26 at 12:52
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  thank you, I got it
  $endgroup$
  – LinTIna
  Mar 26 at 12:53
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "69"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3161992%2fhow-to-minimize-the-0-norm-with-quadratic-constraint%23new-answer', 'question_page');

);

Post as a guest

Name

Email

Required, but never shown

0

$begingroup$

Suppose you know the sparsity pattern of $x$ (enumeration works here), then the problem reduces to $y=Ax+n$ where $x$ is $5 times 1$, $y$ and $n$ are $25times 1$ and $A$ is $25times 5$. My assertion is that the best $x$ is the one for that maximizes the maximum likelihood of $n$:
$$max_n,x left (2pi)^-frack2(detSigma)^-frac12 e^-frac12left(n-muright)Sigma^-1left(n-muright) : y = Ax+n right,$$
or that minimizes the negative log likelihood:
$$min_n,x left frack2log(2pi) + frac12logdetSigma + frac12left(y-Ax-muright)Sigma^-1left(y-Ax-muright) right.$$
Assuming that $nsim N(0,I)$, this simplifies to:
$$min_x leftAx-y.$$
So, just do least squares estimation of the linear model $y=Ax$, and select the sparsity pattern for which $||Ax-y||$ is as small as possible.

share|cite|improve this answer

answered Mar 25 at 18:36

LinAlgLinAlg

10.1k1521

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  thank you. By the way, if the number of non-zero elemsnts of x is uncertain, for example, can be 5, 6 or 7. Then is there any good idea?
  $endgroup$
  – LinTIna
  Mar 26 at 5:03
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @LinTIna you have to formulate a criterion to decide between 5, 6 or 7. For example, you can look at the difference in likelihood.
  $endgroup$
  – LinAlg
  Mar 26 at 12:17
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  do you mean I need to enumerate 5,6 and 7. Then look at the result (maybe the norm of y-Ax) to decide it?
  $endgroup$
  – LinTIna
  Mar 26 at 12:52
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  yes, that is one way
  $endgroup$
  – LinAlg
  Mar 26 at 12:52
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  thank you, I got it
  $endgroup$
  – LinTIna
  Mar 26 at 12:53
  

  
  
  
  

add a comment | 

0

$begingroup$

Suppose you know the sparsity pattern of $x$ (enumeration works here), then the problem reduces to $y=Ax+n$ where $x$ is $5 times 1$, $y$ and $n$ are $25times 1$ and $A$ is $25times 5$. My assertion is that the best $x$ is the one for that maximizes the maximum likelihood of $n$:
$$max_n,x left (2pi)^-frack2(detSigma)^-frac12 e^-frac12left(n-muright)Sigma^-1left(n-muright) : y = Ax+n right,$$
or that minimizes the negative log likelihood:
$$min_n,x left frack2log(2pi) + frac12logdetSigma + frac12left(y-Ax-muright)Sigma^-1left(y-Ax-muright) right.$$
Assuming that $nsim N(0,I)$, this simplifies to:
$$min_x leftAx-y.$$
So, just do least squares estimation of the linear model $y=Ax$, and select the sparsity pattern for which $||Ax-y||$ is as small as possible.

share|cite|improve this answer

answered Mar 25 at 18:36

LinAlgLinAlg

10.1k1521

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  thank you. By the way, if the number of non-zero elemsnts of x is uncertain, for example, can be 5, 6 or 7. Then is there any good idea?
  $endgroup$
  – LinTIna
  Mar 26 at 5:03
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  @LinTIna you have to formulate a criterion to decide between 5, 6 or 7. For example, you can look at the difference in likelihood.
  $endgroup$
  – LinAlg
  Mar 26 at 12:17
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  do you mean I need to enumerate 5,6 and 7. Then look at the result (maybe the norm of y-Ax) to decide it?
  $endgroup$
  – LinTIna
  Mar 26 at 12:52
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  yes, that is one way
  $endgroup$
  – LinAlg
  Mar 26 at 12:52
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  thank you, I got it
  $endgroup$
  – LinTIna
  Mar 26 at 12:53
  

  
  
  
  

add a comment | 

$begingroup$

Suppose you know the sparsity pattern of $x$ (enumeration works here), then the problem reduces to $y=Ax+n$ where $x$ is $5 times 1$, $y$ and $n$ are $25times 1$ and $A$ is $25times 5$. My assertion is that the best $x$ is the one for that maximizes the maximum likelihood of $n$:
$$max_n,x left (2pi)^-frack2(detSigma)^-frac12 e^-frac12left(n-muright)Sigma^-1left(n-muright) : y = Ax+n right,$$
or that minimizes the negative log likelihood:
$$min_n,x left frack2log(2pi) + frac12logdetSigma + frac12left(y-Ax-muright)Sigma^-1left(y-Ax-muright) right.$$
Assuming that $nsim N(0,I)$, this simplifies to:
$$min_x leftAx-y.$$
So, just do least squares estimation of the linear model $y=Ax$, and select the sparsity pattern for which $||Ax-y||$ is as small as possible.

share|cite|improve this answer

answered Mar 25 at 18:36

LinAlgLinAlg

10.1k1521

$endgroup$

share|cite|improve this answer

answered Mar 25 at 18:36

LinAlgLinAlg

10.1k1521

answered Mar 25 at 18:36

LinAlgLinAlg

10.1k1521

answered Mar 25 at 18:36

LinAlgLinAlg

10.1k1521