# Questions tagged [regularization]

Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: http://en.wikipedia.org/wiki/Regularization_(mathematics))

368 questions
Filter by
Sorted by
Tagged with
1 vote
26 views

### Divergent Tail Sums of Approximations of Non-trace Class Compact Operators

I'm working on approximations of compact operators that are not trace class, and I'm looking for ways to provide meaningful approximation error estimates for truncated eigenfunction expansions. I ...
33 views

### convex optimization with L1 and L2 regularization

Can we solve the problem with the form $$min_{x}||Ax - b||^2_2+\lambda||x||_1+\mu||x^2-y||^2_2$$ where y is a given vector. This problem is a combination of least-squares regression with L1 and L2 ...
1 vote
44 views

### Solve the Soft SVM Dual Problem with L1 Regularization

I'm considering a support vector regression model with a prediction $$\hat{y}(\mathbf{x}_\star)=\boldsymbol{\theta}^{\top} \boldsymbol{\phi}(\mathbf{x}_\star)$$ where $\boldsymbol{\theta}$ are the ...
• 239
83 views

### Make a non-smooth function smooth [duplicate]

I am dealing with a piecewise affine function $f$ defined as follows: $f(x)=0$ if $x<1$, $f(x)=1-x$ if $x\in [1,2]$ and $f(x)=-1$ otherwise. I want to make it smooth. I looked at sigmoid functions ...
• 147
100 views

21 views

### Penalty function $f(a, b)$ for $a, b \in \mathbb{R}^+$ which adds a high positive penalty only when both $a, b$ are very small

I'm doing Bayesian recruitment curve fitting where my curve has two parameters $a$ and $b$. Both $a, b \in \mathbb{R}^+$. I have put a Truncated Normal prior on $a$ and a Half Normal prior on $b$. I'm ...
• 33
76 views

42 views

### regularization and punishment method in least square fitting

I have a least square problem: $$\min \sum_j^p| f(x_j)-y_j |^2 \\ \text{where } f(x) = a x - \sum_{i}^{n} b_i J_1(c_i x)$$ where $J_1$ is the first order Bessel function. I have to find a set ...
• 113
90 views

### Sobolev regularity required for $|\nabla u|^2 \in H^1(\Omega)$

We have $n=2$ and $\Omega=\{x \in \mathbb{R}^2\| ||x||_2<1 \}$. I need $|\nabla u|^2 \in H^1(\Omega)$. So i would choose $u \in W^{2,4}(\Omega)$. But since we have $n=2$ I thought that a lower ...
• 21
1 vote
21 views

### Morozov Discrepancy for PDE-Constrained Optimization With Bound Constraints

$\textbf{Background and context}$ Let $\beta:\Omega\rightarrow\mathbb{R}$, be an unknown spatially distributed parameter (over the spatial domain $\Omega$). Let $\mathcal{F}$ represent a parameter-to-...
• 2,120
40 views

### Weighted sum of N images, which minimizes their TV norm

I have $K$ images $I_i, i\in{1 \ldots K}$ of the size $M \times N$. I wish to find weights $w_i$, s.t. $w_i \in [0,1]$ and $\sum_1^K w_i = 1$ so that $$|\sum_{i=1}^{K} w_i I_i|_{TV}$$ is minimal. I ...
• 368
1 vote
75 views

1 vote
29 views

### How do these definitions of regularization match?

I know regularization from the following point of view: You add a term R(f) to a loss function, e.g: $min_{f} \sum V(f(x_i),y_i) + \lambda R(f)$ where $\lambda$ is a parameter. I recently read about ...
• 107
1 vote
233 views

### Regularized optimization : adding a vanishingly small penalty term does not change the solution set?

Say I am trying to minimize a differentiable function $R: \Theta\to\mathbb R^+$, where $\Theta\subseteq\mathbb R^p$ is a compact subset. Now, for $\lambda\ge0$, I define the $\lambda$-regularized ...
62 views

### Definition of $a=0$ limit of Hurwitz Zeta function

The definition of Hurwitz zeta function is $$\zeta(s,a) =\sum_{k=0}^\infty \frac{1}{(k+a)^s}$$ where $a=0$ limit is obvious singular. But in functionsite: https://functions.wolfram.com/...
• 109
54 views

### Rewriting the solution of $\ell_2$-reweighted least-squares

Consider the problem $$\underset{x \in \Bbb R^n}{\operatorname{argmin}} \|Ax-y\|_2^2 + \lambda\|Wx\|_2^2$$ where $A \in \Bbb R^{m \times n}$ with $m<n$, $W \in \Bbb R^{n \times n}$ is an diagonal ...
1 vote
63 views

### Best inverse / minimization solution of ill conditioned matrix and underdetermined system

I have an ill-conditioned matrix G representing Green's function estimates of a strain model for a rock mass that has been instrumented with optical fiber measuring strain. My problem is seemingly the ...
1 vote
74 views

### Proximal operator of the $L^1$ norm, constrained to $x\ge0$

A standard variational problem (arising, e.g., in imaging) reads $$\operatorname{argmin}_x \frac{1}{2}\Vert Ax - y\Vert ^2 + \Vert x\Vert_1$$ where the $1$-norm serves as a sparsifying regularizer. ...
• 779
55 views

### L1 regularization success rates (MATLAB minL1lin function)

Following a study on Making do with less: An introduction to Compressed Sensing I am attempting to simulate the weighing of 100 coins. There are five sets of coins. In each set there are counterfeit ...
• 25
222 views

• 183
35 views

### Does exchanging the role data fit term and regularization term make sens?

For example the total variation regularization formulation is given as $$\min_x \frac{1}{2} || Ax -y ||_2^2 + \alpha ||\nabla x||_1.$$ Now I am wondering, would it make sense to ask for a solution ...
• 51
33 views

### Finding the smoothest function bounded by two other functions

Suppose I have two polynomial functions $f(x)$ and $z(x)$ defined over some closed interval $[a,b]$ $\in$ $\mathbb{R}$. The functions are subject only to the restriction that $f(x) > z(x)$ $\forall$...
1 vote
478 views

• 131