# Questions tagged [regularization]

Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: http://en.wikipedia.org/wiki/Regularization_(mathematics))

345 questions
Filter by
Sorted by
Tagged with
1 vote
34 views

6 views

9 views

### Can I call a loss term 'regularizer' which is not related backpropagation?

I have a loss function for a supervised learning task, classification. Let's say $$\min_{\theta} L = L_1 + L_2$$ where $L_1$ is a BCE loss, and $L_2$ is a distance between the feature vector and a ...
• 11
1 vote
34 views

### Why estimation of markov parameters are extreamly noise sensitive? [closed]

Assume that you have input $u(k)$ and output $y(k)$ and you want to find the impulse response $g(k)$, or even called markov parameters So let's say that I create some inputs $u(k)$ and outputs $y(k)$ ...
• 2,700
115 views

### Constant terms of asymptotic expansions of smoothed sums of all prime numbers

Consider the series $$\sum_{n=1}^\infty n e^{-n \varepsilon}$$ For $\varepsilon \leq 0$, it diverges. For $\varepsilon > 0$, it converges and equals $$\frac{e^\varepsilon}{(e^\varepsilon - 1)^2}$$ ...
• 5,547
40 views

### Convergence of Tikohnov regularization for convex objective

I have the following question: Assume we want to minimize some convex (not strictly convex) and coercive ($f(x) \to \infty$ as $||x|| \to \infty$) function $f \in C^2(R^n,R)$, which has possibly ...
• 1
58 views

• 181
40 views

### derive the proximal operator for a squared error

To calculate the proximal operator of $f=\frac{1}{2} \Vert x \Vert_2^2$: $\mathbb{prox}_{\lambda f}(x)=\left(\frac{1}{1 + \lambda} \right) x$ following, what is the proximal operator of the pairwise ...
• 101
1 vote
77 views

### Indefinite generalized Tikhonov regularization

Suppose we want to choose $x$ to minimise the following (generalized) Tikhonov regularized least squares objective: $$(Ax-b)^\top (Ax-b) + \lambda [(x-c)^\top W (x-c)],$$ where $W$ is symmetric, but ...
• 623
1 vote
91 views

### Principal value integral with cosine

We are doing a practice set for an exam where we have to find the principal value of the integral $$I = P.V. \int^\infty_{-\infty} \frac{\cos(x)}{(x-1)(x^2+1)} dx$$ Firstly, I am unsure if this ...
• 11
109 views

### Optimal Transport and Entropic Regularization

We are working with discrete optimal transport. Let $P$ be a matrix and let $H(P) =- \sum_{i,j} P_{i,j} (\log(P_{i,j})-1)$. Let $C$ be the cost matrix. And $\langle C,P\rangle$ the Frobenius inner ...
• 1,436
63 views

### Best optimization technique for solving overdetermined systems with a constraint

I am trying to make a prediction model based on a system of linear equations: $A\vec{x}=\vec{b}$, where $\vec{x}$ ($m\times1$) is my learning parameters, $A (m\times n)$ and $\vec{b}$ $(m\times1)$ are ...
• 43
1 vote
101 views

### For $u \in L^2([0,T])$ there exists $u_m \in C^\infty$ such that $u_m \to u$ in $L^2_{loc}(]0,T[)$.

I am reading Navier-Stokes Equations by Roger Temam and there is a point in a proof I do not understand. Let me explain: We have a function $u: [0,T] \to H \in L^2([0,T]; H)$, for $H$ some Hilbert ...
• 3,375
37 views

### Defining higher order (hypersingular) integrals in terms of derivatives of the Cauchy principal value?

In A Generalization of the Cauchy Principal Value, the author presents a way to assign values for hypersingular integrals of the form $$I=\int_a^b\frac{f(x)\,\mathrm dx}{(x-u)^n},\quad u\in(a,b)$$ ...
• 4,625
1 vote
61 views

It is possible to understand (perhaps somewhat non-rigorously) the Fourier transform through interchanging order of integration and use of the delta function, like so: $$\hat{f}(k)\equiv\int_{-\infty}^... • 361 0 votes 0 answers 28 views ### Regularity results for ODEs weaker than Lipschitz in a single point I have to solve the following minimal example for t\geq 0 :$$ \dot s(t)= \frac{1}{2s(t)}, \quad s(0)=0 $$This is a minimal example, I know I could solve it easily using the separation of variables.... 0 votes 0 answers 21 views ### Regularizing Jacobian by altering the differentiated function? I have a function f such that I want to solve for a linear system with its Jacobian:$$ \left[\frac{\partial f}{\partial x}\right] a = b $$I want to impose constraints to x so that the space ... 0 votes 1 answer 63 views ### least squares with L1 regularization in selected entries Say for x \in \mathbb{R}^n, I'm minimizing \|Ax - b \|_2^2 with L1 regularization on selected entries of x. i.e. instead of directly add a \|x\|_1 regularization term, it would be on |x_i| + |... • 255 0 votes 1 answer 126 views ### Closed-form solution for quadratic optimization with L2 regularization? Assume I have the following matrix equations, which I want to minimize over x$$\min_{x}\left(\frac{1}{2}||Mx+b||^{2}_{2}+\lambda\left<x,x\right>\right)$$where x is a variable column vector,... • 895 -2 votes 1 answer 57 views ### Minimizing a linear objective plus \| \cdot \|_1 [closed] What is the best method for solving the following convex regularization problem?$$ \text{minimize} ~~ c^T x + \| A x \|_{1} \quad \text{subject to} \quad -1 \leq x_i \leq 1 $$where c is a vector ... • 6,644 0 votes 0 answers 38 views ### Difference / similarities between Tikhonov's regularization and the least squares method? What is the difference / similarities between Tikhonov's regularization and the least squares method? I have tried to find information on both but can´t find any clear answers to this question, so I ... • 359 0 votes 0 answers 35 views ### Deconvolution experimental data by solving as Tikhonov regulation of Fredholm integral equation From an experiment, I have data for time t and a function of time f(t). Data can be described by a Fredholm integral equation such that:$$ F(t) = \int_{0}^{1}ke^{-kt}f(k) \,{\rm d} k $$Here is the ... 0 votes 1 answer 72 views ### Working out the derivative of the log-likelihood for group LASSO I'm following the working of the sparse group LASSO in the paper 'A Sparse-Group LASSO' by Simon. For the linear case, we have the problem given as$$\text{min}_\beta \frac{1}{2}||y-\sum_{l=1}^m X^{(l)...
• 3
My objective is regularizing a function over $X\in \mathbb{R}^{I\times J}$ such that its covariance is jointly diagonalizable with a positive semi-definite matrix $A \in \mathbb{R}^{I\times I}$. For ...