# Questions tagged [regularization]

Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: http://en.wikipedia.org/wiki/Regularization_(mathematics))

303 questions
Filter by
Sorted by
Tagged with
1answer
84 views

### Heat Kernel on a compact manifold without boundary

I was wondering if the conservation of mass which is obvious for the heat equation in $\mathbb{R}^n$ holds also for the heat kernel in a general compact manifold without boundary. I mean I want to ...
0answers
37 views

### Regularization for function in $S^{d-1}$

Given a function $f:S^{d-1}\rightarrow \mathbb{R}$ where $S^{d-1}:=\{x\in \mathbb{R}^d: |x|=1\},$ I want to find a sequence of continuous functions $f_n$ converging to $f$. I thought that maybe, given ...
0answers
6 views

### Minimum of regularized cost function for a movie rating system

I'm currently reading Rafael Irizarry's Machine Learning notes and I was a little confused with the approach towards regularization. To analyse a movie recommendation system, the following equation is ...
0answers
100 views

### Do these constants appear in other areas of mathematics?

I decided to consider divergent (to infinity) integrals as some new kind of number. Towards this end, I began by establishing certain rules defining equivalence of the integrals. It seems the usual ...
0answers
44 views

### Dos the Euler-Mascheroni constant $\gamma$ correspond to infinite hyperbolic angle?

So, I am trying to find the regularized value of the divergent integral $I=\int_1^\infty \sqrt{x^2-1}dx$. Since the area of $\int_0^1 \sqrt{1-x^2}dx=\frac\pi4$, I wonder whether the area under ...
0answers
85 views

1answer
55 views

### The Wasserstein Metric. Computational Optimal Transport. Weights.

Let $\mu,\nu$ be two probability measures on the space $\mathbb{R}^n$, and let $\Pi(\mu,\nu)$ be the space of joint probability measures with marginals $\mu$ and $\nu$. After a discretisation of space ...
1answer
16 views

### Interpretation of the relation between regularized least-squares and minimum-norm solution for an underdetermined system

For a linear system $Ax = y$, define $J_1 = ||{Ax-y}||^2$ and $J_2 = ||x||^2$. We wish to minimize the weighted-sum objective $J_1 + \mu J_2$. If we interpret $J_1$ as a cost function and $J_2$ as an ...
1answer
75 views

### Is $\| B(AB)^\dagger \|_2$ uniformly bounded for all positive diagonal matrices $B$?

Consider $\| B(AB)^\dagger \|_2$ where $A$ is a real matrix, $B$ is a real, square and symmetric matrix, and $(AB)^\dagger$ is the Moore-Penrose pseudoinverse of $AB$. Is $\| B(AB)^\dagger \|_2$ ...
1answer
59 views

### Why can FFT accelerate the procedure to solve the ridge regression problem?

For a ridge regression problem $$\arg \min_x {\|Ax-b\|^2+\lambda\|x\|^2}$$ where $A$ is a symmetric matrix. The solution can be achieved through gradient-based iterative method like Gradient Descent ...
0answers
23 views

1answer
32 views

### Ridge Regression with two estimators

Given a loss $E$: $$E = (Y - X\beta)^T(Y - X\beta) + \lambda\left \| \beta \right \|^2$$ The value of $\beta$ that minimizes the loss can be obtained by setting $\frac{\partial E}{\partial \beta} =0$ ...
1answer
26 views

### Discrete regularisation

Consider the following least squares problem in $X$: $||AX-B||_2^2\rightarrow\min$, where $A$ and $B$ are known, real-valued matrices. Is it there a regularisation method which ensures that the ...
2answers
50 views

### Quadratic cost function solution [closed]

Why the solution to the following cost function: $$\frac{1}{2}\|Lm-d\|^2 + \frac{1}{2} \mu \|W_m m\|^2_2$$ the below equation: $$(L^Td+\mu W^T_m W_m)^{-1} L^Td$$
1answer
25 views

### Proof of the Moreau Envelope of $l_1$ norm [closed]

Given function $| \cdot | : \mathbb{R} \rightarrow \mathbb{R}_{+}$ and $\alpha > 0$, its Moreau envelope $e_{\alpha}|\cdot|: \mathbb{R} \rightarrow \mathbb{R}_{+}$, reads: \begin{equation} e_{\...
0answers
17 views

### A simple book to learn Regularization

I am new to regularization and optimization and I am going to learn more about it. I am looking for some useful resources to learn these subjects. Is there any books, lecture notes, websites etc. that ...
1answer
58 views

### What's meant by “regularization” of ODEs?

What's meant by "regularization" of ODEs? Such as, "in order to be solved by conventional ODE solvers such as ode45"? The context where I encountered this was related to ...
0answers
18 views

0answers
20 views

### Problem in replicating graphical lasso study

I was learning graphical lasso algorithm, and trying to replicating the study in book "Machine Learning: A Probabilistic Perspective", P940. I downloaded the same dataset cited in the book ...
0answers
28 views

### Iteratively reweighted least squares for LASSO problem

I'm trying to solve the following (here simplified) problem (here 1D, and $x>0$): $$\arg \min_{x} \frac{1}{2} \left\| A x - b \right\|_{2}^{2} + \frac{1}{2} \left\| x \right\|_{1}$$ I need to ...
0answers
21 views

### Regularization with range of hyperparameter in [0,1]?

In regularization theory, we frequently minimize expressions with the following form: $$x + \lambda y,\ \ \lambda \in \mathbb{R}^+$$ where $x, y$ represent some type of functional and \$\lambda \in \...
0answers
21 views

0answers
21 views

### What are the standard ways of deriving and verifying the formulas for integral transforms where the formal formula for the transform diverges?

There are multiple formulas for integral transforms of various functions in the tables of integral transforms, but in many cases the integral, formally representing the transform diverges. What is the ...