Questions tagged [regularization]

Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: http://en.wikipedia.org/wiki/Regularization_(mathematics))

Filter by
Sorted by
Tagged with
4
votes
1answer
84 views

Heat Kernel on a compact manifold without boundary

I was wondering if the conservation of mass which is obvious for the heat equation in $\mathbb{R}^n$ holds also for the heat kernel in a general compact manifold without boundary. I mean I want to ...
1
vote
0answers
37 views

Regularization for function in $S^{d-1}$

Given a function $f:S^{d-1}\rightarrow \mathbb{R}$ where $S^{d-1}:=\{x\in \mathbb{R}^d: |x|=1\},$ I want to find a sequence of continuous functions $f_n$ converging to $f$. I thought that maybe, given ...
0
votes
0answers
6 views

Minimum of regularized cost function for a movie rating system

I'm currently reading Rafael Irizarry's Machine Learning notes and I was a little confused with the approach towards regularization. To analyse a movie recommendation system, the following equation is ...
0
votes
0answers
100 views

Do these constants appear in other areas of mathematics?

I decided to consider divergent (to infinity) integrals as some new kind of number. Towards this end, I began by establishing certain rules defining equivalence of the integrals. It seems the usual ...
2
votes
0answers
44 views

Dos the Euler-Mascheroni constant $\gamma$ correspond to infinite hyperbolic angle?

So, I am trying to find the regularized value of the divergent integral $I=\int_1^\infty \sqrt{x^2-1}dx$. Since the area of $\int_0^1 \sqrt{1-x^2}dx=\frac\pi4$, I wonder whether the area under ...
0
votes
0answers
85 views

Integral over real domain, 0 to $\frac12$, of the Inverse of the Generalized Regularized Incomplete Beta Function. Exact form. All arguments are x.

Here is the link for the main function used. Let: $$\mathscr {I_2}\mathrm{=\int_0^\frac 12I^{-1}_{(x,x)}(x,x)dx=\int_0^\frac12I^{-1}_{I_x(x,x)+x}(x,x)= \int_0^\frac12I^{-1}_{\left(\frac12-x,\frac12-x\...
2
votes
0answers
68 views

Integral over the inverse of the generalized regularized incomplete gamma function over its domain with all 3 arguments equated.Non-integral form hard

I said before that the upper bound was one. As I do not know the complex behavior of this function, it would be even harder to integrate past the real domain. Please try the three “argumented” inverse ...
1
vote
0answers
24 views

Why do we subtract $1$ in this definition of the shannon entropy?

In the context of entropic regularization, the (negative) entropy is involved in such way (last term): Why do we subtract $-1$? So far, I have only known of the entropy being $\sum x_{i,j}\log(x_{i,j}...
6
votes
0answers
318 views

Interesting Simple Integral Over the Unit Interval of the Inverse Regularized Incomplete Gamma Function. Non-Integral Form Needed. Closed Form Option.

Math Overflow Version! I have recently used, $\Bbb {here,}$ Here is a link to this function on the Wolfram Function World website. For simplicity, let the unit interval be expressed as $I=[0,1]$: $$...
3
votes
1answer
55 views

The Wasserstein Metric. Computational Optimal Transport. Weights.

Let $\mu,\nu$ be two probability measures on the space $\mathbb{R}^n$, and let $\Pi(\mu,\nu)$ be the space of joint probability measures with marginals $\mu$ and $\nu$. After a discretisation of space ...
1
vote
1answer
16 views

Interpretation of the relation between regularized least-squares and minimum-norm solution for an underdetermined system

For a linear system $Ax = y$, define $J_1 = ||{Ax-y}||^2$ and $J_2 = ||x||^2$. We wish to minimize the weighted-sum objective $J_1 + \mu J_2$. If we interpret $J_1$ as a cost function and $J_2$ as an ...
0
votes
1answer
75 views

Is $\| B(AB)^\dagger \|_2 $ uniformly bounded for all positive diagonal matrices $B$?

Consider $\| B(AB)^\dagger \|_2$ where $A$ is a real matrix, $B$ is a real, square and symmetric matrix, and $(AB)^\dagger$ is the Moore-Penrose pseudoinverse of $AB$. Is $\| B(AB)^\dagger \|_2$ ...
4
votes
1answer
59 views

Why can FFT accelerate the procedure to solve the ridge regression problem?

For a ridge regression problem $$ \arg \min_x {\|Ax-b\|^2+\lambda\|x\|^2} $$ where $A$ is a symmetric matrix. The solution can be achieved through gradient-based iterative method like Gradient Descent ...
1
vote
0answers
23 views

Showing a renormalization result from an introductory paper on Feynman integrals

I am reading a paper on Feynman integrals, where the author states that from the following equation (96), $$Z_g=1-\frac12\beta_0\frac{g_R^2}{(4\pi)^2}+O(g_R^4),$$ we obtain the equation (99) $$Z_g^{-1}...
0
votes
1answer
81 views

How can a fractional logarithm be defined and calculated?

For context, I am looking back at an old question regarding a curious family of recursively defined logarithm towers. What slightly unsettles me is the "discrete jump" between each curve as $...
0
votes
0answers
9 views

Kernal or basis expansions for 0 centred data

Im currently looking at a regression problem and have considered several machine learning techniques. The issue present in most methods is the fact that the desired objective value is centered at zero ...
1
vote
2answers
155 views

Implementing logistic regression with L2 penalty using Newton's method in R

I need to implement Logistic Regression with L2 penalty using Newton's method by hand in R. After asking the following question: second order derivative of the loss function of logistic regression and ...
0
votes
1answer
115 views

second order derivative of the loss function of logistic regression

For the loss function of logistic regression $$ \ell = \sum_{i=1}^n \left[ y_i \boldsymbol{\beta}^T \mathbf{x}_{i} - \log \left(1 + \exp( \boldsymbol{\beta}^T \mathbf{x}_{i} \right) \right] $...
1
vote
0answers
35 views

Set of divergent integrals

So, we take $\frac{\text{sgn}(x-1)}{x}$ and apply $\mathcal{L}_t[t f(t)](x)$ four times. The transform keeps area. These integrals are minus Euler-Mascheroni constant: $$\int_0^\infty \frac{\text{sgn}(...
4
votes
1answer
143 views

Why is $\ln 0\ne-\ln \infty$?

The title of this post is intentionally sensational, but what I am really going to do is to compare the divergent integrals $\int_0^1\frac1xdx$ and $\int_1^\infty\frac1xdx$. Let's consider the ...
1
vote
0answers
13 views

Regularized optimization with a Poisson/Binominal count data model

I'm currently solving an optimization problem (least squares with an added regularization and inequality constraint) of the form: \begin{equation} \min_{\mathbf{f}} \quad \{ ||G \mathbf{f} - \mathbf{...
0
votes
0answers
20 views

Uniform convergence of a Mollifier Sequence $f_{\rho}:=g_{\rho}*f$

If I have a function $g_{\rho} \in C_c^{\infty}(\mathbb{R^n})$ $($i.e $C^{\infty}$ function compactly supported in $\mathbb{R^n}$$)$. Defined by: $g_{\rho}:=\frac{1}{\rho^n}g(\frac{x}{\rho})$, where $...
0
votes
0answers
53 views

Weighted L1-norm based Contextual Regularization

I am Japanese. I'm not good at English, but thank you. I am currently reading the following treatise. "Efficient Image Dehazing with Boundary Constraint and Contextual Regularization",...
1
vote
1answer
15 views

Cauchy Principal Value for Poles stronger than 1

While calculating the Cauchy Principal Value of $\int_{-\infty}^{+\infty}\frac{\sin x}{x}dx$ we take an indented semicircle (in the upper half plane) of radius $\delta$ centered at the simple pole of $...
1
vote
1answer
32 views

Ridge Regression with two estimators

Given a loss $E$: $$E = (Y - X\beta)^T(Y - X\beta) + \lambda\left \| \beta \right \|^2$$ The value of $\beta$ that minimizes the loss can be obtained by setting $\frac{\partial E}{\partial \beta} =0$ ...
3
votes
1answer
26 views

Discrete regularisation

Consider the following least squares problem in $X$: $||AX-B||_2^2\rightarrow\min$, where $A$ and $B$ are known, real-valued matrices. Is it there a regularisation method which ensures that the ...
2
votes
2answers
50 views

Quadratic cost function solution [closed]

Why the solution to the following cost function: $$\frac{1}{2}\|Lm-d\|^2 + \frac{1}{2} \mu \|W_m m\|^2_2$$ the below equation: $$(L^Td+\mu W^T_m W_m)^{-1} L^Td $$
0
votes
1answer
25 views

Proof of the Moreau Envelope of $l_1$ norm [closed]

Given function $| \cdot | : \mathbb{R} \rightarrow \mathbb{R}_{+}$ and $\alpha > 0$, its Moreau envelope $e_{\alpha}|\cdot|: \mathbb{R} \rightarrow \mathbb{R}_{+}$, reads: \begin{equation} e_{\...
0
votes
0answers
17 views

A simple book to learn Regularization

I am new to regularization and optimization and I am going to learn more about it. I am looking for some useful resources to learn these subjects. Is there any books, lecture notes, websites etc. that ...
1
vote
1answer
58 views

What's meant by “regularization” of ODEs?

What's meant by "regularization" of ODEs? Such as, "in order to be solved by conventional ODE solvers such as ode45"? The context where I encountered this was related to ...
0
votes
0answers
18 views

Partially convex function regularisation preserving partial convexity

Here, $K_x \subset \mathbb{R}^n$ and $K_y \subset \mathbb{R}^m$ are assumed to be compact sets. For clarity, I refer to partial convexity as follows: [Partially convex] Function $f:K_x \times K_y \...
2
votes
1answer
31 views

What does “a function is minorised” mean?

I'm reading a paper about Lasry-Lions regularisation. In Theorem 1 of the paper, considered function $f$ is assumed to be minorised. What does it mean? EDIT: Is it equivalent to "bounded below&...
1
vote
1answer
38 views

Intuition behind optimzation problem of ridge regression?

In one of the texts that I am reading it is given that regularization parameter restricts the choice of functions in case data given is not sufficient for processing of signal. It is given that lambda ...
1
vote
0answers
52 views

Evaluate $\zeta_F(0)$ where $F = \mathbb{Q}(\sqrt{-23})$

Using the modular forms database ([1], [2]) I could find the coefficients of the zeta function corresponding to the number field $F = \mathbb{Q}(\sqrt{-23}) \simeq \mathbb{Q}(x)/(x^2 - x +6)$. We ...
0
votes
0answers
15 views

Is Moreau-Yosida regularisation Lipschitz continuous?

I'm reading a paper. In the proof of theorem 2, the authors refer to the Lipschitzness of Moreau-Yosida regularisation with citation. Like this: For convex function $f$, Moreau-Yosida regularisation $...
0
votes
0answers
20 views

Problem in replicating graphical lasso study

I was learning graphical lasso algorithm, and trying to replicating the study in book "Machine Learning: A Probabilistic Perspective", P940. I downloaded the same dataset cited in the book ...
1
vote
0answers
28 views

Iteratively reweighted least squares for LASSO problem

I'm trying to solve the following (here simplified) problem (here 1D, and $x>0$): $$ \arg \min_{x} \frac{1}{2} \left\| A x - b \right\|_{2}^{2} + \frac{1}{2} \left\| x \right\|_{1}$$ I need to ...
0
votes
0answers
21 views

Regularization with range of hyperparameter in [0,1]?

In regularization theory, we frequently minimize expressions with the following form: $$x + \lambda y,\ \ \lambda \in \mathbb{R}^+$$ where $x, y$ represent some type of functional and $\lambda \in \...
0
votes
0answers
21 views

How regularization term turn ill-possed to well possed problem?

Lets say we have following non-linear convolution equation of F and X with V noise $$ Y = F \ast X + V $$ to solve inverse of it, we use following equation with L2 norm $$ X = arg min ||F \ast X-Y||^...
1
vote
3answers
294 views

Regularized sum of $1+(1+\frac1{2^2}) + (1+\frac1{2^2}+\frac1{3^2})+\cdots$, or $\sum_{k=1}^\infty \sum_{j=1}^k\frac1{j^2}$?

For my exercises on divergent summation I try to find a regularization of $$ S^{^\star}_2 \underset{\mathcal R}= \sum_{k=1}^\infty \sum_{j=1}^k \frac1{j^2} \tag 1 $$ It is not difficult to derive ...
1
vote
0answers
51 views

How do I mollify an indicator function with $x \in$ $\mathbb{R}^n$?

Good morning, I have to find this function : $\gamma (x) = 1, \left | x \right | \leqslant 1 , x \in \mathbb{R}^n$, $\gamma \in C^\infty (\mathbb{R}^n)$ On some subset of $\mathbb{R}^n$ we can find ...
0
votes
0answers
14 views

How to regularize sparsely sampled vector field without expansion on grids and interpolation?

I have worked with many energy minimization problems for differential operators working on vector fields discretized on dense grids. In this scenario we can often regularize by vectorizing the vector ...
0
votes
0answers
51 views

Find a function that is close to the indicator function and infinite derivative

Good afternoon, I need to show that there is a function $\gamma$ that is infinite derivative and where every element of a subset of $\mathbb{R}^n$ gives $\gamma (x) = 0$ Also, this function must ...
0
votes
0answers
23 views

Constraint minimization to find the penalty term

Problem: Assume that the predictors are centered. Let $RSS = \sum_{i=1}^n (y_i-\beta_0-\beta_1 x_{i1}-\beta_2 x_{i2}-....-\beta_p x_{ip})^2$ and $β = (β_1, . . . , β_p)$. Consider minimizing RSS w.r.t....
1
vote
0answers
51 views

1D analytical solution for entropy-regularized Wasserstein distance

\begin{align} \mathcal{W}_{p}(\mu, \nu) &=\left(\int_{X} d^{p}\left(x, F_{\nu}^{-1}\left(F_{\mu}(x)\right)\right) \mathrm{d} \mu(x)\right)^{\frac{1}{p}} \\ &=\left(\int_{0}^{1} d^{p}\left(F_{\...
1
vote
0answers
68 views

Are there any particular reasons why the minimum norm solution should be preferred?

Let us consider a linear system of equations of the form: $$ Ax = b, $$ where $A \in \mathbb{R}^{m\times n}$ and $b \in \mathbb{R}^m$. Such a linear system naturally arises in many fields of ...
1
vote
0answers
34 views

Regularization theorem :$ AC[0,1]$ to $\mathcal{C}^1[0,1]$

For a given absolutely continuous function $x\in AC([0,1],\mathbb{R}^d)$ i.e : there is $f\in L^1[0,1]$ such that : $$x(t)= x(0)+ \int_0^tf(s)ds.$$ Can I find a continuously differentiable $y$, such ...
2
votes
1answer
123 views

If $\alpha = \beta$, why can't the entropy-regularized Wasserstein distance equal $0$?

In optimal transportation theory, the optimal re-allocation of probability distribution $\alpha$'s mass to another distribution $\beta$ is solved by minimizing the Wasserstein distance with respect to ...
0
votes
0answers
26 views

Is there an indefinite sum formula, based on Abel-Plana formula?

I wonder if there is an expression for indefinite sum based on Abel-Plana formula? Faulhaber regularization: $$\operatorname{reg} \sum _{n=0}^{\infty} f(n)= -\sum_{n=1}^{\infty} \frac{f^{(n-1)} (0)}{n!...
1
vote
0answers
21 views

What are the standard ways of deriving and verifying the formulas for integral transforms where the formal formula for the transform diverges?

There are multiple formulas for integral transforms of various functions in the tables of integral transforms, but in many cases the integral, formally representing the transform diverges. What is the ...

1
2 3 4 5
7