Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: ...

learn more… | top users | synonyms

1
vote
0answers
121 views

Zeta regulated product, solving without the zeta function.

Earlier i've asked about how to calculate divergent products, i got some directions which made me curious. Now i'm wondering is this correctly done. Divergent products. The most commen divergent ...
7
votes
0answers
47 views

Does such divergent integral assume the same values for any regularization?

Consider the integral: $$\int_0^\infty\sin(x)dx.\tag1$$ It's clearly divergent, but if we regularize it as $$\int_0^\infty\sin(x)e^{-x/a}dx=\frac{a^2}{a^2+1},\tag2$$ we can take the limit of ...
0
votes
0answers
35 views

How to solve the least square with $L_2$ norm constraint directly?

I answered the question Why are additional constraint and penalty term equivalent in ridge regression? earlier, but I myself still have some questions on it. To solve \begin{align} \min_{\beta} ...
0
votes
0answers
20 views

Dependence of finite part of integral on regularization

Recently I got stuck with some task in which integral $$ I_{\alpha \beta}(r, q) = \int \frac{d^{4}p}{(2 \pi )^{4}}\frac{p_{\alpha}(p_{\beta} + r_{\beta})}{((p - q)^{2} - m_{W}^{2})((p + q)^{2} - ...
0
votes
0answers
24 views

Parabolic PDE with diffusion matrix of zero determinant

Consider a Fokker-Planck type PDE in $\mathbb{R}^2$: \begin{equation} \partial_t\rho=\mathrm{div}(\rho\nabla V)+ D^2:\left[\sigma\rho\right] \hspace{2cm} (*) \end{equation} where we have the ...
1
vote
0answers
17 views

Estimating Markov transition matrix for regularization

Suppose that I have a sequence of discrete distributions: $$ p_j = (p_{1j},...,p_{Cj}), \: j=1...D,\\ p_{ij}>0 \:\: \forall i,j,\: \sum_{k=1}^Cp_{kj}=1\:\:\forall j. $$ I suppose that these ...
1
vote
0answers
20 views

About consistency in an inverse problem formulation

I'm a beginner with inverse problems and I was reading about regularization techniques. Consider the problem: $$d=Kf_{\text{true}}$$ $d$ is a data vector, $K$ is an linear operador, $d=\hat{d}+\eta$ ...
0
votes
1answer
27 views

Relation between Regularization and correlation

I was going through Chapter 3 (page 63 bottom) of Elements of Statistical Learning. While explaining regularization in ridge regression authors make the following statements. "When there are many ...
1
vote
1answer
41 views

Solve the following matrix equation $X'X=A$

I have square matrices $X$,$A$ and $X'X-A=0$. $A$ is given and is positive definite and I need to get matrix $X$. I know $X$ is not unique since $TX$ such that $T'T=I$ will satisfy. My problem is ...
1
vote
1answer
43 views

How do I set a lower bound to the solution's norm in a QP problem

I know that LASSO-regularization can be used to scale into an $L_1$ upper bound for a solution. But what if I want the norm to be within a specific range $[a,b]$? ie. I also want to set a lower bound? ...
0
votes
0answers
13 views

Basis functions which allow non-negativity to be enforced?

I'm working on an inverse problem where I am trying to estimate a probability distribution $f(x)$ as a sum of basis functions: $$ f(x) = \sum_{i}^{M} c_i \phi_i(x) $$ Where the $\mathbf{c}$ are ...
2
votes
1answer
148 views

Solving L1 regularized Joint Least Squares and Logistic Regression

My objective function that is to be minimized is as follows: $f = -\sum_{n=1}^{N}log~p(y_{n}^{a}|x_{n},w) + \sum_{n=1}^{N}(y_{n}^{b}-w^{T}x_{n})^{2} +\lambda\|w\|_1$ The first term models the ...
2
votes
1answer
86 views

When do regularization methods for divergent series disagree?

Sometimes, it is possible to take a divergent series (in the sense of its sequence of partial sums failing to converge) and "regularize" it using one of a variety of methods to assign it a meaningful ...
4
votes
1answer
190 views

Tikhonov regularization vs truncated SVD

To find $\mathbf{x}$ such that $$A\mathbf{x}=\mathbf{b}$$ we can use least squares when the problem is not well posed. Further, we can use Tikhonov regularization when $A$ is ill-conditioned. In ...
0
votes
0answers
18 views

regularization of matrix

I am trying to find a solution for $x$ in the equation $$Ax=y$$ in which $A$ is a underdetermined matrix. After searching on the ineternet I got an idea about regularization using $L_0,L_1$ and ...
0
votes
0answers
39 views

sum of divisors function $\sum \tau(n) = \frac{1}{4}$

These notes on multiplicative number theory mention the convolution $ 1 \ast 1 = \tau$ (where $\tau$ is the divisor function not Ramanujan tau function. Therefore $$ \bigg(\sum \frac{1}{n^s} ...
1
vote
1answer
80 views

Zeta and Gamma function regularization with $\omega=1/0$

I have recently read about zeta function regularization, a way of ascribing values to functions having simple poles in a point and to divergent series. The values obtained are the same as those ...
4
votes
1answer
142 views

Is there a metric in which $1+2+3+4+\cdot$ converges to $-\frac1{12}$?

It is well known that the sum $1+2+3+4+\ldots$, which tends to infinity in the regular sense, can be assigned the value $-\frac{1}{12}$ by different means, e.g., zeta regularization or different ...
0
votes
1answer
49 views

Optimization Problem (Linear Algebra)

I am not trying to cheat or anything, so any reference to online literature or MOOCs, that teach this stuff, will be highly appreciated. The problem is to prove that the following optimization ...
2
votes
0answers
26 views

Can this divergent integral transform be regularized?

The integral $$\int_0^{\infty} e^u \ K_{i t}(u) du$$ is the adjoint Kontorovich-Lebedev transform of the increasing exponential function, but unfortunately this integral is divergent because $$e^u \ ...
0
votes
1answer
15 views

zeta regularization separation of series

in the sense of infinite series and for an integer 'a' is then correct that $$ \sum_{n=1}^{\infty}n^{k} = \sum_{n=1}^{a}n^{k}+ \sum_{n=a+1}^{\infty}n^{k} $$ opther that works only when ยท$ re(k) > ...
0
votes
0answers
28 views

Regularization of equivalent optimization problems

In portfolio optimization three equivalent optimization problems exist. I am wondering if they are still equivalent when regularized, e.g by ridge regularization. E.g. are the following equivalent ...
2
votes
0answers
79 views

Fourier transform of an exponentially singular radial function

I am trying to compute the 3D Fourier transform of a spherically symmetric function of the form $$f(r) = e^{\frac{1}{r} e^{-r}} - 1\, ,$$ which entails the integral $$\begin{aligned}F(k) =& \int ...
0
votes
1answer
13 views

Support of vector $w$ in graph sparsity

I'm reading about graph sparsity and I have one problem in a paper I'm reading I don't understand, maybe someone can clarify: Graph Sparsity: In graph sparsity, we have a directed acyclic graph ...
0
votes
1answer
28 views

Differentiable L-1 Regularization

In machine learning we are often faced with optimization problems where we want to minimize some energy function using L1 regularization over some of the parameters, e.g.: $$ E(a,w) = [\text{sum of ...
0
votes
1answer
45 views

Regularization of underdetermined system to favour low frequency solutions?

Consider the ill-posed system $$ \mathbf A \mathbf x= \mathbf b.$$ One method to regularize the solution is the Tikhonov method which effectively minimizes $ ||\mathbf A \mathbf x - \mathbf b ||^2 + ...
0
votes
0answers
88 views

Function smoothing using convolution

I have a function $\hat f$ which is an estimator of an unknown function $f$. The estimator $\hat f$ looks pretty irregular (see the red line). I would like to smooth it with some kernel function ...
1
vote
1answer
33 views

Integral in regularization involving exp of a parameter in the denominator

Can someone please point me a way to compute $$\int_0^\infty\frac1{s+t}\exp\left(-\alpha t+\frac{t^2\beta}{s+t}\right)dt$$ ? How about the following one? $$\int_0^\infty ds\int_0^\infty ...
4
votes
1answer
99 views

Is it possible to sum the divergent series with prime coefficients?

This is a follow-up of this question. It is known that the divergent series $$ P := \sum_{n=1}^\infty p_n \qquad \text{where } p_n \text{ is the $n$th prime} $$ cannot be summed by means of (prime) ...
0
votes
1answer
116 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
6
votes
0answers
194 views

Connection between integral expression and the factorial of infinity

Does the fact that $$\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}x^2\right)\mathrm{d}x=\sqrt{2\pi}$$ Have something to do with the fact that the regularized factorial of infinity is also ...
1
vote
0answers
48 views

When does l1 regularisation give a sparse solution?

I was maximising a likelihood function, which is convex. I know that the system has a K-sparse solution. I wanted to know the conditions (or some sufficient conditions) on the likelihood function ...
1
vote
0answers
82 views

Regularity of semilinear heat equation

I'm facing following regularity issue and i wonder if anyone of you guys is able to help me. I'd like to show that the solution of a semilinear heat equation is classical, i.e. $C^2$ in space and ...
1
vote
1answer
95 views

Assigning values to divergent integrals

I'm interested in the (obviously divergent) integral $$ \int_{-\infty}^\infty dx e^{-x f}\ ,$$ where $f$ is real. Is there any way to meaningfully assign a value to this integral? I was thinking of ...
1
vote
0answers
131 views

Implementation of Total Variation Regularization Algorithm (Lagged Diffusivity Algorithm)

I am trying to compute the derivative of an experimentally-measured quantity as a function of time. The data are fairly noisy, which causes problems. For instance, using finite differences (central ...
-3
votes
1answer
60 views

Eigenvalues and Eigenvectors of a singular Covariance matrix

I am working on a research in which my data matrix $\bf X$ has dimension of $N\times P$ where $P>>>>N$.ie. its a small sample size problem. I need to compute the covariance of $\bf X$, ...
1
vote
0answers
21 views

integration of a multiple Laurent series

let be an multiple integral given by $$ \int_{0}^{\infty}dx_{1}\dots\int_{0}^{\infty}dx_{n}F(x_{1} ,x_{2},...x_{n}) $$ i have a question can i solve this integral equation by exapnding the integrand ...
4
votes
1answer
123 views

a doubt with the series $ \sum_{n=0}^{\infty}e^{-nx} $

I know that the series is equal to $$ \sum_{n=0}^{\infty}e^{-nx}= \frac{1}{1-e^{-x}}$$ However, if I expand each exponential term into a Taylor series I get $$ \sum_{n=0}^{\infty}e^{-nx}= ...
2
votes
0answers
45 views

Regularized least squares

In Image Restoration, a true image $f$ (in vector form) can be related to degraded data $y$ through a linear model of the form $$y = Hf + n$$ where $H$ is a 2D blurring matrix and $n$ is a noise ...
0
votes
0answers
18 views

$C^\infty$ approximations of $f(r) = |r|^{m-1}r$ [duplicate]

Consider $f(r) = |r|^{m-1}r$ where $m \geq 1$. Is it possible to find $C^\infty$ functions $f_n$, such that $f_n \to f$ uniformly on compact subsets of $\mathbb{R}$, $f_n' \to f'$ uniformly on ...
5
votes
4answers
235 views

Find the regularized sum $\frac{1}{\ln(2)}+\frac{1}{\ln(3)}+\frac{1}{\ln(4)}+…$

By considering the integral Zeta function $$F(s)=s+\frac{1}{2^s\ln(2)}+\frac{1}{3^s\ln(3)}+\frac{1}{4^s\ln(4)}+...$$ Evaluate $$\frac{1}{\ln(2)}+\frac{1}{\ln(3)}+\frac{1}{\ln(4)}+...$$ EDIT: ...
0
votes
0answers
24 views

Regularization for unordered vectors

Let suppose we have two vectors u, v $\in \mathbb{R}^n$ and we want a function that returns $0$ if the ordering of the elements of both vectors are the same or a positive number otherwise, where the ...
2
votes
1answer
407 views

Understanding Regularization parameters in Machine Learning/Statistics

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...
0
votes
2answers
110 views

Regex for strings with at least three unique characters

I'm trying to represent some string conditions in terms of regex. One of those conditions I find hard to transform is that the string must have at least three different characters. So is there any ...
1
vote
0answers
37 views

proving that the series $ 1+2^{s}+3^{s}++ $ is divergent but borel summable

suppose that in the sense of distribution $ \int_{0}^{\infty}dxx^{n}T(x,s) =n^{s} $ for some distribution $ T(x) $ i do not know :( so if we apply borel generalized resummation $$ ...
3
votes
1answer
121 views

Euler-Maclarurin summation formula and regularization

let be $ f(x)= x^{a} $ with $ -1<a<0 $ if i use Euler Maclaurin summation formula $$ \int_{1}^{\infty}dxx^{a}= \frac{1}{2}+ ...
1
vote
1answer
34 views

determinant property of an operator

for a matrix we know that $$\det(aA)=a^n \det(A) $$ but what happens for an INFINITE dimensional operator ?? should we have $$\det(aA)=a^{Z(0)}\det(A) $$ $$ Z(s)= \sum_{n=0}^\infty \lambda_n^{-s} ...
1
vote
0answers
159 views

Second-Order Tikhonov Regularization

In the second-order Tikhonov regularization approach $\min\left\|Gm - d \right\|_2^2 + \alpha\ ^2 \left\|\Gamma x\right\|_2^2$ (1) given that $\Gamma\ $ contains second order derivatives, ...
0
votes
1answer
246 views

The gradient of the standard mollifier

Please check my proof for the following result: I want to prove a result for $D\eta_{\epsilon}$ the gradient of the standard mollifier $\eta$. The function $\eta$ is defined as follows: Let ...
1
vote
1answer
270 views

Uniform convergence with Lp functions

I have a convergence question: Say we have a sequence of functions $\{u_{m}\}_{m=1}^{\infty}$ where $\{u_{m}\}_{1}^{\infty} \subset L^{p}(U)$ and where $U$ is bounded. Consider $u^{\epsilon} := ...