Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: ...

learn more… | top users | synonyms

0
votes
0answers
12 views

Upper estimate between an original function and its sup-convolution under a limitation

My setting maybe look rather special but I'm glad if you give some answers. Let $f:[0,1]\to\mathbb{R}$ be a bounded, upper semicontinuous function and $f^{\varepsilon}:[0,1]\to\mathbb{R}$ be $f$'s ...
0
votes
0answers
15 views

Want to factorize one matrix into three, with L1 regularization, which optimization algorithm to choose?

I need to factorize one matrix $R$ into three component: $ R = P^TAQ $, in which I want to apply L1 regularization on $A$ to encourage sparsity, and apply L2 regularization on $P$ and $Q$ to prevent ...
0
votes
0answers
16 views

Regularize the sum: $\sum_{n=0}^\infty (-1)^{n+1}({2+4n})$

$$\sum_{n=0}^\infty (-1)^{n+1}({2+4n})=2-6+10-14\cdots$$ is a clearly divergent sum, which I am trying to regularize. I need to figure out this sum because I have been trying to solve the sum integral ...
1
vote
1answer
21 views

l1 regularized minimization with equality constraint in ADMM

In section 6.3 of this note there is a method for minimizing a loss function with l1 regularization. i.e. minimize $l(\bf{x})+\lambda||x||_1$ How can I add the equality constraint $\sum\limits_{i} ...
0
votes
2answers
16 views

How to prove non-regularity of a language from the non-regularity of another language?

How can I prove that $L_1=\{a^nb^m\mid n\ne m\}$ is not regular based on the fact that the language $L_2=\{a^nb^n\mid n\in\Bbb N\}$ is not regular? Thank you
1
vote
0answers
12 views

Why divide regularization factor by size of dataset?

Suppose I'm trying to minimize a cost function: $$ J(\theta) = \frac {1} {2m} \sum _{i = 1}^ m (h_\theta (x^{(i)}) - y^{(i)})^2 $$ Adding regularization, as seen here, we get: $$ J(\theta) = \frac ...
2
votes
1answer
28 views

Convex Optimization - nuclear norm regularisation of symmetric matrix

I have a problem of the form $\min_{X\in \mathbb{R}^{n \times n}} g(X) - \lambda ||X||_{*}$ where $g$ is convex and differentiable. I would like to use proximal gradient descent to solve this. How ...
0
votes
0answers
65 views

Solve the problem $Ax = 0$ when $A$ has full rank.

Generally, the answers $x$ of this least square problem $$Ax = 0$$ where $A = []_{m\times n}$ and $x = []_{n\times 1}$ are in the null space of $A$. I know that people usually use the right-most ...
9
votes
1answer
274 views

Is $\prod_{n=1}^\infty P_{2n-1}$ regularizable?

Assume that $P_n$ denotes the $n$'th prime for this entire question. Inspriation: I was dumbfounded by the fact that: $$\hat\prod_\limits{n=1}^\infty P_{n}=4\pi^2$$ After further investigation, I ...
1
vote
0answers
14 views

Extension of $C^1$ function (regularity of mollification on boundary)

In my study I have faced the problem showing the property below: regularity of mollification on boundary. (However, I don't know whether this is true or not although I hope.) Let $f\in ...
5
votes
0answers
108 views

Prove the estimator $\hat{B}$ of ridge regression = mean of the posterior distribution under a Gaussian prior

I want to prove that the estimator of ridge regression is the mean of the posterior distribution under Gaussian prior. $$y \sim N(X\beta,\sigma^2I),\quad \text{prior }\beta \sim N(0,\gamma^2 I).$$ ...
0
votes
0answers
15 views

Penalty Term on Negative Matrix Elements for Gradient Descent Objective Function?

In fact, I'm doing a gradient ascent to estimate the elements of a matrix $X=f(A)$. Nevertheless, I've written "descent" into the title, because I think that this is the common term. Motivational ...
2
votes
2answers
179 views

regularization of sum $n \ln(n)$

I was testing out a few summation using my previous descriped methodes when i found an error in my reasoning. I'm really hoping someone could help me out. The function which i was evaluating was ...
1
vote
0answers
152 views

Zeta regulated product, solving without the zeta function.

Earlier i've asked about how to calculate divergent products, i got some directions which made me curious. Now i'm wondering is this correctly done. Divergent products. The most commen divergent ...
7
votes
0answers
55 views

Does such divergent integral assume the same values for any regularization?

Consider the integral: $$\int_0^\infty\sin(x)dx.\tag1$$ It's clearly divergent, but if we regularize it as $$\int_0^\infty\sin(x)e^{-x/a}dx=\frac{a^2}{a^2+1},\tag2$$ we can take the limit of ...
0
votes
0answers
43 views

How to solve the least square with $L_2$ norm constraint directly?

I answered the question Why are additional constraint and penalty term equivalent in ridge regression? earlier, but I myself still have some questions on it. To solve \begin{align} \min_{\beta} ...
0
votes
0answers
28 views

Parabolic PDE with diffusion matrix of zero determinant

Consider a Fokker-Planck type PDE in $\mathbb{R}^2$: \begin{equation} \partial_t\rho=\mathrm{div}(\rho\nabla V)+ D^2:\left[\sigma\rho\right] \hspace{2cm} (*) \end{equation} where we have the ...
1
vote
0answers
22 views

Estimating Markov transition matrix for regularization

Suppose that I have a sequence of discrete distributions: $$ p_j = (p_{1j},...,p_{Cj}), \: j=1...D,\\ p_{ij}>0 \:\: \forall i,j,\: \sum_{k=1}^Cp_{kj}=1\:\:\forall j. $$ I suppose that these ...
1
vote
0answers
27 views

About consistency in an inverse problem formulation

I'm a beginner with inverse problems and I was reading about regularization techniques. Consider the problem: $$d=Kf_{\text{true}}$$ $d$ is a data vector, $K$ is an linear operador, $d=\hat{d}+\eta$ ...
0
votes
1answer
34 views

Relation between Regularization and correlation

I was going through Chapter 3 (page 63 bottom) of Elements of Statistical Learning. While explaining regularization in ridge regression authors make the following statements. "When there are many ...
1
vote
1answer
51 views

Solve the following matrix equation $X'X=A$

I have square matrices $X$,$A$ and $X'X-A=0$. $A$ is given and is positive definite and I need to get matrix $X$. I know $X$ is not unique since $TX$ such that $T'T=I$ will satisfy. My problem is ...
1
vote
1answer
52 views

How do I set a lower bound to the solution's norm in a QP problem

I know that LASSO-regularization can be used to scale into an $L_1$ upper bound for a solution. But what if I want the norm to be within a specific range $[a,b]$? ie. I also want to set a lower bound? ...
2
votes
1answer
329 views

Solving L1 regularized Joint Least Squares and Logistic Regression

My objective function that is to be minimized is as follows: $f = -\sum_{n=1}^{N}log~p(y_{n}^{a}|x_{n},w) + \sum_{n=1}^{N}(y_{n}^{b}-w^{T}x_{n})^{2} +\lambda\|w\|_1$ The first term models the ...
2
votes
1answer
106 views

When do regularization methods for divergent series disagree?

Sometimes, it is possible to take a divergent series (in the sense of its sequence of partial sums failing to converge) and "regularize" it using one of a variety of methods to assign it a meaningful ...
4
votes
1answer
340 views

Tikhonov regularization vs truncated SVD

To find $\mathbf{x}$ such that $$A\mathbf{x}=\mathbf{b}$$ we can use least squares when the problem is not well posed. Further, we can use Tikhonov regularization when $A$ is ill-conditioned. In ...
1
vote
0answers
61 views

sum of divisors function $\sum \tau(n) = \frac{1}{4}$

These notes on multiplicative number theory mention the convolution $ 1 \ast 1 = \tau$ (where $\tau$ is the divisor function not Ramanujan tau function. Therefore $$ \bigg(\sum \frac{1}{n^s} ...
1
vote
1answer
87 views

Zeta and Gamma function regularization with $\omega=1/0$

I have recently read about zeta function regularization, a way of ascribing values to functions having simple poles in a point and to divergent series. The values obtained are the same as those ...
6
votes
2answers
266 views

Is there a metric in which 1+2+3+4+… converges to -1/12?

It is well known that the sum $1+2+3+4+\ldots$, which tends to infinity in the regular sense, can be assigned the value $-\frac{1}{12}$ by different means, e.g., zeta regularization or different ...
0
votes
1answer
50 views

Optimization Problem (Linear Algebra)

I am not trying to cheat or anything, so any reference to online literature or MOOCs, that teach this stuff, will be highly appreciated. The problem is to prove that the following optimization ...
2
votes
0answers
35 views

Can this divergent integral transform be regularized?

The integral $$\int_0^{\infty} e^u \ K_{i t}(u) du$$ is the adjoint Kontorovich-Lebedev transform of the increasing exponential function, but unfortunately this integral is divergent because $$e^u \ ...
0
votes
1answer
21 views

zeta regularization separation of series

in the sense of infinite series and for an integer 'a' is then correct that $$ \sum_{n=1}^{\infty}n^{k} = \sum_{n=1}^{a}n^{k}+ \sum_{n=a+1}^{\infty}n^{k} $$ opther that works only when ยท$ re(k) > ...
2
votes
0answers
117 views

Fourier transform of an exponentially singular radial function

I am trying to compute the 3D Fourier transform of a spherically symmetric function of the form $$f(r) = e^{\frac{1}{r} e^{-r}} - 1\, ,$$ which entails the integral $$\begin{aligned}F(k) =& \int ...
0
votes
1answer
14 views

Support of vector $w$ in graph sparsity

I'm reading about graph sparsity and I have one problem in a paper I'm reading I don't understand, maybe someone can clarify: Graph Sparsity: In graph sparsity, we have a directed acyclic graph ...
0
votes
1answer
35 views

Differentiable L-1 Regularization

In machine learning we are often faced with optimization problems where we want to minimize some energy function using L1 regularization over some of the parameters, e.g.: $$ E(a,w) = [\text{sum of ...
0
votes
1answer
59 views

Regularization of underdetermined system to favour low frequency solutions?

Consider the ill-posed system $$ \mathbf A \mathbf x= \mathbf b.$$ One method to regularize the solution is the Tikhonov method which effectively minimizes $ ||\mathbf A \mathbf x - \mathbf b ||^2 + ...
1
vote
1answer
36 views

Integral in regularization involving exp of a parameter in the denominator

Can someone please point me a way to compute $$\int_0^\infty\frac1{s+t}\exp\left(-\alpha t+\frac{t^2\beta}{s+t}\right)dt$$ ? How about the following one? $$\int_0^\infty ds\int_0^\infty ...
4
votes
1answer
107 views

Is it possible to sum the divergent series with prime coefficients?

This is a follow-up of this question. It is known that the divergent series $$ P := \sum_{n=1}^\infty p_n \qquad \text{where } p_n \text{ is the $n$th prime} $$ cannot be summed by means of (prime) ...
0
votes
1answer
158 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
6
votes
0answers
213 views

Connection between integral expression and the factorial of infinity

Does the fact that $$\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}x^2\right)\mathrm{d}x=\sqrt{2\pi}$$ Have something to do with the fact that the regularized factorial of infinity is also ...
1
vote
0answers
53 views

When does l1 regularisation give a sparse solution?

I was maximising a likelihood function, which is convex. I know that the system has a K-sparse solution. I wanted to know the conditions (or some sufficient conditions) on the likelihood function ...
1
vote
0answers
101 views

Regularity of semilinear heat equation

I'm facing following regularity issue and i wonder if anyone of you guys is able to help me. I'd like to show that the solution of a semilinear heat equation is classical, i.e. $C^2$ in space and ...
1
vote
1answer
111 views

Assigning values to divergent integrals

I'm interested in the (obviously divergent) integral $$ \int_{-\infty}^\infty dx e^{-x f}\ ,$$ where $f$ is real. Is there any way to meaningfully assign a value to this integral? I was thinking of ...
1
vote
0answers
203 views

Implementation of Total Variation Regularization Algorithm (Lagged Diffusivity Algorithm)

I am trying to compute the derivative of an experimentally-measured quantity as a function of time. The data are fairly noisy, which causes problems. For instance, using finite differences (central ...
-3
votes
1answer
76 views

Eigenvalues and Eigenvectors of a singular Covariance matrix

I am working on a research in which my data matrix $\bf X$ has dimension of $N\times P$ where $P>>>>N$.ie. its a small sample size problem. I need to compute the covariance of $\bf X$, ...
1
vote
0answers
22 views

integration of a multiple Laurent series

let be an multiple integral given by $$ \int_{0}^{\infty}dx_{1}\dots\int_{0}^{\infty}dx_{n}F(x_{1} ,x_{2},...x_{n}) $$ i have a question can i solve this integral equation by exapnding the integrand ...
4
votes
1answer
128 views

a doubt with the series $ \sum_{n=0}^{\infty}e^{-nx} $

I know that the series is equal to $$ \sum_{n=0}^{\infty}e^{-nx}= \frac{1}{1-e^{-x}}$$ However, if I expand each exponential term into a Taylor series I get $$ \sum_{n=0}^{\infty}e^{-nx}= ...
2
votes
0answers
53 views

Regularized least squares

In Image Restoration, a true image $f$ (in vector form) can be related to degraded data $y$ through a linear model of the form $$y = Hf + n$$ where $H$ is a 2D blurring matrix and $n$ is a noise ...
0
votes
0answers
18 views

$C^\infty$ approximations of $f(r) = |r|^{m-1}r$ [duplicate]

Consider $f(r) = |r|^{m-1}r$ where $m \geq 1$. Is it possible to find $C^\infty$ functions $f_n$, such that $f_n \to f$ uniformly on compact subsets of $\mathbb{R}$, $f_n' \to f'$ uniformly on ...
8
votes
4answers
280 views

The sum $\frac{1}{\ln(2)}+\frac{1}{\ln(3)}+\frac{1}{\ln(4)}+…$ is divergent. Find the regularized evaluation

By considering the integral Zeta function $$F(s)=s+\frac{1}{2^s\ln(2)}+\frac{1}{3^s\ln(3)}+\frac{1}{4^s\ln(4)}+...$$ Evaluate $$\frac{1}{\ln(2)}+\frac{1}{\ln(3)}+\frac{1}{\ln(4)}+...$$ EDIT: ...
0
votes
0answers
26 views

Regularization for unordered vectors

Let suppose we have two vectors u, v $\in \mathbb{R}^n$ and we want a function that returns $0$ if the ordering of the elements of both vectors are the same or a positive number otherwise, where the ...