Regularization, in mathematics and statistics and particularly in the fields of machine learning and inverse problems, refers to a process of introducing additional information in order to solve an ill-posed problem or to prevent overfitting. (Def: ...

learn more… | top users | synonyms

0
votes
0answers
25 views

Image processing, optimization via regularization - efficient strategy

i would like to solve the following system: $J(x) = |Ax-b|_2^2+\gamma|\nabla x|_2^2$ subject to: $x \geq 0, \sum_i x_i = 1$ The underlying problem is to derive the PSF from a sharp and blurry ...
1
vote
1answer
80 views

How does zeta of zero equal to negative one half rather than to infinity?

$$\zeta(0)=(1/1^0)+(1/2^0)+(1/3^0)+(1/4^0)+(1/5^0)...$$ Am I right? Anything raised to the power of zero is one. One to the power of zero is one. One divided by one is one. $$1/1^0=1$$ Am I right? ...
2
votes
0answers
49 views

Regularizing the sum of all factorials

Consider the series $$\sum_{n=0}^\infty n! = 0! + 1! + 2! + 3! + 4! + \ldots = 1 + 1 + 2 + 6 + 24 + \ldots$$ This series clearly diverges. Now, given that the Gamma function is defined by $$n! = ...
1
vote
1answer
25 views

If $f \in L^p(\Omega)$, then $(\rho_n *f) \to f $ in $L^p(\Omega)$, for a sequence $(\rho_n)$ of mollifiers.

I am reading Functional Analysis, Sobolev Spaces and Partial Differential Equations, by Haim Brezis. There it is shown that: If we have $f \in L^p(\mathbb{R}^n)$, then $(\rho_n *f) \to f $ in ...
0
votes
0answers
4 views

Regularization as an alternating objective or combined objective

I have a "primary" task loss function $L=L1$ which I want to minimize. Adding a regularization term via $L=L1+\lambda L2$ can be thought of as "forcing" the optimal solution to be meaningful for a ...
1
vote
0answers
11 views

regularized least squares (L1 norm)

My objective function that is to be minimized is as follows: $$\|y-Ax\|_2^2 + \alpha\|Lx\|_1$$ where $L$ is the gradient operator. Now this problem seems convex because the first term is quadratic ...
1
vote
1answer
12 views

How to derive the hard thresholding estimator?

The minimization problem is $$ \min_{\mu\in R^p} \sum_{i=1}^p (y_i-\mu_i)^2 + \lambda^2\sum_{i=1}^p \mathbb{1}(\mu_i\neq0) $$ for $y\in R^p$. This is also known as $l_0$ regularization. The solution ...
1
vote
0answers
7 views

LASSO variable selection for proportinal hazards

Can anybody recommend software that I could port into SAS to do LASSO (or related) variable selection on a proportional hazards model?
0
votes
1answer
20 views

Reverse economization of Chebyshev series

Suppose I have some function which is represented as converging series of Chebyshev polynomials of first kind in $[-1;1]$: $$ f(x)=\sum\limits_{n=1}^\infty a_n T_{2n}(x) $$ I need to transform this ...
0
votes
0answers
18 views

Upper estimate between an original function and its sup-convolution under a limitation

My setting maybe look rather special but I'm glad if you give some answers. Let $f:[0,1]\to\mathbb{R}$ be a bounded, upper semicontinuous function and $f^{\varepsilon}:[0,1]\to\mathbb{R}$ be $f$'s ...
0
votes
0answers
16 views

Want to factorize one matrix into three, with L1 regularization, which optimization algorithm to choose?

I need to factorize one matrix $R$ into three component: $ R = P^TAQ $, in which I want to apply L1 regularization on $A$ to encourage sparsity, and apply L2 regularization on $P$ and $Q$ to prevent ...
0
votes
1answer
29 views

Regularize the sum: $\sum_{n=0}^\infty (-1)^{n+1}({2+4n})$

$$\sum_{n=0}^\infty (-1)^{n+1}({2+4n})=2-6+10-14\cdots$$ is a clearly divergent sum, which I am trying to regularize. I need to figure out this sum because I have been trying to solve the sum integral ...
1
vote
1answer
26 views

l1 regularized minimization with equality constraint in ADMM

In section 6.3 of this note there is a method for minimizing a loss function with l1 regularization. i.e. minimize $l(\bf{x})+\lambda||x||_1$ How can I add the equality constraint $\sum\limits_{i} ...
0
votes
2answers
18 views

How to prove non-regularity of a language from the non-regularity of another language?

How can I prove that $L_1=\{a^nb^m\mid n\ne m\}$ is not regular based on the fact that the language $L_2=\{a^nb^n\mid n\in\Bbb N\}$ is not regular? Thank you
1
vote
0answers
12 views

Why divide regularization factor by size of dataset?

Suppose I'm trying to minimize a cost function: $$ J(\theta) = \frac {1} {2m} \sum _{i = 1}^ m (h_\theta (x^{(i)}) - y^{(i)})^2 $$ Adding regularization, as seen here, we get: $$ J(\theta) = \frac ...
2
votes
1answer
32 views

Convex Optimization - nuclear norm regularisation of symmetric matrix

I have a problem of the form $\min_{X\in \mathbb{R}^{n \times n}} g(X) - \lambda ||X||_{*}$ where $g$ is convex and differentiable. I would like to use proximal gradient descent to solve this. How ...
0
votes
0answers
70 views

Solve the problem $Ax = 0$ when $A$ has full rank.

Generally, the answers $x$ of this least square problem $$Ax = 0$$ where $A = []_{m\times n}$ and $x = []_{n\times 1}$ are in the null space of $A$. I know that people usually use the right-most ...
9
votes
1answer
283 views

Is $\prod_{n=1}^\infty P_{2n-1}$ regularizable?

Assume that $P_n$ denotes the $n$'th prime for this entire question. Inspriation: I was dumbfounded by the fact that: $$\hat\prod_\limits{n=1}^\infty P_{n}=4\pi^2$$ After further investigation, I ...
1
vote
0answers
17 views

Extension of $C^1$ function (regularity of mollification on boundary)

In my study I have faced the problem showing the property below: regularity of mollification on boundary. (However, I don't know whether this is true or not although I hope.) Let $f\in ...
5
votes
0answers
161 views

Prove the estimator $\hat{B}$ of ridge regression = mean of the posterior distribution under a Gaussian prior

I want to prove that the estimator of ridge regression is the mean of the posterior distribution under Gaussian prior. $$y \sim N(X\beta,\sigma^2I),\quad \text{prior }\beta \sim N(0,\gamma^2 I).$$ ...
0
votes
0answers
18 views

Penalty Term on Negative Matrix Elements for Gradient Descent Objective Function?

In fact, I'm doing a gradient ascent to estimate the elements of a matrix $X=f(A)$. Nevertheless, I've written "descent" into the title, because I think that this is the common term. Motivational ...
3
votes
4answers
246 views

regularization of sum $n \ln(n)$

I was testing out a few summation using my previous descriped methodes when i found an error in my reasoning. I'm really hoping someone could help me out. The function which i was evaluating was ...
1
vote
0answers
159 views

Zeta regulated product, solving without the zeta function.

Earlier i've asked about how to calculate divergent products, i got some directions which made me curious. Now i'm wondering is this correctly done. Divergent products. The most commen divergent ...
7
votes
0answers
57 views

Does such divergent integral assume the same values for any regularization?

Consider the integral: $$\int_0^\infty\sin(x)dx.\tag1$$ It's clearly divergent, but if we regularize it as $$\int_0^\infty\sin(x)e^{-x/a}dx=\frac{a^2}{a^2+1},\tag2$$ we can take the limit of ...
1
vote
0answers
27 views

Estimating Markov transition matrix for regularization

Suppose that I have a sequence of discrete distributions: $$ p_j = (p_{1j},...,p_{Cj}), \: j=1...D,\\ p_{ij}>0 \:\: \forall i,j,\: \sum_{k=1}^Cp_{kj}=1\:\:\forall j. $$ I suppose that these ...
1
vote
0answers
29 views

About consistency in an inverse problem formulation

I'm a beginner with inverse problems and I was reading about regularization techniques. Consider the problem: $$d=Kf_{\text{true}}$$ $d$ is a data vector, $K$ is an linear operador, $d=\hat{d}+\eta$ ...
0
votes
1answer
41 views

Relation between Regularization and correlation

I was going through Chapter 3 (page 63 bottom) of Elements of Statistical Learning. While explaining regularization in ridge regression authors make the following statements. "When there are many ...
1
vote
1answer
57 views

Solve the following matrix equation $X'X=A$

I have square matrices $X$,$A$ and $X'X-A=0$. $A$ is given and is positive definite and I need to get matrix $X$. I know $X$ is not unique since $TX$ such that $T'T=I$ will satisfy. My problem is ...
1
vote
1answer
61 views

How do I set a lower bound to the solution's norm in a QP problem

I know that LASSO-regularization can be used to scale into an $L_1$ upper bound for a solution. But what if I want the norm to be within a specific range $[a,b]$? ie. I also want to set a lower bound? ...
2
votes
1answer
384 views

Solving L1 regularized Joint Least Squares and Logistic Regression

My objective function that is to be minimized is as follows: $f = -\sum_{n=1}^{N}log~p(y_{n}^{a}|x_{n},w) + \sum_{n=1}^{N}(y_{n}^{b}-w^{T}x_{n})^{2} +\lambda\|w\|_1$ The first term models the ...
2
votes
1answer
114 views

When do regularization methods for divergent series disagree?

Sometimes, it is possible to take a divergent series (in the sense of its sequence of partial sums failing to converge) and "regularize" it using one of a variety of methods to assign it a meaningful ...
4
votes
1answer
416 views

Tikhonov regularization vs truncated SVD

To find $\mathbf{x}$ such that $$A\mathbf{x}=\mathbf{b}$$ we can use least squares when the problem is not well posed. Further, we can use Tikhonov regularization when $A$ is ill-conditioned. In ...
1
vote
0answers
99 views

sum of divisors function $\sum \tau(n) = \frac{1}{4}$

These notes on multiplicative number theory mention the convolution $ 1 \ast 1 = \tau$ (where $\tau$ is the divisor function not Ramanujan tau function. Therefore $$ \bigg(\sum \frac{1}{n^s} ...
1
vote
1answer
92 views

Zeta and Gamma function regularization with $\omega=1/0$

I have recently read about zeta function regularization, a way of ascribing values to functions having simple poles in a point and to divergent series. The values obtained are the same as those ...
6
votes
2answers
286 views

Is there a metric in which 1+2+3+4+… converges to -1/12?

It is well known that the sum $1+2+3+4+\ldots$, which tends to infinity in the regular sense, can be assigned the value $-\frac{1}{12}$ by different means, e.g., zeta regularization or different ...
0
votes
1answer
50 views

Optimization Problem (Linear Algebra)

I am not trying to cheat or anything, so any reference to online literature or MOOCs, that teach this stuff, will be highly appreciated. The problem is to prove that the following optimization ...
2
votes
0answers
41 views

Can this divergent integral transform be regularized?

The integral $$\int_0^{\infty} e^u \ K_{i t}(u) du$$ is the adjoint Kontorovich-Lebedev transform of the increasing exponential function, but unfortunately this integral is divergent because $$e^u \ ...
0
votes
1answer
24 views

zeta regularization separation of series

in the sense of infinite series and for an integer 'a' is then correct that $$ \sum_{n=1}^{\infty}n^{k} = \sum_{n=1}^{a}n^{k}+ \sum_{n=a+1}^{\infty}n^{k} $$ opther that works only when ·$ re(k) > ...
2
votes
0answers
133 views

Fourier transform of an exponentially singular radial function

I am trying to compute the 3D Fourier transform of a spherically symmetric function of the form $$f(r) = e^{\frac{1}{r} e^{-r}} - 1\, ,$$ which entails the integral $$\begin{aligned}F(k) =& \int ...
0
votes
1answer
15 views

Support of vector $w$ in graph sparsity

I'm reading about graph sparsity and I have one problem in a paper I'm reading I don't understand, maybe someone can clarify: Graph Sparsity: In graph sparsity, we have a directed acyclic graph ...
0
votes
1answer
39 views

Differentiable L-1 Regularization

In machine learning we are often faced with optimization problems where we want to minimize some energy function using L1 regularization over some of the parameters, e.g.: $$ E(a,w) = [\text{sum of ...
0
votes
1answer
62 views

Regularization of underdetermined system to favour low frequency solutions?

Consider the ill-posed system $$ \mathbf A \mathbf x= \mathbf b.$$ One method to regularize the solution is the Tikhonov method which effectively minimizes $ ||\mathbf A \mathbf x - \mathbf b ||^2 + ...
1
vote
1answer
37 views

Integral in regularization involving exp of a parameter in the denominator

Can someone please point me a way to compute $$\int_0^\infty\frac1{s+t}\exp\left(-\alpha t+\frac{t^2\beta}{s+t}\right)dt$$ ? How about the following one? $$\int_0^\infty ds\int_0^\infty ...
4
votes
1answer
111 views

Is it possible to sum the divergent series with prime coefficients?

This is a follow-up of this question. It is known that the divergent series $$ P := \sum_{n=1}^\infty p_n \qquad \text{where } p_n \text{ is the $n$th prime} $$ cannot be summed by means of (prime) ...
0
votes
1answer
176 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
7
votes
0answers
224 views

Connection between integral expression and the factorial of infinity

Does the fact that $$\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}x^2\right)\mathrm{d}x=\sqrt{2\pi}$$ Have something to do with the fact that the regularized factorial of infinity is also ...
1
vote
0answers
55 views

When does l1 regularisation give a sparse solution?

I was maximising a likelihood function, which is convex. I know that the system has a K-sparse solution. I wanted to know the conditions (or some sufficient conditions) on the likelihood function ...
1
vote
0answers
107 views

Regularity of semilinear heat equation

I'm facing following regularity issue and i wonder if anyone of you guys is able to help me. I'd like to show that the solution of a semilinear heat equation is classical, i.e. $C^2$ in space and ...
1
vote
1answer
116 views

Assigning values to divergent integrals

I'm interested in the (obviously divergent) integral $$ \int_{-\infty}^\infty dx e^{-x f}\ ,$$ where $f$ is real. Is there any way to meaningfully assign a value to this integral? I was thinking of ...
1
vote
0answers
235 views

Implementation of Total Variation Regularization Algorithm (Lagged Diffusivity Algorithm)

I am trying to compute the derivative of an experimentally-measured quantity as a function of time. The data are fairly noisy, which causes problems. For instance, using finite differences (central ...