# Questions tagged [machine-learning]

How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

3,357 questions
Filter by
Sorted by
Tagged with
41 views

### How to estimate the inverse of a non-invertible matrix?

So I'm working on a machine learning problem where my solution requires taking the inverse of a matrix at some point. The problem is that this matrix is sometimes non-invertible. In theory the the ...
• 11
30 views

### Distribution of two combined ML models

Due to the complexity of the problem, the problem was divided into two models: a stationary model and a model that corrects the stationary model for temporal effects, i.e. $X = X_{stat} + X_{time}$ ...
96 views

### EM algorithm for estimating worker ability [closed]

I cannot understand how to get results in M-step. This formulation is from https://papers.nips.cc/paper_files/paper/2012/file/cd00692c3bfe59267d5ecfac5310286c-Paper.pdf
29 views

### Formulating a solution ansatz for the 1D heat equation in polar coordinates to learn the PDE in a PINN setting

Hello Math Stack Exchange Community, I am working on solving a partial differential equation (PDE) with a neural network in a PINN-like fashion, and I am seeking advice on identifying an appropriate ...
68 views

### What is the collection of functions that a given finite neural network can approximate with ease?

To my understanding, one version of the universal approximation theorem runs as follows: Let $\Phi$ be the family of (trained) feedforward neural networks of bounded width, arbitrary depth, and mild ...
• 259
23 views

### Question about likelhood function of discriminative models

Im a little confused with the likelihood function. For discriminative models, we have a hypothesis function $h_{\theta}(x) = p(y \mid x ; \theta)$. Using the principles of maximim likelihood we want ...
• 139
26 views

### Calculating functional derivative for a Physics-Informed Neural Network (PINN) using Automatic Differentiation

I'm working with a Physics-Informed Neural Network (PINN) to approximate the solution of a 1D Poisson equation: $\frac{d^2u}{dx^2} = f$ Here, I have an MLP with weight parameters $\theta$ that takes a ...
1 vote
19 views

### How many vectors can be placed in $n$ dimensions given max cosine similarity? [duplicate]

In machine learning we usually use the concept of cosine similarity to compare things. Similar things should have embeddings with high cosine similarity and different things should have embeddings ...
48 views

### Self-Organizing maps: why input vectors (x) are dependent on steps (t)?

Based on the paper Essentials of the Self-Organizing maps, I rephrase paragraph 4.1. ->The original, stepwise recursive SOM algorithm: In the mathematical framework $\{\mathbf{x}(t)\}$ represents ...
• 51
1 vote
42 views

### FGSM for logistic regression

In arXiv:1412.6572 (https://arxiv.org/pdf/1412.6572, a seminal article), it is stated that $$\mathrm{sgn}(\nabla_{\mathbf{x}} L(\mathbf{x},y,\mathbf{w})) = -\mathrm{sgn}(\mathbf{w})$$ for the softplus ...
• 47
22 views

### Sample complexity bounds of $L_S(h)$

Fix $\mathscr{H} \subset \mathscr{Y}^\mathscr{X}$ and a loss $\ell : \hat{Y} \times Y \to [0,1]$. Fix $S \in (\mathscr{X} \times \mathscr{Y})^{2m}$. Assume for now that $S$ is not random. Suppose we ...
• 41
1 vote
48 views

### sorting functions by amount of conditions for a random dataset to be described using it?

Given a finite dataset like 1, 2, 3, 4 You could find infinite functions, for simplicity I found 2: Add 1 for the next data point, so the sequence continues as 5, 6, etc. 2.Cycle through 1, 2, 3, 4, ...
31 views

### Strong convexity and Lipschitz-continuous gradients, how restrictive are these assumptions in practice?

I am reading a paper on stochastic gradient descent and different varieties of it. For all the convergence proofs the author assumes strong convexity and Lipschitz-continuous gradients for the ...
• 453
14 views

### Information coefficient as loss function of XGBoost

$$IC = \frac{\frac{1}{n}\hat{y}^Ty-\mathrm{E}\left[ \hat{y} \right] \mathrm{E}\left[ y \right]}{\sigma \left[ \hat{y} \right] \sigma \left[ y \right]}$$ XGBoost requires a gradient and a Hessian of ...
1 vote
71 views

### Relation between values of $ξ_i$ and $\alpha_i$ in SVM?

I have a question in about a property of support vectors of SVM which is stated in subsection "12.2.1 Computing the Support Vector Classifier" of "The Elements of Statistical Learning&...
11 views

### Paired bootstrap test p-value formula in binary classification

Background For a binary classification task, let $M(A, Z)$ denote an evaluation metric, such as accuracy, for classifier $A$ and test examples $Z.$ Then, let $$\delta(Z) = M(A, Z) - M(B, Z)$$ denote ...
• 802
39 views

### least squares minimum test error solution

assume we want to learn a model $y=x^T \beta + \varepsilon$ where $\beta \in \mathbb{R}^d$ is constant $x \in \mathbb{R}^d$ is the input vector with Gaussian distribution $\mathcal{N}(0,\Sigma_x)$ ...
24 views

### Does clustering actually reduce the number of rows in a dataset? [migrated]

I am reading the book "grokking Machine Learning" by Luis G. Serrano and came across the following sentence: "It seems that clustering and dimensionality reduction are nothing like each ...
• 8,204
13 views

### Defining Unsupervised Learning Problem

I have read a paper that says Unsupervised learning concerns modeling and understanding the structure of complex data ... Of course, the distribution over such complex data as images and sounds ...
• 31
23 views

### Analyzing a deep learning model by constructing a Matrix from input and output data

I am currently completing my bachelor’s thesis, and recently my supervisor suggested that I could strengthen my arguments by explaining why deep learning models perform so well in my case. To give you ...
• 109
1 vote
98 views

### Why does learning theory study generalization bounds?

Disclaimer: I know that mathematics needs no external motivation to be developed, and that such view is (in the long term) helpful even for applications. Nonetheless, I believe it is crucial for ...
18 views

### minimization involved $l_2$ norm

I am trying to find the minimum of the following problem $$\frac{\theta}{2}\lVert\beta-x\rVert_2^2+\lambda\lVert x\rVert_2-\frac{1}{2\tau}\lVert x\rVert_2^2+\alpha^Tx$$ by taking the derivative with ...
• 3,633
1 vote
29 views

### Understanding of KL divergence

I am learning machine learning and encountered KL divergence: $$\int p(x) \log\left(\frac{p(x)}{q(x)}\right) \, \text{d}x$$ I understand that this measure calculates the difference between two ...
1 vote
42 views

### Solve the Soft SVM Dual Problem with L1 Regularization

I'm considering a support vector regression model with a prediction $$\hat{y}(\mathbf{x}_\star)=\boldsymbol{\theta}^{\top} \boldsymbol{\phi}(\mathbf{x}_\star)$$ where $\boldsymbol{\theta}$ are the ...
• 239
21 views

### Would like to validate whether the AUC equation is correct or not

I found a paper "Chapi, Kamran, et al. "A novel hybrid artificial intelligence approach for flood susceptibility assessment." Environmental modelling & software 95 (2017): 229-245&...
• 95
1 vote
24 views

• 702
1 vote
173 views

### Explain the proof of Kolmogorov Arnold representation theorem

Can someone explain the outline of proof strategy of Kolmogorov Arnold representation theorem? Any proof of any variant (eg. George Lorentz's variant) would suffice. I would be grateful if you could ...
• 451