Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [machine-learning]

How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

0
votes
0answers
23 views

Why is normalisation of a Laplacian matrix defined as $L - I$?

I have a piece of code that computes the Laplacian matrix using $I - D^{-\frac{1}{2}} A D^{-\frac{1}{2}}$, that is, it computes the symmetric normalised Laplacian. This Laplacian is not defined when $...
0
votes
0answers
6 views

Which classical classification model is suitable when prior probabilities can change?

I have an image (quality) inspection application that I want to use a classical method for, as opposed to deep learning. This is for the ease of interpretation. I thought about using Naive Bayes ...
1
vote
0answers
10 views

Non-markovian random walks and their applications in machine learning

I'm searching applications of random walks in machine learning. In particular, applications of random walks with long memory. An example of this kind of processes is the so called ELEPHANT RANDOM WALK....
0
votes
1answer
24 views

I ran an ANN model and got an extremely low R2 but a pretty good MSE, what does this mean?

I ran an artificial neuron network on data with about 2,000 rows and 3 features. I got a R2 of .06 which is really low, but a good MSE of .41. Why are these performance evaluators of this model ...
0
votes
0answers
18 views

Inference can be the goal of an unsupervised learning method?

I am new to machine learning, and I am reading a pair of machine learning books. Those references talk about 2 different learning approaches: Prediction and inference, I understand the difference ...
2
votes
0answers
19 views

Directional Curvature

What is Directional Curvature and how can I achieve it for any function? A common approach with an example would be much appreciated. (Reference: I am reading "The Non-convex Geometry of Low-rank ...
0
votes
0answers
24 views

Proof for normal posterior distribution for a data for linear regression.

Let $D=(x_1,x_2,....X_N)$ be the data and $x_i$ is D-dimensional data. For linear regression the likelihood is as follows $$p(y|X,w, u, \sigma^2) = N(y|u + Xw, \sigma^2 I_N) \propto exp(-\frac{1}{2\...
0
votes
0answers
22 views

Help me identify this decay function

I'm interested in identifying a formula/equation that i used as a decay function in order to learn more about it - the equation is formatted in Python code as that is where it was originally expressed....
0
votes
0answers
10 views

Tree Pruning: SSE

Can someone show me an instance of a data set where a no split causes a reduction of SSE, however can be modeled by a tree with 3 splits? Basically, an example that shows that it's sometimes needed to ...
0
votes
0answers
20 views

AdaBoost what is the hypothesis

I read the article about AdaBoost on Wikipedia an stumbled across a problem. What is meant by the word hypothesis. In this case I mean that section where they refer to the hypothesis h(x) of a "weak ...
-1
votes
0answers
29 views

Gaussian mixture model: sum of 2 random variables [closed]

We have data for $X$ (integers). Model $X = Y+Z$. Where $Y\sim N(\mu_1,\sigma_1)$ and $Z\sim N(\mu_2,\sigma_2)$ are latent variables. How to use a Gaussian mixture model or EM algorithm to estimate:...
1
vote
0answers
18 views

Suggestion for choosing (building) optimization function

I would like to build a supervised learning model M satisfying the following conditions: Training data $\{X, Y\}$, where $x \in R^m$ and $y \in R^n$ Assume: $M(x) = p$, then: $0 < p[k] <= y[k]$,...
1
vote
0answers
24 views

Proof in Variational Gaussian Approximation

Here is the result from the appendix of paper The Variational Gaussian Approximation Revisited (Manfred Opper and Cedric Archambeau) This result has been used in the popular paper Practical ...
-1
votes
1answer
108 views

How do I prove that the SBAF activation function is not a probability density function?

The SBAF activation function is as follows - Note : 0<=x<=1 $$ f(x) = \frac{1}{1+ kx^a(1-x)^{1-a}} $$ Where k and a are constants. I know we have to show that integral $\int_{-\infty}^{\infty} f(...
1
vote
1answer
44 views

Derive $ \frac{1}{1 + exp(-(\beta_0 + \beta_1x))} $ from conditional and total probabilities

Related to: Show posterior probability takes the form of the logistic function I basically want to derive the sigmoid function from conditional and total probabilities. In other words, I want to ...
2
votes
1answer
16 views

How is the continuous input probabalistic generative model derived from single class model?

So for the single valued model we have: $p(C_1|\textbf{x}) = \frac{p(\textbf{x}|C_1)p(C_1)}{p(\textbf{x}|C_1)p(C_1)+p(\textbf{x}|C_2)p(C_2)}$ If we rearrange the terms, we can write this as a ...
2
votes
0answers
20 views

What do we mean by spectral multipliers?

I am reading a research paper in the context of geometric deep learning. In that paper, the following equation is defined $$f_l^{\text{out}} = \xi \left( \sum_{l'=1}^p \Phi_k \hat{G}_{l, l'} \Phi_k^T ...
1
vote
0answers
33 views

Finding the coordinates of points from Geometry matrix

I know this question has been asked before, so please do not asign it as a duplicated question. This link is similar question but it is not useful for my case. I tried to understand the EAST Module....
1
vote
1answer
35 views

Doubt about how exactly was calculated this gradient descent cost function using Octave\MatLab. How is it exactly working?

I am following a machine learning course on Coursera and I am doing the following exercise using Octave (MatLab should be the same). The excercise is related to the calculation of the cost function ...
3
votes
1answer
29 views

Compute component probabilities in EM-algorithm with log densities?

I coded up an implementation of the EM-algorithm for Gaussian mixtures. In the E-step I compute, for each row in the data matrix, the probability $p_i$ that it has been drawn from the component $i \in ...
1
vote
1answer
19 views

Gaussian complexity bound

I am reading Foundations of Machine Learning (1st edition). It seems that most generalization bounds in the literature are based on Rademacher complexity, rather than Gaussian complexity. So, I was ...
0
votes
0answers
15 views

looking for a simple example in machine learning with step-by-step procedure

I am looking for a simple example in the area of machine learning as well as a step-by-step procedure. For example, if I have 3 measurement devices and each one reports 5 data hourly how can I ...
1
vote
0answers
46 views

Gradient Descent vs Lagrange Multipliers

I'm bit confused between Gradient descent and convex optimization using Lagrange Multipliers. I know that we use Lagrange multipliers when we have an optimization problem with one or more constraints. ...
1
vote
1answer
44 views

Policy gradient reinforcement learning for continuous state and action space

I am a novice in the field of machine learning, I have a moderate level understanding of linear/non-linear regression, support vector machines, neural networks, and q-learning (for discrete finite ...
1
vote
0answers
37 views

how to show the K-nearest-neighbor density model is an improper distribution: Bishops 2.61

In Bishop's pattern recognition and machine learning, KNN is defined from a starting distribution: $$p(x) = \frac{K}{NV}$$ where K is the number of observed points in a region of measure V, out of a ...
2
votes
0answers
26 views

Is it appropriate to use clustering to partition the dependent variable into separate datasets for a home price prediction model?

I'm struggling to decide how to deal with a heteroskedasticity problem in a home price prediction model I'm developing. The training set residuals are normally distributed around zero, but they have ...
1
vote
1answer
39 views

Clarification regarding Parameter Estimation (Andriy Burkov's book)

So I recently decided to read Andriy Burkov's "The 100-Page Machine Learning Book" and got confused in Chapter Two (Page 11) where he discusses Parameter Estimation techniques. A picture of the ...
0
votes
0answers
11 views

Adaboosting with linear kernels

If I wish to apply Adaboosting to a SVMs. Is it a good idea to apply with linear kernels? Or should do it with polynomial?
1
vote
1answer
35 views

Kernels and finite maps

Let $\bf{x}, \bf{z} \in \mathbb{R}^n$. If the Gaussian kernel is defined by: $$K(\bf{z}, \bf{x}) = \exp\left( - \frac{\|\bf{z} - \bf{x}\|_2^2}{\sigma^2}\right) $$ I'd like to know if there is a ...
1
vote
0answers
26 views

softmax cross entropy derivative

I am working with Logistics Regression with multiclass classfication(softmax with entropy): $$L(w)=-\sum_{n}\sum_{k}y_{nk}log(\frac{e^{w_{k}^Tx_{n}}}{\sum_{i}e^{w_{i}^Tx_{n}}})$$ differentiation: $$\...
0
votes
0answers
36 views

How do we defined our prediction models to be generalized well enough to be applied to unseen dataset?

How do we define our prediction models to be generalized well enough to be applied to an unseen dataset? And if there is an outlier in the data do we need to keep it or remove it? Have to justify the ...
1
vote
1answer
27 views

Orthogonal complement in infinite-dimensional space.

In A Generalized Representer Theorem page 6, the author wriites: ...Given $x_1,...,x_m$, any $f\in\mathcal{F}$ can be decomposed into a part that lives in the span of the φ($x_i$) and a part ...
0
votes
1answer
19 views

What is the VC dimension of a d-dimensional quadratic function?

I have an indicator function $I(M, x, y) = sign[(M(x - u))^{T} (M(x - u)) - y]$. $M$ is an invertible matrix of size $d \times d$. $x, u$ are vectors of size $d$. $u$ is a parameter for the ...
1
vote
1answer
41 views

integration of probability distribution

How do you simplify or find the closed form of the following probability distribution? $$ p(y|x,X,Y) = \int^{10}_{-10} \int^{10}_{-10} p(y|x,w_1,w_2)p(w_1)p(w_2)dw_1dw_2 $$ Where: $$ p(w_1) = ...
2
votes
0answers
94 views

Compare euclidean distances from different dimensions

EDIT: I'd like to reformulate my primary question. I have a set of points in $\mathbb{R}^D$ and I reduce the dimensions of the points to some $\mathbb{R}^L$. I do this with multiple configurations ...
0
votes
0answers
36 views

Expected value of composite function

If we have two random functions f and g with Bernoulli distributions of p and ...
1
vote
0answers
31 views

Variable transformation for training a machine learning model

Suppose you have a train set $\mathbf{T}$ and you want to train some Machine Learning models. Each row of $\mathbf{T}$ consists in a set(vector) of attributes or variables $\mathbf{x} = (x_1, x_2...)$ ...
1
vote
0answers
8 views

Regression task on desirable subsets with limited supervision

Suppose I have a set of n elements. I want to have a model for how "desirable" a subset of those elements is when evaluated by a person. The training/testing data is a number of given subsets, each ...
2
votes
0answers
53 views

No Free Lunch in statistics

I was wondering if the No Free Lunch (NFL) theorem applies to even the estimation problem. Suppose there are $N$ points in the input. We are trying to estimate the mean value say weights associated ...
1
vote
0answers
27 views

regularized least squares Generalized Tikhonov Regularization on real dataset

I am using regularized least squares more specifically Generalized Tikhonov Regularization on real dataset where rows << cols: $$𝑥=(A^TA+\lambda I)^{-1}(A^Tb)$$ I am implementing it using C ...
0
votes
0answers
8 views

RKHS for matrix valued input.

The question is related to the link : https://en.wikipedia.org/wiki/Reproducing_kernel_Hilbert_space As I understand the Mercer theorem we can get RKHS. This lead to the kernel trick as mentioned in ...
1
vote
0answers
11 views

Separating vectors and minimization of norm

I am solving the following exercise from the book Understanding Machine Learning (ex 14.3), the problem is I am not very strong in geometry: Let $S=((x_1,y_1),\ldots,(x_m,y_m)) :\, (x_i,y_i) \in \...
0
votes
1answer
33 views

What is weighted and unweighted linear regression in machine learning?

I'm taking Stanford's CS229 ML course and while studying about "parametric algorithms", Prof. Andrew Ng says that this class of algorithms has a fix number of parameters (parameters are also called as ...
0
votes
0answers
10 views

Problem on finding the set of biases and weights in a specific neural network

I have a doubt regarding an exercise here. Suppose that we have a neural network that tries to map a $28\times 28$ image of a digit to what digit it actually represents. So we have a neural network ...
0
votes
0answers
24 views

Minimize the number of y<0

I have observations $x_1,x_2,...,x_N$ and a fixed function $y=f(x;c)$, where $c$ is a parameter. Now how can I find the optimal $c$ to minimize the total number of $y<0$, i.e., min$\sum_{i=1}^N ...
2
votes
1answer
134 views

Understanding the effect of $C$ in soft margin SVMs

I'm learning soft margin support vector machines form this book. It's written that in soft margin SVMs, we allow minor errors in classifications to classify noisy/non-linear dataset or the dataset ...
0
votes
0answers
19 views

error covariance of MMSE estimator relation to other error covariance estimators

I'm trying to prove the following: let $ \Lambda_{e}$ be the error covariance of an estimator $\,\hat{\theta}(y)$ of $\,\theta$ based on $\,y$. I want to show that the error covariance of MMSE ...
1
vote
0answers
18 views

EM algorithm - E-step notation

I think I understand the gits of Expectation-Maximization algorithm and its altering nature, but I am puzzled by the notation. Lets see the following examples: in Stanford notes , the E-step is ...
-1
votes
0answers
6 views

Justify that empirical risk satisfies inequality

We have empirical risk: $$f_n \in argmin_{f\in \mathscr{F}}\frac{1}{n}\sum_{i=1}^{n}\mathbb{1}\{Y_i \neq f(X_i)\}$$ Prove that it satisfies: $$P(Y \neq f_n(X)) - inf_{f \in \mathscr{F}}P(Y \neq f(X)) ...
0
votes
0answers
10 views

Constructing signal vectors for machine learning

I have daily stock returns related to sectors. At the end of each month I want to construct a vector of signals using the past data with different methods over different moving windows like EWMA over ...