Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Questions tagged [machine-learning]

How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

0
votes
0answers
8 views

Is there a computationally easier method to determine if a dataset is predictive other than trial and error?

So as many of you are aware, Nvidia has a platform for doing GPU-accelerated deep learning. Usually a machine learning developer proceeds by engineering a dataset and then training a neural net, for ...
0
votes
0answers
6 views

How to costumize margin (epsilon) range in SVM regression

I am trying to implement a SVM regression in order to predict my target variable. I want to customize the margin for my particular task. That means I want to have an epsilon boundary with a different ...
-1
votes
1answer
34 views

transpose(u) * Matrix * u

I've seen this a couple of times during a machine learning lecture, for example in context of LDA, when looking at the Fisher Criterion. It can be expressed in two ways: $$J(w) = \frac{(m_1 - m_2)^2}{...
0
votes
0answers
15 views

Best parameters for SVM

I have studied about SVM with this pdf. Then I came across the problem in page 13. It is here \begin{align*} b^* = - \frac{ \max_{i:y^{(i)}=-1} {w^*}^T x^{(i)} + \min_{i:y^{(i)}=1} {w^*}^T x^{(i)} }{...
2
votes
0answers
25 views

Is there an equivalence between subgradient and stochastic gradient?

Consider the optimization problem $$\min_x \; f(x) := \sum_{i=1}^m f_i(x).$$ A subgradient method at each iteration takes a subgradeint descent step $$ x^+ = x - \alpha g, \quad g\in \partial f(x)...
-2
votes
0answers
27 views

Which of the following is not Linear Regression [on hold]

Which of the following is NOT a linear regression model: $y = β_1x_1 + β_2 x_2 x_3$ $y = β_1 x_1 + β_2 \log(x_2)\cdot(\exp(x_3))$ $y = β_1 + β_2(x_1^{-β_3})$ $y = β_1x_1 + β_2 \sin(x_2) + β_3 \cos(...
0
votes
1answer
7 views

Applying a Stochastic Computation Graph + DiCE operator

I am following this paper and I cannot workout the example in 3.3. https://arxiv.org/abs/1802.05098 In the paper, they propose the $DICE$ operator and before they give the following example in 3.3: ...
-3
votes
0answers
16 views

Why is my regularized gradient descent algorithm for linear regression diverging?

I am applying a regularized gradient descent algorithm on a dataset for linear regression. Since there are too many features, I am programming using the matrix notations. Following expression is being ...
0
votes
0answers
29 views

What programming language is best suited for Machine Learning? [on hold]

There are multiple languages that professors use in their Machine Learning/Artificial Intelligence courses. Currently, I am alternating between R, Python, and Java and only because my professors say ...
1
vote
2answers
77 views

Can a matrix's rank be greater than one of its dimensions?

I am reading a paper on deep learning. Kawaguchi et al, Generalization in Deep Learning If $ϕ$ is a matrix of dimensions $m \times n$. Is the assumption valid that the rank of $ϕ$ can be greater than ...
0
votes
0answers
25 views

How to formally define what a reading comprehension question answering problem is?

I'm trying to formally define what Intelligent Agents with Reading Comprehension Question Answering agents are in mathematical terms for a dissertation. To my mind we can say we have on the one hand ...
0
votes
0answers
24 views

Bayes theorem: Prior depending on observation

In the context of autoencoders, it is useful to have a preference on the parameters depending on a latent result that the autoencoder calculates from observations. A simplified case might be a ...
1
vote
0answers
28 views

Solve linear equations system with only positive coefficients in solution

Can you advice the most efficient way to solve a system of linear equations where the solution vector has only positive coordinates? $$A \times b = c$$ $$\forall i,\, b_i > 0$$ Of course, in most ...
2
votes
0answers
113 views

Linear regression with feature representation confusion - is design matrix column space the feature space?

I am trying to visualise the geometry of linear regression with feature representation. I have a regression problem with $n$ data pairs $\mathcal{D}:=\{(\mathbf{x},y)_{i}\}_{i=1}^{n}$, independent ...
1
vote
0answers
18 views

Difference between “given” and “sampled from” in an expectation

I'm reading a paper that states that trajectories $\tau$ are sampled from a distribution $\pi$ and they use the notation for the expectation $\mathbb{E}_{\tau \sim \pi}$ but also use $\mathbb{E}_{\tau ...
0
votes
0answers
23 views

optimization problem using softmax and cross-entropy function

I am encountering a problem concerning finding the optimal weights using cross-entropy loss function in the context of passive-aggressive algorithms. Let the following optimization problem:(eq1) $$J(...
2
votes
1answer
33 views

On the Rademacher complexity of a set.

Given a sequence $\{ \theta_j \}_{j=1}^{\infty} \in l^2(N)$, i.e., such that $\sum_{j=1}^{\infty} \theta_j^2 \le \infty$. Given a strictly positive sequence $\{ \mu_j \}_{j=1}^{\infty}$ we define $$...
-1
votes
1answer
21 views

How to select histogram bins for machine learning task?

I'm using a histogram values as features in the machine learning task. How to select the best bins? I thought at the beginning to break the range into large bins, than break significant bins again ...
-3
votes
0answers
57 views

math behind Support Vector Regression

I have been working with support vector machine for both regression and classification problems. But somehow I am not sure if I really get it. Therefore, Let me explain my understanding of SVR in ...
1
vote
0answers
31 views

Machine of maximum number of support vectors (SVM)?

I have learned a thing or two about Support Vector Machines (SVM) and it seems to me that maximum margin machines are popular. I came to wonder if there exist any flavour of SVM which not only strive ...
1
vote
0answers
18 views

Blind Signal Separation for sparse sources

Assume we have $N$ measurements $z_1, ..., z_N \in \mathbb{R}^{n_z} $ that generated by $$ z_i = M v_i + e_i $$ where $v_i \in \mathbb{R}^{n_v}$, $n_v < n_z$ and $e_i$ an error sampled from ...
1
vote
1answer
33 views

Two questions about the derivative of Softmax function.

Actually i have some problems with the derivative of softmax: $$y_k = \frac{e^{a_k}}{\sum_{i=0}^K e^{a_i}}$$ The first think i want to know is why the derivative of $\frac{\partial (\sum_{i=0}^K e^{...
0
votes
0answers
26 views

finding the optimum value of $w$ in passive aggressive classifire using cross entropy loss function

I am doing a multilabel online classification using categorical cross-entropy as the loss function and RBF neural network. I want to use the passive-aggressive method to train network but I have a ...
2
votes
0answers
48 views

Sampling from a categorical distribution [migrated]

I am confused by these lines of code: https://github.com/openai/maddpg/blob/master/maddpg/common/distributions.py#L174 ...
1
vote
0answers
27 views

Gradient descent versus finding where the gradient vanishes via solving systems of equations

I started learning machine learning and got stuck at the following questions: Why do we need to iterate the gradient descent algorithm? Why don't we equate the gradient to zero and find all local ...
-1
votes
0answers
41 views

Derivation from “The concrete distribution: A continuous relaxation of discrete random variables”

I am looking at "The concrete distribution: A continuous relaxation of discrete random variables". https://arxiv.org/pdf/1611.00712.pdf I do not understand the step "integrating our $r$" on page $...
1
vote
0answers
183 views

How to find diversified vectors in a given set?

I will describe my problem in details. I have a set of $N$ vectors, each of them defines a logistic regresion model. From this $N$ vectors (models) I want to take $n$ which are the most unique. I'm ...
0
votes
0answers
49 views

Can someone help me derive this equation?

I have read this paper Translation-based Recommendations and I have some question about the derivation. I'm not familiar with the derivative of vector. I want to derive $$\frac{\partial (\hat{p}_u,...
2
votes
2answers
54 views

Understanding L2 Regularization Formula

I am currently following the Machine Learning Crash Course on Tensorflow and came across this formula: $$L_2\text{ regularization term} = \|\boldsymbol w\|_2^2 = {w_1^2 + w_2^2 + \cdots + w_n^2}$$ I ...
0
votes
3answers
128 views

Good list of books for Year 13 about to apply for a BSc in Mathematics [closed]

I'm an international student about to go into my last year of high school, and I haven't found many mathematical books that interest me. I'm looking for a mixture of interesting, but respected books, ...
0
votes
0answers
10 views

Best learning algorithm for curve pattern determination

I have several intensity curves with bumps. Each bump is characteristic of one material(among A,B,C possibilities), The width of a bump and the material are the output I want to predict. I can have ...
1
vote
0answers
28 views

gradient for vector or matrix valued functioms

here (pageg 21) it says : It is very important to remember that the gradient of a function is only defined if the function is real-valued, that is, if it returns a scalar value. We can not, for ...
1
vote
1answer
61 views

Confused between Single Value Decomposition(SVD) and Diagonalization of matrix

I'm studying Principle Component Analysis (PCA), and came across this post. In which it's written that diagonalization of co-variance matrix ($C$) can be given by $$C = VLV^T$$ But as per difference ...
5
votes
3answers
163 views

Understanding the difference between Ridge and LASSO

I've asked this question a few days ago in the statistics site of this network, but although it's received a fair amount of views, I got no answer. If this kind of double posting is inappropriate let ...
0
votes
1answer
20 views

Reverse of Cauchy-Schuarz for two vectors [closed]

If $\|x\|_2\leq g\|y\|_2$, then according to Cauchy-Schuarz, the maximum of $ \langle x,y \rangle$ would be $ g\|y\|_2^2$ $$ \langle x,y \rangle \leq g\|y\|_2^2 $$ Can we prove the reverse, i.e., if $...
0
votes
0answers
24 views

Modeling with Support Vector Machine in regression problem

I have 20 independent variables(explanatory)and 1 dependent(respond) variable and I wish to use SVM as a regression problem to predict my dependent variable. but the point is that my independent(...
0
votes
1answer
56 views

Exercise 1.1 from Introduction of Pattern Recognition and Machine Learning by Christopher Bishop

So i'm stuck of exercise 1.1 because I am getting confused with the i's and j's I think. Also, I have done maths and further maths A level in the UK so have an okay maths background but I had to study ...
1
vote
0answers
11 views

Where does the hyper-plane begin with a SVM, an how does it iterate to it's ideal positioning?

This is a little tricky for me to word and have been studying support vectors. I understand where the hyper-plane should should sit on a graph, but I can't seem to find how this process starts out. ...
1
vote
0answers
24 views

Choosing a kernel embedding for a linear separator

The following is a slide from a presentation on machine learning, the context is using the kernel trick for non-linear separation. Does it make any difference that the mapping is $x\mapsto (x,x^{2})$...
1
vote
0answers
26 views

Backpropagation: Derivaiton of Loss Gradient

In the book "Artifical Intelligence: A Modern Approach" from S. Russel there is a derivation of the gradient of the loss with respect to the weights w used for backpropagation on page 735. I stumbled ...
1
vote
1answer
39 views

How do you take the gradient vector of the cross entropy cost function?

Let $\displaystyle J(\Theta) = -\frac{1}{m} \sum _{i=1}^m \sum_{k=1}^Ky_k^{(i)} \log(p)$ be the cross entropy cost function where $$p = \hat p_k^{(i)} = \frac{\large e^{(\theta^k)^T x^{(i)}}}{\large\...
2
votes
0answers
22 views

noise-free Gaussian Process likelyhood

I am learning Gaussian Process reading GPML. I am a bit confused with understanding the Bayesian analysis. Let consider the standard linear regression model with "Gaussian noise", i.e, $$ f(\textbf{x}...
1
vote
1answer
28 views

How does one code the generative adversarial network loss function?

I was reading Ian Goodfellow paper on GAN and I read that the loss function for GANs are : $J^{(G)} = -J^{(J)} = \frac{1}{2} \mathbb{E}_{x \sim p_{\rm data}}\Big[ \log D(x)\Big] + \frac{1}{2} \mathbb{...
0
votes
2answers
55 views

Is variance of mean equals mean of covariance?

I am trying to finish a problem, my method requires to prove variance of mean equals mean of covariance, but I have trouble proving it. Is it correct? Or more condition needed? Now I use $$Var(X)=E(X^...
1
vote
0answers
18 views

What is the summation of squared deviation wrt $\ell_2$-norm from vector mean in terms of individual deferences?

Let $\{\textbf{y}_t\}_{t=1}^T$ is a vector sequence. Summation of squared deviation wrt $\ell_2$-norm from vector mean of $\frac{1}{T}\sum \limits_{t=1}^{T}\textbf{y}_t$ is $$ \sum \limits_{t=1}^{T} \|...
1
vote
1answer
51 views

Proof/Example of finding co-variance matrix of normalized data matrix

This might be a very trivial question but I'm asking because I couldn't found any proper explanation. I'm currently studying Principle Component Analysis (PCA) and came across this post. Here it's ...
3
votes
0answers
31 views

Why is the Error surface for a 2 input neural network with 2 weights a parabolic bowl

I am new to machine learning and AI in general and had a quick question regarding the error function surface regarding a simple neural net: 2 input neural net After reading the following wiki: https:/...
2
votes
1answer
39 views

Stuck in derivation of Principal Component Analysis (PCA)

I'm currently studying Principal component analysis from this lecture notes. I under stand that we are trying to find the axis on which the variance of projection of all the data points is maximum. ...
1
vote
0answers
39 views

Understanding maths in adanet paper

I am reading up on adanet from here In the network architecture section the author defines it as let $l$ denote the number of intermediate layers in the network and $n_k$ the maximum number of ...
1
vote
1answer
38 views

Gradient/Steepest Descent: Solving for a Step Size That Makes the Directional Derivative Vanish?

The following excerpt is from chapter 4.3 of Deep Learning, by Goodfellow, Bengio, and Courville: The authors state that sometimes we can solve for the step size that makes the directional derivative ...