Questions tagged [machine-learning]

How can we build computer systems that automatically improve with experience, and what are the fundamental laws that govern all learning processes?

Filter by
Sorted by
Tagged with
0 votes
1 answer
25 views

What is the growth function of the hypothesis set of linear classifiers on $\mathbb{R}^2$?

Recently I've been studying machine learning. I want to find out the growth function of the following hypothesis set: $$ \mathcal{H}=\left\{h_\mathbf{w}\equiv\mbox{sign}\big(\langle\mathbf{w}, \cdot\...
yanruijie's user avatar
  • 448
0 votes
0 answers
21 views

differential-privacy: show $\epsilon$ -differentially privacy

In this problem we consider a sensitive dataset $x \in \{−1, 1\}^n$. We consider the bounded setting where neighboring n-dimensional datasets differ in one coordinate. $A$ mechanism is available that ...
Lifeni's user avatar
  • 656
-2 votes
1 answer
24 views

When is my lightgbm going to find cut points in random variables that reduce entropy more than a naturally correlated variable with the target? [closed]

In machine learning sometimes we build models using hundreds of variables/features that we don't know (at least at first) if they might have a relation with the target. Usually we find that some of ...
Alejandro Gómez's user avatar
0 votes
0 answers
10 views

What's the probability of one binary classifier better than another knowing the results over a sample of size $N$?

Assuming that classifier_1, classifier_2 have an unknown hit ratio $α, β$, what is the probability that $α>β$ if after doing an experiment on a sample of size 20 classifier_1 gets a hit ratio of 80%...
Alejandro Gómez's user avatar
1 vote
0 answers
57 views

Optimization of the sum of a quadratic form and the L1-norm of logarithm

Given the symmetric positive definite matrix $W\in\mathbb{R}^{n,n}$ and the positive scalar $\lambda $, the objective reads $$\min_{x} x^{T}Wx + \lambda \|\log(x)\|_1, $$ where $\|\cdot\|_1$ denotes ...
yuuuu's user avatar
  • 21
-2 votes
0 answers
11 views

Can Midas regression involve independent variables of different frequnecies?

Can the dependent variable (GDP) be quarterly and it dependent on mix of monthly and weekly independent variables?
J_Neik's user avatar
  • 1
-1 votes
0 answers
40 views

How to find an "distance maintaining" mapping $f:X \to Y$, where $X \subset \mathbb{R}^m$, $Y \subset \{0,1\}^n$, and $m<n$ in a relatively easy way?

As stated in title, the function can map data points from a lower dimensional Euclidean space into a higher dimensional space, and by "isometric" (may not be strictly correct in this context ...
Aloner's user avatar
  • 93
0 votes
1 answer
34 views

Show that the equivalence of MAP in Bayesian estimation to Structural Risk Minimization (SRM)

I'm sorry that I didn't explain my problem clearly😭I would like to add something. I saw this problem in Machine Learning Method written by Li Hang(p16). The book states the problem as below: "...
chenqile's user avatar
0 votes
0 answers
7 views

Log probability of following a trajectory under an optimal policy

In reinforcement learning, is the log probability of following a trajectory under an optimal policy equal to the sum of rewards for that trajectory? i.e. $\log(p(\tau)) = \sum^T_{t=1}r(s_t,a_t)$ I've ...
Nicholas James Bailey's user avatar
2 votes
0 answers
29 views

Empirical distribution learns w.r.t total variation distance

I am trying to prove or disprove that the empirical distribution can learn any continuous distribution w.r.t the total variation distance. The context is the one of statistical learning. I am quite ...
pppp0l's user avatar
  • 21
0 votes
0 answers
19 views

Optimizing function defined by integral

Let the two functions $q: \mathbb{R}^d \rightarrow\mathbb{R}^{+}$ and $s: \mathbb{R^d} \times \mathbb{R^d} \rightarrow \mathbb{R}^{+}$, $d \in \mathbb{N,}$ where both are assumed to be continuous and ...
blockchain187's user avatar
0 votes
0 answers
17 views

variance maximization in PCA: doubt for the correctness of an Algerian passage.

I know trying to perform the steps omitted in Bishop's machine learning book (from point 12.4 to 12.5): there is only one step that leaves me with a doubt of correctness namely whether I can, taken ...
Gianni's user avatar
  • 235
1 vote
0 answers
12 views

Unclear maximization step in the PCA for machine learning [closed]

In Bishop's book (Pattern Recognition and Machine Learning, chapter PCA) there is this passage for calculating the gradient with respect to $\vec{u}_{1}$. In the passage, $\vec{u}_{1}^{T}$ are ...
Gianni's user avatar
  • 235
1 vote
1 answer
108 views

Shapley Kernel Proof

I read the paper about SHAP. I think this paper is very interesting ! I would like to understand the algorithm, but I cannot follow the below fact. \begin{equation} X^T WX=\dfrac{1}{M-1}I+cJ \end{...
user18000142's user avatar
2 votes
0 answers
41 views

How to make the income distribution of a country follows the 80:20 rule?

My main question: Let's imagine a country with a population of $n$ people. Each person has a certain amount of income in a certain year. When we calculate the income distribution of this country, we ...
MonsterTruck123's user avatar
0 votes
1 answer
19 views

How is this reinforcement learning value formula read / understood?

$$V_\pi(s) = E[R_t|s_t=s,\pi]$$ This is a value function for state s under policy $\pi$ where $R_t$ is the return value, all of which occurs at time t. I was wondering how I should read/ understand ...
Connor McLoon's user avatar
0 votes
0 answers
27 views

Limiting probability of classifying correctly with the k-NN algorithm as the number of data points increases.

Let $x_1, \ldots, x_n$ be random variables that are uniformly distributed on [0,2]. If $x_i \leq 1$, we'll classify it as green ($y_i=0$), and if $x_i > 1$, we'll classify it as red ($y_i=1$). We ...
Tom's user avatar
  • 413
0 votes
0 answers
9 views

KKT Conditions for SVM Problem

I am reading about SVMs and want to confirm that I understand the optimality conditions. Details below: Consider the $n$ points $x_1, x_2, \dots, x_n$, each with $ d$ dimensions, and consider $ n$ ...
user35083's user avatar
0 votes
0 answers
62 views

VC dimension of indicator functions is equal to pseudo dimension

I am reading the "Foundation of machine learning" by Mehryar Mohri (https://cs.nyu.edu/~mohri/mlbook/). In the proof of Theorem 11.8, it said the following statement, which I can not ...
Harry's user avatar
  • 659
1 vote
0 answers
25 views

Are we finding the density of $x$ or evaluating the density of $\theta$ at $x$? | Alpyadin Machine Learning

In section $4.4$ The Bayes Estimator of Alpaydin he discusses the use of the prior density of $p(\theta)$ to construct a posterior density for $\theta$. This is standard Bayesian estimation to get a ...
AnzBh's user avatar
  • 11
0 votes
0 answers
23 views

Maximizing the summation of reward of some users (waiting in different positions and lines) using reinforcement learning or other learning methods

There is a mathematical problem that I think can be solved using reinforcement learning and it would be great if you could help me with it. Some users are standing in some lines. There are N lines. In ...
MHB's user avatar
  • 1
2 votes
0 answers
37 views

Recommendations for Information Geometry in Machine Learning

I am fairly new to machine learning, but I have a 22-dimensional dataset, which I would like to increase the interpretability of by dimension reduction. I am relatively familiar with principal ...
sam wolfe's user avatar
  • 3,083
0 votes
0 answers
12 views

Kernel density estimators in Bishop book unclear formulas

In Bishop's book (Pattern recognition and machine learning, pag 122) there is an unclear passage for me in deriving certain formulas: $E[K/N] = P$ and $var[K/N] = P(1-P)$ Considering binomial ...
Gianni's user avatar
  • 235
1 vote
0 answers
36 views

Approximating an isometry with a neural network

I'm trying to link a machine learning framework to more theoretical considerations about linear isometries. Let's say we have an input dataset $\mathcal{D}=\{(\textbf{x}_1,y_1), (\textbf{x}_2,y_2), \...
James Arten's user avatar
  • 1,884
0 votes
0 answers
46 views

About the inputs of the Wasserstein Distance $W_1$

Introduction (this is just supporting my questions, but you can skip it and go directly to the questions). Let's consider the following Proposition from "Ramdas & Trillos(2015) On ...
Ommo's user avatar
  • 255
-1 votes
0 answers
9 views

Why does gradient vector update parameters based on how the loss changes due to the parameters? [closed]

Essentially, I want to know why the gradient vector points in the direction of steepest increase. Like why does a vector that moves in the x direction the same amount as the loss changes due to x ...
Power lord's user avatar
1 vote
0 answers
35 views

Is this smooth linear ramp function already a thing?

In machine learning (particularly with regards to Neural Nets), there's a bunch of "ramp" functions that're used. For example, the ReLU is $0$ for $x\leq 0$ and $x$ for $x>0$. I was ...
chausies's user avatar
  • 2,060
0 votes
0 answers
16 views

Finding an algorithm EF[1,1] and PO division for more than two agents

From this research paper I want to write an algorithm for finding envy-freeness(EF) and Pareto optimality(PO) division for more than two agents. We consider the problem of fairly and efficiently ...
user19121278's user avatar
5 votes
1 answer
162 views

ODE's: Continuity Equation

The context of this question is Machine Learning (more specifically, my question results from this paper, yet I have a math question, so I'm posting it here). First of all, some definitions (Sec. 2 of ...
Hermi's user avatar
  • 642
0 votes
0 answers
11 views

Expectation of linear form multiplied by quadratic form for MVN distribution

Assume that $\bf{x}$ is a random vector that is distributed multivariate normal with mean $\boldsymbol{\mu}$ and covariance matrix $\boldsymbol{\Sigma}$. Let $\bf{A}$ be a matrix of constants. I'm ...
max's user avatar
  • 194
0 votes
0 answers
23 views

How do I obtain the primal and dual for the regression estimator $\min _\beta[\|\beta\|^2+\sum_{i=1}^n \xi_i^2]$ s.t. $\xi_i=y_i-h(x_i)^\top \beta$?

I am working on a statistical learning exercise that requires some knowledge of convex optimization which I am unfortunately lacking. Consider the linear regression model $$y_i=h(x_i)^\top\beta+\...
Leon's user avatar
  • 61
1 vote
1 answer
66 views

$\min$-entropy for the uniform distribution on $[𝑛]$

The min-entropy of a distribution $\nu$ on $[n]$ is given as: $$H_{\infty}(\nu)=\min_{i} \log(\frac{1}{\nu(i)})$$ Now we will prove that that for every distribution $\nu$ on $[n]$ and for $U$ being ...
Lifeni's user avatar
  • 656
0 votes
0 answers
45 views

Partial derivate of scalar-valued function of a vector variable

Working through Mathematics for Machine Learning and got stuck on the end-of-chapter exercises 5.8a): Calculating $\frac{\partial g}{\partial y}$ for $$g(y) = y^T S^{-1} y$$ with $y \in \mathbb R^{D}$...
TassiloPitrasch's user avatar
0 votes
0 answers
38 views

Derivative Least Square Regression (Numerator vs denominator layout)

Assume I have the following expression: $$\frac{\partial}{\partial w} \lVert X^Tw - y \rVert^2 = 0$$ which is trying to find the solution for the Least Squares Approach in Regression. Let's assume $X \...
Leon0402's user avatar
1 vote
0 answers
37 views

Coefficient for the gradient term in stochastic gradient descent (SGD) with momentum

I'm studying SGD with momentum and have come across two versions of the update formula. The first is from a wiki same as from the original paper: $$ \Delta w^t = \alpha * \Delta w^{t-1} - lr * \nabla ...
W Lewis's user avatar
  • 23
0 votes
0 answers
12 views

Does skipgram model uses backpropagation? [migrated]

I just started to get interested in natural language processing and I was trying to understand the skipgram model from word2vec. I was reading this interesting website. However, in the mentioned ...
edamondo's user avatar
  • 1,208
0 votes
1 answer
47 views

Is this a closed-form analytical solution for the hard-margin SVM dual problem?

I have been searching, without much success, for some dicussion on the possibilities (or impossibilities) of a general closed-form analytical solution for the hard-margin (only) support vector ...
Papyrus's user avatar
1 vote
1 answer
89 views

Understanding a statement in Sutton's Reinforcement Learning Section 5.5 on Importance Sampling

I am trying to understand chapter $5.5$ of Sutton's Book on Reinforcement leaning, in a particular a statement on page $104$ related to off policy prediction via importance sampling. Supposing $b$ is ...
IntegrateThis's user avatar
0 votes
0 answers
50 views

How can I understand the (no) independence property in this very simple setting of first order logic?

My understanding of logic is limited to first order logic without functions with finite set of domain constants, and with herbrand semantics. Now in this setting, I would like to understand the ...
SagarM's user avatar
  • 1,777
0 votes
0 answers
14 views

Modeling Regional Quality Based on SUM of GCL / SUM of TAP - Comparing Two Approaches

I'm learning Data Science and I'm currently working on analyzing loan data across different regions. My goal is to build a model that can assess the quality of a given region based on the ratio $$\...
WawMathematician's user avatar
0 votes
0 answers
17 views

convert a 1-dimensional set of points to a 2-dimensional parabola with explicit embedding

I am trying to rephrase to better understand concepts regarding discriminant functions for classification using explicit embedding. I report a very easy diagram found online that from 1-dimension ...
Gianni's user avatar
  • 235
4 votes
0 answers
76 views

Book suggestion on "Banach space geometry for machine learning"

Is there any book for a Mathematics student who can learn Machine learning in the aspect of Banach space geometry? Or, one can understand the connection between Geometry of Banach spaces and Machine ...
Tutun's user avatar
  • 129
3 votes
1 answer
174 views

Rearrangement property of diagonal matrices [closed]

Is it true that for a diagonal matrix $B\in\mathbb{R}^{n\times n}$, a matrix $A\in\mathbb{R}^{n\times p}$, the following property holds: $$ A^\top B A = A^\top A B, $$ where in essence I am asking ...
Resu's user avatar
  • 694
0 votes
0 answers
16 views

Reason behind objective function in Linear Discriminant Analysis

I don't really understand the objective function to be optimized in Linear Discriminant Analysis (LDA). My question is centered around the same concepts mentioned in this this other one. The analysis ...
Alberto's user avatar
  • 453
0 votes
1 answer
41 views

Unclear passages in Bishop's machine learning book for Probabilistic Generative Models

I am trying to prove the relationship (4.65) expressed in Bishop's book (Pattern Recognition and Machine Learning) in chapter (4.2.1) Continuous inputs: I found the following passages but there is ...
Gianni's user avatar
  • 235
0 votes
0 answers
17 views

Margin width and normalization in SVM

Let $x_1$ and $x_2$ be two support vectors and $w$ an orthogonal vector to the decision hyperplane. To find the width of the margin, I don't understand why we have to calculate the dot product ...
weldon's user avatar
  • 15
0 votes
1 answer
26 views

Define 'accuracy' for numerical data?

Normally, people use 'accuracy' to describe the output quality (from a model or methodology https://en.wikipedia.org/wiki/Precision_and_recall) for categorical data. However, I am wondering could the ...
Edamame's user avatar
  • 113
0 votes
0 answers
30 views

Perceptron Convergence: Monotonically Approach Solution?

I'm new to learning about perceptrons, but I saw a proof (for perceptron for binary classification with the caveat of forcing the separator through the origin) that, assuming the data is linearly ...
mishar's user avatar
  • 101
0 votes
1 answer
26 views

Unclear formula in the book of linear model for classification in the Bishop book (machine learning)

From Bishop's book pag 183 - (4.11) (Pattern Recognition and Machine Learning, 2009), a mathematical relationship is immediately shown without an explanation of where it is derived from: I cannot ...
Gianni's user avatar
  • 235
1 vote
1 answer
34 views

Understanding an inequality related to Diffusion Processes in Machine Learning

I'm trying to understand an equality from a blog post about diffusion models here in the Reverse diffusion process section. Here $\beta_t \in (0, 1)$, $\alpha_t = 1-\beta_t$, and $\bar{\alpha_t} = \...
IntegrateThis's user avatar

1
2 3 4 5
65