Questions tagged [logistic-regression]

For questions about logistic regressions, a regression model where the dependent variable is categorical.

Filter by
Sorted by
Tagged with
0 votes
0 answers
23 views

Why in logistic regression for every threshold the decision boundary is a hyperplane?

I'm a beginner in machine learning study and I can't figure it out an exercise: The function h(x) = θ(w ̃x) is used to approximate ...
user avatar
0 votes
0 answers
9 views

Normality assumption for predictors?

In data science, should we ensure that all variables are normally distributed? I do understand that the y-value would need to be normally distributed as a logistic regression assumption. For instance, ...
user avatar
  • 1
0 votes
0 answers
50 views

Compute Lipschitz constant of Hessian of logistic loss

Convergence theory of Newton's method assumes that the Hessian $\nabla^2 l(w)$ of the loss function l is Lipschitz continuous, i.e. $\big\Vert \nabla^2 l(w_1) - \nabla^2 l(w_2)\big\Vert_2 \leq L \big\...
user avatar
0 votes
0 answers
27 views

Gradient descent calculation split in 2 steps

In a machine learning course I am taking we have an assignment with a notebook (In linear models). So far I have calculated the cost and the sigmoid in a logistic regression and the next exercise ...
user avatar
1 vote
0 answers
12 views

How to explain covariance in logistic regression + analogy to linear regression

Introduction Linear model In linear regression we predict continuous variable $Y \in R^n$ with use of $n \times p$ deterministic plan matrix $X$ and theoretical model (let's ignore intercept ...
user avatar
0 votes
0 answers
16 views

Condition logit model: Weighted mean of ratios of coefficients of subgroups does NOT equal the ratios of coefficients of the whole sample, why?

I am well aware that when one splits the sample into subgroups (e.g. sex, country, whatever), and then estimates any logistic regression model, the coefficients are not comparable between (otherwise ...
user avatar
0 votes
0 answers
18 views

Understanding derivation of Logitboost

I am trying to understand the derivation of LogitBoost. I looked at the original paper by (Friedman, Hastie and Tibshirani, 2000) "Additive Logistic Regression: A Statisitcal View OF Boosting&...
user avatar
  • 304
1 vote
0 answers
37 views

Intuition of Negative entropy as a Fenchel conjugate of logistic cost function

Let us first recall the logistic cost function $$f_1(x)=\log(1+e^x)$$ and the negative entropy function $$f_2(y)=y\log(y)+(1-y)\log(1-y).$$ Here $\log$ denotes the natural logarithm and $x\in \mathbb ...
user avatar
  • 1,208
0 votes
0 answers
31 views

How do you transform a Derivative to a Recurrence Equation?

In the logistic equation, they say $\frac{dx}{dt} = rx(1-x)$ is equal to $x_{n+1} = rx_{n}(1-x_{n})$. How do they do that? And if you were to transform $\frac{dx}{dt} = 2$ to the recurrence equation, ...
user avatar
-1 votes
1 answer
55 views

Machine learning - Cost function for non linear functions [closed]

The cost function is some indication of the 'cost'/how the predicted value differs from the actual value. In linear regression, this can be measured using MSE. In the case of the logistic function, ...
user avatar
  • 77
2 votes
0 answers
102 views

Proof that the MLE of logistic regression doesn't have a closed-form solution

It is well known that the maximum likelihood estimator of logistic regression does not admit a closed form solution, at least in the general case where the predictors are not binary or categorical. ...
user avatar
0 votes
1 answer
24 views

explanation for cross entropy for logistic regression

as far as I know, cross entropy of two distributions is: $$ C(p,q) = -\sum_{s \in classes}p(s)\log(q(s)) $$ however, the loss function for logistic regression (called "crossentropy loss") it'...
user avatar
0 votes
1 answer
22 views

What is the best way to estimate the parameters of a logistic regresion model?

I recently read about logistic regression model. $$y=\frac{1}{1+e^{-(\beta_0+\beta_1x)}}$$ where y is a categorical variable with either 0 or 1 output. What seems to be perplexing to me is, I can see ...
user avatar
  • 154
3 votes
1 answer
54 views

How to determine labeled balls for logistic regression

I am studying this paper about logistic regression. In section 4.2 (Randomly Generated Problems) on page 1534, they say "Features of positive (negative) examples are independent and identically ...
user avatar
  • 931
0 votes
1 answer
38 views

Working out the derivative of the log-likelihood for group LASSO

I'm following the working of the sparse group LASSO in the paper 'A Sparse-Group LASSO' by Simon. For the linear case, we have the problem given as $$\text{min}_\beta \frac{1}{2}||y-\sum_{l=1}^m X^{(l)...
user avatar
  • 13
0 votes
1 answer
51 views

Mathematical derivation from Deep learning book

In https://www.deeplearningbook.org/contents/mlp.html p. 179 this derivation is made but lacks details that I need to understand it: $P(y)= \frac{exp(yz)}{\sum_{y’=0}^{1}exp(y’z)} = \sigma((2y-1)z)$ ...
user avatar
0 votes
0 answers
10 views

Does every neuron in a feedforward neural network produce a binary output?

My question is very simple, but different sources seem to contradict each other. Given a neuron of a simple feedforward neural network, we know it takes the scalar product of its input vector and its ...
user avatar
0 votes
0 answers
12 views

Binary to Multiclass logistic regression and vice versa

As I was working on a problem, I came across the mention of logistic regression being used for binary and multiclass problems. Specifically, I am very keen on the problem with the below equations. How ...
user avatar
  • 1
0 votes
0 answers
40 views

Logistic Regression Problem

You are testing a new drug and have gathered binary data on whether the drug performed its desired effects. From the control trial, $102$ people saw improvement with a placebo and $241$ did not. With ...
user avatar
  • 35
0 votes
0 answers
27 views

Find z as $\tau\to0$

Let $x$ be $(x_1, x_2, ..., x_K)^T$ and $z$ be $(z_1, z_2, ..., z_K)^T$ be 2 K-dimensional vectors. Each dimension of z is defined as $$z_j=\frac{e^{x_j/\tau}}{\sum_{k=1}^{K}e^{x_k/\tau}},\tau>0, 1\...
user avatar
  • 145
2 votes
3 answers
143 views

Implementing multiclass logistic regression from scratch

This is a sequel to a previous question about implementing binary logistic regression from scratch. Background knowledge: To train a logistic regression model for a classification problem with $K$ ...
user avatar
  • 48k
2 votes
1 answer
59 views

Why there is no $y_i$ term in $\frac{d^{2} J(\boldsymbol{\alpha})}{d \alpha_{i}^{2}}$?

$$\frac{d^{2} J(\boldsymbol{\alpha})}{d \alpha_{i}^{2}}=\lambda^{-1} \mathbf{x}_{i}^{\mathrm{T}} \mathbf{x}_{i}+\frac{1}{\alpha_{i}\left(1-\alpha_{i}\right)}$$ I cannot understand why there is no $...
user avatar
  • 371
0 votes
0 answers
35 views

Single variable logistic regression

I am trying to write out the code for using Newton-Raphson to solve for a coefficient and intercept in a single variable logistic regression. However in both the math and in the code I see that the ...
user avatar
3 votes
1 answer
68 views

Implementing binary logistic regression from scratch

Background knowledge: To train a logistic regression model for a classification problem with two classes (called class $0$ and class $1$), we are given a training dataset consisting of feature vectors ...
user avatar
  • 48k
1 vote
1 answer
31 views

Solving for the Logistic Regression Coefficients

I have been searching for a source on how to compute the coefficients for logistic regression, but haven't found any. I suppose it is very easy to find like linear regression. Can you kindly suggest a ...
user avatar
  • 1,608
0 votes
0 answers
57 views

How to compute the gradient of this multinomial logistic loss function?

How to compute the first and second derivative of the multinomial logistic loss function with respect to $w_k$ that is minimized in this statement? $$ \begin{array}{ll} \min _{\mathbf{w}} & \frac{...
user avatar
  • 371
0 votes
2 answers
132 views

How to plot a decision boundary for binary logistic regression in matlab

let me preface by saying this is from a homework question, but the question is not to plot the decision boundary, just to train the model and do some predictions. I have already done that and my ...
user avatar
  • 145
0 votes
1 answer
22 views

confused about beta weights in logistic regression

I've recently started getting into machine learning, and I saw that making a simple classifier using logistic regression was a good place to start. I was following through this article and got to the ...
user avatar
1 vote
0 answers
16 views

How to estimate the covariance matrix for outcome of logistic mixed-effects model

I have a logistic mixed-effects model below. $\mathbf{y}=(y_1, y_2, ..., y_n)^T$ is a n dimensions vector. $\mathbf{p}=(p_1,p_2, ...,p_n)^T$ is a n dimensions vector. $logit(\mathbf{p}) = (logit(p_1),...
user avatar
  • 155
1 vote
1 answer
34 views

For the Logistic model, why is the objective function unbounded below if two sets are linearly seperated?

I am reading Approximate linear discrimination via logistic modeling in the Section 8.6.1 of B & V's Convex Optimization book. On Page 428, $$ \operatorname{minimize} \ -l(a, b) \tag{8.27} $$ ...
user avatar
  • 194
0 votes
0 answers
14 views

Logistic discrete equation: analitic solution

I have to proof that the solution for the logistic discrete equation: $x_{n+1}=4x_n\left(1-x_n\right)$ with $r=4$, and $x_0 \in (0,1)$ when $n=0$, it's of the form $$x_n=\sin^2\left(\beta\alpha^n\...
user avatar
  • 9
0 votes
0 answers
31 views

Why is softmax normalization so popular?

Is any theoretical grounding that suggests that the softmax normalization performs better than other normalization functions? So far, the only valid reasons I've seen are that it scales values to $[0, ...
user avatar
0 votes
0 answers
9 views

Interpretation of Proportion change in regression

I have two equations: expenditure = 10 + 20*income where income is measured as a proportion between 0 and 0.7 ln(expenditure) = 3.5 + 1.2 * income Assumption: expenditure measures in thousand ...
user avatar
  • 13
2 votes
1 answer
108 views

Show that the joint distribution of $X_1, X_2, \dots, X_n$ belongs to a two-parameter exponential family.

Let $X_1, X_2, \dots, X_n$ be ind $ \sim\text{Ber}(\theta_i)$ where \begin{equation} \theta_i = P(X_i=1)=\frac{\exp(\alpha+\beta t_i)}{1+\exp(\alpha + \beta t_i)} \end{equation} where $t_1, ...
user avatar
  • 77
0 votes
0 answers
24 views

Which classification method to choose?

I am working on a statistics project in which I analyze 4 different classification methods (mostly for binary classification and with quantitative input variables). On the one hand I'm studying ...
user avatar
0 votes
1 answer
63 views

Fitting data to a logistic function

I've spent my whole entire weekend struggling with this problem. If we are given a set of data and are asked to fit it into a logistic equation: $\frac{dP}{dt} = bP(\frac{a}b-P)$, where a and b is ...
user avatar
0 votes
0 answers
26 views

How probabilities and predictions are calculated in logistic regression?

I have these values of y and x: y x 0 3 0 4 0 7 1 8 1 11 0 14 1 15 1 16 1 17 I want to calculate prediction for values 9 and 20 using logistic regression....
user avatar
2 votes
1 answer
735 views

What is the derivative of binary cross entropy loss w.r.t to input of sigmoid function?

I want to compute the derivative of binary cross entropy loss w.r.t to the input of the sigmoid function and was wondering if there's a closed form expression? I've seen derivations of binary cross ...
user avatar
1 vote
1 answer
45 views

Comparing two explanatory variables explanatory variables

I am new to stats, I am working on a problem and I need some direction. I have a binary dataset and I have three explanatory variables x,y, and z that I want to analyze. There is no guarantee of ...
user avatar
  • 13
0 votes
1 answer
63 views

How can we derive the Normal equations for Logistic Regression?

I was wondering from numerical linear algebra point of view that since solving OLS $\|b-Ax\|_{2}=\min_{w\in\mathbb{R}^{n}}\|b-Aw\|$ is equivalent to solving the normal system $$ A^{T}Ax=A^{T}b $$ then ...
user avatar
  • 331
1 vote
0 answers
25 views

How to know a classifier is a linear or not linear in classification problems?

I read a lot , but still not able to get the following concepts -: (1) If a classifier is given, how do we know whether its a linear or non linear classifier? (Interested in step by step procedure to ...
user avatar
1 vote
1 answer
60 views

Likelihood values from Sigmoid

There are multiple doubts of mine associated around this theme: In MLE, we try to find the PDF parameters ($\theta$) which maximise the likelihood of the observed data ($L(\theta | data)$). To get ...
user avatar
  • 324
0 votes
0 answers
7 views

Variance Covaraince Matrix of Parameters in Logistic Regression

The given below image is taken from book Introduction to Linear Regression Analysis (Douglas C Montgomery) My apologies in advance for not typing , I just want to understand the concept. (1) First ...
user avatar
1 vote
2 answers
93 views

Deriving Probability from logit or log-odds

How can I motivate the derivation of the $p$ below? From: https://en.wikipedia.org/wiki/Logistic_regression#Logistic_model $l = log_b\frac{p}{1-p}=\beta_0 + \beta_1x_1+\beta_2x_2 $ $p = \frac{b^{\...
user avatar
  • 225
1 vote
0 answers
17 views

Account Resurrection as a function of Days Dead and Time to Resurrect

I am a data scientist working for a company that takes user deposits. I wanted to answer the question of how likely an account that's dropped to $0 on deposit - or dies, in other words - would refund ...
user avatar
0 votes
0 answers
4 views

Creating adversarial sample by projecting my data on the other side of the hyperplane

I performed a binary classification using logistic regression. My goal is the following: I know the coefficient w of the hyperplane equation y = wTx + b. What I would like to do is create opposing ...
user avatar
0 votes
0 answers
25 views

Why is the sigmoid of a linear model equal to the probability of the target being $1$?

From this resource, the writer starts with a linear model: $$ y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + ... + \beta_m x_m $$ and then makes the RHS sigmoidal. This must then make the LHS sigmoidal to ...
user avatar
  • 1,871
0 votes
1 answer
15 views

Why does linear regression output numbers but logistic regression output probabilities?

From reading this resource, the writer wrote: A linear model does not output probabilities, but it treats the classes as numbers (0 and 1) and fits the best hyperplane (for a single feature, it is a ...
user avatar
  • 1,871
0 votes
0 answers
28 views

Binary Logistic Regression with labels $\{-1, 1\}$

I am reading about binary logistic regression and I have found an equivalent formulation given by $$\min_{w} \frac{1}{m}\sum_{i} \log \left(\exp(-y_iX_iw) + 1\right),$$ where $X \in \mathbb{R}^{m \...
user avatar
  • 721
1 vote
0 answers
65 views

How to prove the non convexity of logistic regression?

In linear Regression we have the following loss function :- $$L(x)=\frac1{n}\left(\sum_{i=1}^n ((a+bx_i)-y_i)\right)^2$$ Hence, we can observe that L(x) is having a convex graph but when we use the ...
user avatar

1
2 3 4 5