Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [logistic-regression]

For questions about logistic regressions, a regression model where the dependent variable is categorical.

9
votes
4answers
2k views

What is e in this equation, and how do I solve it?

Apologies for the rudimentary question. I haven't studied math and can't find an answer to this online. Is the '$e$' in this equation for logistic regression Euler's number? If so, it doesn't matter ...
8
votes
2answers
2k views

Invert the softmax function

Is it possible to revert the softmax function in order to obtain the original values $x_i$? $$S_i=\frac{e^{x_i}}{\sum e^{x_i}} $$ In case of 3 input variables this problem boils down to finding $a$, ...
7
votes
2answers
95 views

How to estimate of coefficients of logistic model

Consider model $logit(p)=a+bx$. I would like to get a analytic formula of $a$ and $b$ like in linear regression. In linear regression, we can get a formula of estimates of $a$ and $b$. I tried using ...
6
votes
2answers
3k views

Show that logistic regression with squared loss function is non-convex

How would you show that if you do logistic regression with a squared loss function, it is not a convex optimization problem (in parameters)? In other words, your loss function for an individual ...
4
votes
3answers
12k views

Logistic regression: Prove that the cost function is convex

I'm reading this You can do a find on "convex" to see the part that relates to my question. Background: $h_\theta(X) = sigmoid(\theta^T X)$ --- hypothesis/prediction function $y \in \{0,1\}$ ...
4
votes
1answer
312 views

Find the MLE of a GLM

(Note this is not an assignment, but revision for a topic from Cambridge past exam papers) I have been trying to attempt the below question, and I am struggling with part (b). For (a) it is ...
4
votes
0answers
78 views

What do the parameters of a multinomial logistic regression correspond to?

I've recently started learning about data science/statistics and learned how to derive such models as linear regressors and logistic regressors. What I don't understand, however, is what the ...
3
votes
1answer
2k views

Likelihood function for logistic regression

In logistic regression, the regression coefficients ($\hat{\beta_0}, \hat{\beta_1}$) are calculated via the general method of maximum likelihood. For a simple logistic regression, the maximum ...
3
votes
0answers
51 views

Vehicle Routing Problem that minimizes Total Time instead of Total Cost, with a few alterations

A company has to collect waste for different customers, at different locations, using two vehicles both with a certain capacity. The vehicles have to begin and end at the depot and when the capacity ...
3
votes
1answer
560 views

Logistic regression and cross-entropy

Cross-entropy is a good perspective to understand logistic regression, but I have the following question: the objective function of LR: $$\max L(\theta) = \max \sum_{i=1}^N y_i \log \hat y_i + (1-y_i)...
2
votes
1answer
2k views

How is the cost function $ J(\theta)$ always non-negative for logistic regression?

I am studying Logistic Regression from Andrew Ng's Machine Learning Course. A quiz in the course stated that The cost function J(θ) for logistic regression trained with m≥1 examples is always ...
2
votes
1answer
721 views

Cost Function of Neural Network (Forward Propagation)

This question is related to Andrew Ng's machine learning course on Coursera. Basically, when I calculate the cost function of a neural network, I use the following formula that was described by Ng: $$ ...
2
votes
1answer
114 views

Monotonic transformation to smooth the probabilities

I am studying some event for a set of objects that can be plotted on a square $[0, 100] ^ 2$. I have used logistic regression to calculate probabilities that event occur for different objects and the ...
2
votes
2answers
72 views

Fitting logistic regression models

I am studying The Elements of Statistical Learning book and I have a question. On pages 120-121 the logistic regression problems is rewritten in the form of matrix and vectors products as follows:(4....
2
votes
1answer
440 views

why do we use logistic function to build logistic regression.

Why do we use logistic function to build logistic regression. I know the output value for logistic function bounds between 0 & 1, and bcos of this we can express as probability. Is this the reason ...
2
votes
1answer
54 views

Logistic regression - Transposing formulas

I am trying to understand the math behind Logistic regression. I am confused about transposing one formula to another. Here is what I have: Our regression formula $$\ y = b_0 + b_1x$$ Our sigmoid ...
2
votes
2answers
576 views

If logistic is the log odds ratio, what's softmax?

I recently saw a nice explanation of logistic regression: With logistic regression, we want to model the probability of getting success, however you define that in the context of the problem. ...
2
votes
1answer
177 views

Solving the logistic equation [closed]

I need to solve the logistic equation $$\frac{dP}{dt} = P(a-b\ln P)$$ How do I go about solving this?
2
votes
1answer
117 views

Simple Logistic Regression - how do I use real data?

Binomial Logistic Regression to predict probability Confusion Point 1: I think I'm right in saying one of the steps of Logistic Regression is to get: $$\log(\mathrm{Odds})$$ Now take this very ...
2
votes
1answer
504 views

Logistic curve through three points

I need to find a logistic curve that passes through three points exactly. This means I cannot do a best fit but rather must use simultaneous equations. Essentially this is used to model population ...
2
votes
1answer
85 views

Deriving the odds ratio of a 3-way interaction logistic regression model

Suppose a logistic regression model has three binary explanatory variables $x_1$, $x_2$ and $x_3$ used to estimate the probability of success. This model includes all three main effects, the three $2$-...
2
votes
0answers
36 views

Is there a nice matrix expression for the gradient of the cross-entropy for multinomial logistic regression?

Does the gradient of the cross-entropy have a nice matrix expression? Let $\mathbf X$ be a matrix whose row vectors are features, and $$\mathbf Y_{ij} = \begin{cases} 1 & \text{if the $j$th row ...
2
votes
0answers
85 views

Distribution of logistic regression estimators

It is well known that for OLS estimators, the parameters are asymptotically normal, i.e. for the regression $y_i = \beta_i x_i$, $$\hat{\beta_i} \sim \mathcal{N}(\beta_i, \sigma^2 (X_i^T X_i)^{-1})$$ ...
2
votes
0answers
275 views

Sum of Sigmoid of Normal distribution

I have: $$p_{\text{sigmoid}_i} =\frac{e^{a+b{x_i}}}{1+e^{a+b{x_i}}}$$ where $x_1, \ldots, x_n$'s are generated from a normal distirbution with mean $\mu_0$ and $x_{n+1},\ldots,x_{2n}$ are generated ...
2
votes
0answers
34 views

Using Linear Regression for Classification

I am using Vowpal Wabiit to explore the power of different loss functions (e.g. squared, hinge, logistic, quantile) for classification. I've trained different models using each of these loss functions....
1
vote
2answers
2k views

What is a saturated function?

I couldn't find a definition online. I know that the sigmoid function is saturated but what does it mean.
1
vote
1answer
2k views

Logistic Regression: When can the cost function be non-convex?

I'm studying the lecture on the Machine Learning at Coursera. When introducing the cost function for Logistic Regression, they said that we shouldn't use the same cost function as the one for Linear ...
1
vote
1answer
26 views

Logistic Regression Explanation

I have two questions regarding logistic regression. 1) I understand that the results of a logistic regression model yield a table stating coefficients together with a p-statistic for each variable . ...
1
vote
1answer
26 views

I have one question about mathematical modeling

My school does Orienteering in PE, and basically the grades are from 0 to 20, the is the best,0 is the worst and 10 is 50%, well my teacher calculates the grades this way, if you do the task in 5 ...
1
vote
1answer
111 views

How is the log-likelihood for a multinomial logistic regression calculated?

In a multinomial logistic regression, the predicted probability $\pi$ of each outcome $j$ (in a total of $J$ possible outcomes) is given by: $ \pi_j = \frac{e^{A_j}}{1+\sum_{g \neq j}^Je^{A_j}} $ ...
1
vote
1answer
80 views

fix point solution or approximation available? logistic regression?

please, is there a simple closed-form or approximation to the following fixed-point problem in $x$? $x$ is the value searched for. $m$, $g$ and $N$ are real parameters, all greater than 0, \begin{...
1
vote
1answer
453 views

The derivative of logistic regression

The logistic regression is $\frac{1}{1+e^{-x}}$, and the derivative of logistic regression is $f(x)*(1-f(x))$. In the following page on Wikipedia, it shows the following equation: $$f(x) = \frac{1}{1+...
1
vote
2answers
931 views

Non-linear regression for cumulative distribution function

I have twenty probability distributions based on a simulation. The corresponding cumulative distribution plot for one distribution looks like this: Simulated result I believe that most of the ...
1
vote
1answer
88 views

What is the relationship between the logistic function and the logistic loss function?

The standard logistic function is [1]: $$ f(x) = \frac{1}{1+e^{-x}} $$ But the logistic loss function is typically defined as [2]: $$ l(w^{\top} \cdot x) = \ln(1 + e^{-y(w^{\top} \cdot x)}) $$ I ...
1
vote
1answer
124 views

Logistic Regression: Asymptotic confidence interval for the lethal dose

For the logistic model: $$\log \Big( \frac{\pi(x)}{1-\pi(x)}\Big) = b_0 +b_1x$$ I want to construct a asymptotic confidence interval for the ratio of the m.l.e's of $b_0$, $b_1$: $LD50 = -\frac{\...
1
vote
1answer
45 views

Logistic Equation through three points

I am trying to solve a logistic equation $\frac{dP}{dt}=rP\left ( 1-\frac{P}{K} \right )$ for the values of $r$ and $K$ through the points $(0,100),(10,130),(20,160)$. I have derived a general ...
1
vote
1answer
31 views

Maxvalue of Growth (nummerical) on Excel or other program

I need to solve this problem numerically: $$ n'(t)=0.12\left(1-\frac{1}{10000}\cdot n(t)\right)\cdot n(t)-x,\\ n(0)=2000. $$ I need to find the right $x$ so that $n'(t)=0$. I know that the right ...
1
vote
1answer
43 views

Logistic Regression Adjusting for True Population Proportion

Suppose that a logistic regression model is fit in order to predict which survivors of major strokes will suffer another major stroke within the next $60$ days. The single predictor $X$ is used, ...
1
vote
1answer
53 views

Layman's explanation of how we can deduce certain qualitative properties of the sigmoid function from its formula?

Is it possible to explain in layman's terms how the sigmoid formula ($y=\frac{1}{1+e^{-x}}$) defines some aspects of the function? If it is, could someone please give a layman's explanation? For ...
1
vote
1answer
1k views

Understanding partial derivative of logistic regression cost function

I'm following along in Andrew Ng's great lecture series on machine learning, and he presents the following as the cost function for a logistic regression model [link]: $$L(a,y) = -(y \log(a) + (1 - y)...
1
vote
1answer
594 views

What is the error function in multi-class classification?

I am in trouble to know what is the loss function of a neural network. For a binary classification problem, is it mean squared error, as described in the following video :https://www.youtube.com/watch?...
1
vote
1answer
343 views

why do we need regularization when there is a lot of data

I am reading the textbook Machine Learning - A Probability Perspective, and in Chapter 8 (which talks about logistic regression) there is a paragraph that says: Just as we prefer ridge regression ...
1
vote
1answer
73 views

How to solve this differential equation (Logistic derivation)

https://www.ma.utexas.edu/users/davis/375/popecol/lec5/logist.html How do I get from here: $$\frac{dN}{dt}=rN=r_0 N\left(1-\frac{N}{K}\right)$$ To here: $$N_t=\frac{N_0\cdot K}{N_0+(K-N_0)\cdot \...
1
vote
0answers
34 views

A “softmax”-like function for deciding on a partition

Softmax can be derived as follows. Say that we are given $k$ "log priors" $b_i$ that our data belongs to the $i$th category in some categorical distribution. Then we can solve for the category ...
1
vote
1answer
53 views

Derive $ \frac{1}{1 + exp(-(\beta_0 + \beta_1x))} $ from conditional and total probabilities

Related to: Show posterior probability takes the form of the logistic function I basically want to derive the sigmoid function from conditional and total probabilities. In other words, I want to ...
1
vote
0answers
25 views

Can anyone help with the inverse problem and tuning parameters

For my final year project i want to model the population of London using the Verhulst logistic model. However, to gain more marks i wish to use the inverse problem to tune the parameters to make the ...
1
vote
1answer
32 views

Finding population growth rate given initial population and average pubs per female reproduction

I am new to mathematical modeling and having trouble modelling the following population growth scenario: Starting population: $100$ individuals Average age: $7$ years Assumption: population equally ...
1
vote
0answers
24 views

How to do prediction in a binary classification with logistic regression when we care much more about type I error than type II error?

How to do prediction in a binary classification with logistic regression when we care much more about type I error than type II error? Which criteria should I use to select the threshold value and ...
1
vote
1answer
59 views

Bayesian Logistic Regression, conditional probability integration

In Andrew NG's Lectures (CS229), the Bayesian Logistic Regression section contained a formula; $$P(Y|X,S)=\int_\theta P(Y|X,\theta)P(\theta|S)d\theta$$ Here, $\theta$ is treated as a random variable....
1
vote
0answers
29 views

In logistic regression: What is the proper way to report the overall “odds ratios” for a non-linear continuous variable

I am fitting a logistic regression with multiple binary variables as well as a single continuous variable (AGE) for which I have a linear and a quadratic term. $$\log\frac P{1-P} = \text{constant} + ...