Questions tagged [logistic-regression]
For questions about logistic regressions, a regression model where the dependent variable is categorical.
230
questions
0
votes
0answers
10 views
Logistic regression loss function not zero for perfect model
The definition of the logistic regression loss function I use is this:
We draw the data i.i.d. according to some distribution $D$, realised by some $\langle X, Y \rangle$. Now if $h_w$ was the real ...
1
vote
1answer
52 views
Is logistic regression cost function in SciKit Learn different from standard derivations?
I am trying to understand the math behind logistic regression. Going through a couple of websites, lectures and books, I tried to derive the cost function by thinking of it as the negative of the ...
1
vote
0answers
16 views
Nearest neighbor classifier on a simple dataset
Consider the following data set for two classes $X_1 = \{ (0,0) \}, X_2 = \{(1,0),(0,1)\}$
I'm trying to come up with a rule to separate the data using k-nn for $k = 1$. But I don't know how to apply ...
1
vote
0answers
58 views
How does the Richards' differential equation (RDE) change, if I add the lower asymptote in the generalised logistic function?
The Question
Let's consider the Richards' differential equation (RDE) as written here below, from Wikipedia:
How does the RDE change if I add the lower asymptote "A" in the generalised ...
0
votes
0answers
47 views
Why does regularization work?
Yes, we get smaller thetas (if I understand correctly) with added regularization but I fail to see why it is good. Why is it?
0
votes
0answers
36 views
Proof of Polya Gamma Laplace Transformation
If $w$ follows a Polya-Gamma Distribution, denoted as $w\sim PG(b,0)$ with $b>0$ then
$$w\overset{D}{=}\frac{1}{2\pi^{2}}\sum_{k=1}^{\infty}\frac{g_{k}}{(k-1/2)^{2}},$$
where $g_{k}\sim\Gamma(b,1)$ ...
1
vote
4answers
63 views
Using generalized logistic curve to create a mathematical model from data.
The first row is time and the second row is height of a plant. We need to use generalized logistic curve to model the behavior of the plant. The equation of the logistic curve is :
$$N = \frac{N_*}{1+(...
0
votes
1answer
19 views
Linear and logistic regression
Let a sample $(x,y) \in \mathbb{R}^{2n}$ be given, where $y$ only attains the values $0$ and $1$. We can try to model this data set by either linear regression
$$y_i = \alpha_0 + \beta_0 x_i$$
with ...
1
vote
0answers
106 views
Logistic Regression | Exercise
I am trying to solve questions 2 to 7 of this exercise exercise sheet
At the moment, I don't know how to answer these questions.
But I have some ideas for the second question :
Do I have to study ...
0
votes
0answers
36 views
Gradient descent vs Newton's method
rn im a little confused with gradient descent because it looks pretty similar to Newton's method but it does alpha*the derivative, instead of something like alpha/derivative which means that its ...
0
votes
0answers
19 views
How to add logistic term to numeric series?
I have found a formula to calculate the number of items y after a certain value x (e.g. time) given the size of items at the beginning of the series:
$$
y_x = a^x y_0
$$
I would like to convert it ...
1
vote
4answers
68 views
Sigmoid function with a longer, straighter middle
Can I modify the following function $y = 1 / (1 + e^{-x})$ such that I can make the curve straighter in the middle?
I would like the function to resemble a straight line until it approaches $0$ or $1$....
0
votes
0answers
7 views
Logistic regression problems with $ K $-class derivation
Derive the update equation for $ 2 $-class logistic regression with $ L _ 2 $ regularization.
Derive the update equation for $ K $-class logistic regression.
Derive the update equation for $ K $-class ...
1
vote
0answers
28 views
Multiclass logistic regression gradient descent update rule for weights using softmax/categorical cross entropy?
I'm trying to find the general update rule (for gradient descent) for multiclass logistic regression for all weights.
Say the logistic regression model has 3072 inputs, and 10 classes. That would make ...
0
votes
0answers
56 views
Ordinal logistic regression
I'm working on a project that needs to be done in databricks. So that means working with pyspark. I'm working with ordinal data and so require ordinal logistic regression. How do I go about doing this ...
0
votes
0answers
16 views
Finding the asymptotic distribution of Cook's Distance for a simple binary logistic regression problem
I recently encountered a problem in logistic regression about finding the asymptotic distribution of Cook's Distance for a simple binary logistic regression problem.However, I could not find enough ...
1
vote
1answer
32 views
Independent and Identically distributed, conditional independent and Naive bayes
I'm reading about Naive Bayes classification concept, noting that we make the conditionally independence assumption. But isn't this the general assumption that is always made dealing with machine ...
0
votes
0answers
34 views
Can output from logistic regression be interpreted as probabilites for individual estimates?
When doing logistic regression, the derivative of the likelihood $L$ with respect to parameters $\theta$ is
$\frac{\partial L(\theta)}{\partial \theta_j} = \sum_{i=0}^n \big[y^{i} - \sigma(\theta^Tx^{...
0
votes
0answers
23 views
Test Error of Binary Logit Model
Consider the logistic regression model, where $X\sim \mathcal{N}(0,\Sigma)$ and unkown parameter $\theta\in \mathbb{R}^d$. Furthermore for the random variable $Y\in \{-1,1\}$ we have
$$P(Y=y|X,\theta)=...
0
votes
0answers
13 views
odds ratio logistic regression
Let us suppose that a 2 x 2 factorial with variable coded to Ā± 1 is used to
fit a logistic regression model in a drug study in which 20 subjects were
allocated to each of the 4 treatment combinations. ...
0
votes
0answers
39 views
Can anyone explain how the operator $\oplus$ is defined here?
Can anyone explain how the operator $\oplus$ is defined here? It seems it is a cumulative operator, but I have never encountered it before.
0
votes
2answers
17 views
Logistic Regression - Converting Equations
I am learning logistic regression by watching this video.
How I can reduce eqn?
$$p = \frac{e^{\beta_0 + \beta_1 * age}}{e^{\beta_0 + \beta_1 * age} + 1} $$
to
$$\log_e \frac{p}{1-p} = \beta_0 + \...
0
votes
0answers
29 views
Interaction between dummy variables in Linear Regression
I know as a fact that dummy variables always reflect the deviation from the reference category of the original variable after controlling for relevant other variables.
Unfortunately I don't understand ...
0
votes
0answers
10 views
Questions on Logistic Regression.
I have a few questions about Logistic Regressions.
we can get a probability that certain 'y' happen in the situation of certain 'X'.
And, Is there any error range of that probability?
And Is there ...
0
votes
0answers
94 views
Logistic Regression closed form solution when using binary cross-entropy
Let's say that I want to find the stationary points of the Cross-Entropy Loss function when using a logistic regression
The 1 D logistc function is given by :
\begin{equation}\label{eq2}
\begin{split}
...
0
votes
0answers
35 views
How to treat the Sigmoid function when proving the gradient of regularized logistic regression is Lipschits?
I have come across a question where I need to prove the regularized logistic regression has Lipschits continuous gradient. I have derived the gradient as:
$$
\nabla f(\theta) = \sum_{i=1}^na_i (\frac{...
0
votes
0answers
26 views
How should I proceed and treat the Sigmoid in proving the Lipschitz condition for regularized logistic regression?
I have come across this question to prove the gradient of a regularized logistic regression is lipschitz continuous. I have derive the gradient, I can easily see the proof for linear regression ...
0
votes
0answers
41 views
Newton Raphson Method for Probit Link in Logistic regression model
I have an assignment in which there is a question of deriving the likelihood equations of coefficients of logistic regression model using probit link. Then I need to derive second derivatives. Then I ...
0
votes
0answers
52 views
Prove Convexity Multinomial Logistic Loss
I have been asked to prove the multinomial logistic loss is convex with respect to the model parameters.
I have managed to compute the first gradient:
$$\nabla_W-\log[softmax(z)]_j = -(\phi(x_j) - \...
0
votes
0answers
14 views
Binary regression Model with Binary Covariate
I have two single covariate binary logistic models. I am given a datset where the response variable y takes 0 and 1. The covariate x also takes 0 and 1 values.
model 1:
$P[Y=1|X=x]=\beta_0 + \beta_1x$
...
2
votes
0answers
30 views
Cross-entropy loss and stationary points
I am trying to find the stationary points of the cross-entropy function for binary classification :
$$
L(w) = -y \cdot \log(\sigma(wx)) - (1-y) \cdot \log (1-\sigma(wx))
$$
with
$$
\sigma(wx) = \frac{...
1
vote
0answers
54 views
Generalization error in logistic regression
Consider the logistic regression model, where the input data is distributed as $X\sim N(0,\Sigma)$ and the labels $Y\in \{-1,1\}$ have the the following conditional distribution: $$P(Y=1|X,\theta^*)=\...
0
votes
1answer
19 views
Which type of regression should i use for output having 3-4 states?
If output has 2 states, we can use logistics regression. But which type of regression to use when there are 3-4 output states?
0
votes
0answers
36 views
Variance of the sigmoid function of a Gaussian variable
Consider a Gaussian variable $x$ with mean $\mu$ and variance $\sigma^2$. I've found in https://arxiv.org/pdf/1703.00091.pdf the following way to derive an analytical approximation to the variance of $...
1
vote
0answers
37 views
Expectation of a product between a random variable and the logistic function
Consider a Gaussian variable $x$ with mean $\mu$ and variance $\sigma^2$, can I calculate or approximate $\mathbb{E}\!\left[x\, \phi(x)\right]$, such that $\phi$ is the sigmoid function given by $\phi(...
1
vote
1answer
34 views
Deep Learning Log Loss Function Analysis
Given the set of parameters of a logistic regression model, and a small set of data points, calculate the j^{th} partial derivative of the log-loss function for some j.
What kind of data points could ...
0
votes
0answers
33 views
Type 1 error and Confusion Matrix
Type 1 error occurs when you reject a true null hypothesis, Type 1 error are also called False Positives.Ā
False Positive implies that we are wrongly predicted a negative as positive, How does it ...
0
votes
0answers
13 views
Using Ordinal (Star Rating) variables to predict outcomes in Log lin regressions + Taking Median significant coefficients of multiple regressions
Framing the regression I am attempting to analyze the effects of several variables on clicks for Google My Business listings. Currently I'm using a Log-Lin regression model to predict the % increase ...
0
votes
1answer
49 views
Is Wilks' Theorem (the LRT is asymptotically chi-squared-distributed) not applicable to logistic regression because of no absolute continuity?
Wilks' Theorem is given in the source below as Theorem 12.4.2, p. 515. It is part of the chapter "Quadratic Mean Differentiable Families" where parametric families $\{P_{\theta}, \theta \in \...
1
vote
1answer
65 views
What to do if partial integration yields $\infty-\infty$?
I have an integral where partial integration yields
$$\begin{aligned}\int_{-\infty}^{\infty}\underbrace{f(x)}_{=u}~\underbrace{\log\left(1+\exp\left(x\right)\right)}_{=V}~\mathrm{d}x
&= \left[UV\...
1
vote
2answers
52 views
Suppose a pair of random variable is independent from another pair, does it mean that each random variable is independent from the other?
Let $(X_1, Y_1)$, and $(X_2, Y_2)$ be two pairs of random variables, and they are assumed to be independent.
Does it mean that:
$X_1$ is independent from $X_2$?
$X_1$ is independent from $Y_2$?
$Y_1$ ...
0
votes
0answers
67 views
What is the probability prediction given only proportional data?
I'm trying to use a simple technique of predicting if someone has a disease given the statistics of the country. I know that Logistic Regression for example is a good model but that only works for ...
1
vote
1answer
73 views
Fenchel conjugate of $\| \cdot \|_1$ and dual of logistic regression
I am trying to replicate some results from
Koh, K., Kim, S. J., & Boyd, S. (2007). An interior-point method for large-scale l1-regularized logistic regression. Journal of Machine learning research,...
1
vote
0answers
20 views
Derive the decision rule for a logistic regression function with a Gaussian kernel.
I am trying to derive a decision rule for a logistic regression function that has been localized with a Gaussian kernel. I know that I need to have the probability at one side $P(Y=1\mid X)$ or some ...
0
votes
0answers
71 views
Likelihood ratio tests and pseudo R2 for four-parameter logistic regression model (dose response)
I am using a four-parameter log logistic function to fit curves to dose response data. The underlying equation I use is the following:
$$f(x)= c+ \frac{d-c}{1+ ((log(x)-log(e))^b}$$
Where b is the ...
0
votes
0answers
15 views
Options Market: Estimating Actual Probabilities Using Conditional 2-Step Logit Approach
I am working on a problem that requires the use of both fundamental variables and market-generated variables to determine the probability of a stock option finishing in-the-money by the expiration ...
0
votes
0answers
11 views
Analyzing binomial outcome data with a binomial predictor, accounting for differences in probability per sample (preferably in R)
I'm not sure how to go about describing this kind of data, but here goes.
I am collecting data that has a binomial outcome/DV (either something occurs or doesn't)
The predictor/IV I'm using also ...
0
votes
1answer
31 views
how to find an equation representing a decision boundary in logistic regression
I'm new to machine learning and currently working on logistic regression. but i don't know how to deal this problem. let us consider the logistic regression for a dataset $(x_n,y_n)\ (x_i \in \mathbb ...
0
votes
0answers
18 views
How to interpret Log linear model output?
So for example, let's say I have a Log-linear model with 3 variables: income, age, and health level; where the saturated model is all the individual effects with all the two-way interactions and the 3-...
0
votes
0answers
11 views
Is there any other way to conduct ROC on multiclass logistic regression model?
Suppose I want to conduct a Receiver Operating Characteristics Analysis on multi-class logistic regression model. Suppose there are three classes $X,Y,Z$. One of the ways is proceed by conducting ROC ...
