Questions tagged [logistic-regression]

For questions about logistic regressions, a regression model where the dependent variable is categorical.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

Logistic regression loss function not zero for perfect model

The definition of the logistic regression loss function I use is this: We draw the data i.i.d. according to some distribution $D$, realised by some $\langle X, Y \rangle$. Now if $h_w$ was the real ...
1
vote
1answer
52 views

Is logistic regression cost function in SciKit Learn different from standard derivations?

I am trying to understand the math behind logistic regression. Going through a couple of websites, lectures and books, I tried to derive the cost function by thinking of it as the negative of the ...
1
vote
0answers
16 views

Nearest neighbor classifier on a simple dataset

Consider the following data set for two classes $X_1 = \{ (0,0) \}, X_2 = \{(1,0),(0,1)\}$ I'm trying to come up with a rule to separate the data using k-nn for $k = 1$. But I don't know how to apply ...
1
vote
0answers
58 views

How does the Richards' differential equation (RDE) change, if I add the lower asymptote in the generalised logistic function?

The Question Let's consider the Richards' differential equation (RDE) as written here below, from Wikipedia: How does the RDE change if I add the lower asymptote "A" in the generalised ...
0
votes
0answers
47 views

Why does regularization work?

Yes, we get smaller thetas (if I understand correctly) with added regularization but I fail to see why it is good. Why is it?
0
votes
0answers
36 views

Proof of Polya Gamma Laplace Transformation

If $w$ follows a Polya-Gamma Distribution, denoted as $w\sim PG(b,0)$ with $b>0$ then $$w\overset{D}{=}\frac{1}{2\pi^{2}}\sum_{k=1}^{\infty}\frac{g_{k}}{(k-1/2)^{2}},$$ where $g_{k}\sim\Gamma(b,1)$ ...
1
vote
4answers
63 views

Using generalized logistic curve to create a mathematical model from data.

The first row is time and the second row is height of a plant. We need to use generalized logistic curve to model the behavior of the plant. The equation of the logistic curve is : $$N = \frac{N_*}{1+(...
0
votes
1answer
19 views

Linear and logistic regression

Let a sample $(x,y) \in \mathbb{R}^{2n}$ be given, where $y$ only attains the values $0$ and $1$. We can try to model this data set by either linear regression $$y_i = \alpha_0 + \beta_0 x_i$$ with ...
1
vote
0answers
106 views

Logistic Regression | Exercise

I am trying to solve questions 2 to 7 of this exercise exercise sheet At the moment, I don't know how to answer these questions. But I have some ideas for the second question : Do I have to study ...
0
votes
0answers
36 views

Gradient descent vs Newton's method

rn im a little confused with gradient descent because it looks pretty similar to Newton's method but it does alpha*the derivative, instead of something like alpha/derivative which means that its ...
0
votes
0answers
19 views

How to add logistic term to numeric series?

I have found a formula to calculate the number of items y after a certain value x (e.g. time) given the size of items at the beginning of the series: $$ y_x = a^x y_0 $$ I would like to convert it ...
1
vote
4answers
68 views

Sigmoid function with a longer, straighter middle

Can I modify the following function $y = 1 / (1 + e^{-x})$ such that I can make the curve straighter in the middle? I would like the function to resemble a straight line until it approaches $0$ or $1$....
0
votes
0answers
7 views

Logistic regression problems with $ K $-class derivation

Derive the update equation for $ 2 $-class logistic regression with $ L _ 2 $ regularization. Derive the update equation for $ K $-class logistic regression. Derive the update equation for $ K $-class ...
1
vote
0answers
28 views

Multiclass logistic regression gradient descent update rule for weights using softmax/categorical cross entropy?

I'm trying to find the general update rule (for gradient descent) for multiclass logistic regression for all weights. Say the logistic regression model has 3072 inputs, and 10 classes. That would make ...
0
votes
0answers
56 views

Ordinal logistic regression

I'm working on a project that needs to be done in databricks. So that means working with pyspark. I'm working with ordinal data and so require ordinal logistic regression. How do I go about doing this ...
0
votes
0answers
16 views

Finding the asymptotic distribution of Cook's Distance for a simple binary logistic regression problem

I recently encountered a problem in logistic regression about finding the asymptotic distribution of Cook's Distance for a simple binary logistic regression problem.However, I could not find enough ...
1
vote
1answer
32 views

Independent and Identically distributed, conditional independent and Naive bayes

I'm reading about Naive Bayes classification concept, noting that we make the conditionally independence assumption. But isn't this the general assumption that is always made dealing with machine ...
0
votes
0answers
34 views

Can output from logistic regression be interpreted as probabilites for individual estimates?

When doing logistic regression, the derivative of the likelihood $L$ with respect to parameters $\theta$ is $\frac{\partial L(\theta)}{\partial \theta_j} = \sum_{i=0}^n \big[y^{i} - \sigma(\theta^Tx^{...
0
votes
0answers
23 views

Test Error of Binary Logit Model

Consider the logistic regression model, where $X\sim \mathcal{N}(0,\Sigma)$ and unkown parameter $\theta\in \mathbb{R}^d$. Furthermore for the random variable $Y\in \{-1,1\}$ we have $$P(Y=y|X,\theta)=...
0
votes
0answers
13 views

odds ratio logistic regression

Let us suppose that a 2 x 2 factorial with variable coded to ± 1 is used to fit a logistic regression model in a drug study in which 20 subjects were allocated to each of the 4 treatment combinations. ...
0
votes
0answers
39 views

Can anyone explain how the operator $\oplus$ is defined here?

Can anyone explain how the operator $\oplus$ is defined here? It seems it is a cumulative operator, but I have never encountered it before.
0
votes
2answers
17 views

Logistic Regression - Converting Equations

I am learning logistic regression by watching this video. How I can reduce eqn? $$p = \frac{e^{\beta_0 + \beta_1 * age}}{e^{\beta_0 + \beta_1 * age} + 1} $$ to $$\log_e \frac{p}{1-p} = \beta_0 + \...
0
votes
0answers
29 views

Interaction between dummy variables in Linear Regression

I know as a fact that dummy variables always reflect the deviation from the reference category of the original variable after controlling for relevant other variables. Unfortunately I don't understand ...
0
votes
0answers
10 views

Questions on Logistic Regression.

I have a few questions about Logistic Regressions. we can get a probability that certain 'y' happen in the situation of certain 'X'. And, Is there any error range of that probability? And Is there ...
0
votes
0answers
94 views

Logistic Regression closed form solution when using binary cross-entropy

Let's say that I want to find the stationary points of the Cross-Entropy Loss function when using a logistic regression The 1 D logistc function is given by : \begin{equation}\label{eq2} \begin{split} ...
0
votes
0answers
35 views

How to treat the Sigmoid function when proving the gradient of regularized logistic regression is Lipschits?

I have come across a question where I need to prove the regularized logistic regression has Lipschits continuous gradient. I have derived the gradient as: $$ \nabla f(\theta) = \sum_{i=1}^na_i (\frac{...
0
votes
0answers
26 views

How should I proceed and treat the Sigmoid in proving the Lipschitz condition for regularized logistic regression?

I have come across this question to prove the gradient of a regularized logistic regression is lipschitz continuous. I have derive the gradient, I can easily see the proof for linear regression ...
0
votes
0answers
41 views

Newton Raphson Method for Probit Link in Logistic regression model

I have an assignment in which there is a question of deriving the likelihood equations of coefficients of logistic regression model using probit link. Then I need to derive second derivatives. Then I ...
0
votes
0answers
52 views

Prove Convexity Multinomial Logistic Loss

I have been asked to prove the multinomial logistic loss is convex with respect to the model parameters. I have managed to compute the first gradient: $$\nabla_W-\log[softmax(z)]_j = -(\phi(x_j) - \...
0
votes
0answers
14 views

Binary regression Model with Binary Covariate

I have two single covariate binary logistic models. I am given a datset where the response variable y takes 0 and 1. The covariate x also takes 0 and 1 values. model 1: $P[Y=1|X=x]=\beta_0 + \beta_1x$ ...
2
votes
0answers
30 views

Cross-entropy loss and stationary points

I am trying to find the stationary points of the cross-entropy function for binary classification : $$ L(w) = -y \cdot \log(\sigma(wx)) - (1-y) \cdot \log (1-\sigma(wx)) $$ with $$ \sigma(wx) = \frac{...
1
vote
0answers
54 views

Generalization error in logistic regression

Consider the logistic regression model, where the input data is distributed as $X\sim N(0,\Sigma)$ and the labels $Y\in \{-1,1\}$ have the the following conditional distribution: $$P(Y=1|X,\theta^*)=\...
0
votes
1answer
19 views

Which type of regression should i use for output having 3-4 states?

If output has 2 states, we can use logistics regression. But which type of regression to use when there are 3-4 output states?
0
votes
0answers
36 views

Variance of the sigmoid function of a Gaussian variable

Consider a Gaussian variable $x$ with mean $\mu$ and variance $\sigma^2$. I've found in https://arxiv.org/pdf/1703.00091.pdf the following way to derive an analytical approximation to the variance of $...
1
vote
0answers
37 views

Expectation of a product between a random variable and the logistic function

Consider a Gaussian variable $x$ with mean $\mu$ and variance $\sigma^2$, can I calculate or approximate $\mathbb{E}\!\left[x\, \phi(x)\right]$, such that $\phi$ is the sigmoid function given by $\phi(...
1
vote
1answer
34 views

Deep Learning Log Loss Function Analysis

Given the set of parameters of a logistic regression model, and a small set of data points, calculate the j^{th} partial derivative of the log-loss function for some j. What kind of data points could ...
0
votes
0answers
33 views

Type 1 error and Confusion Matrix

Type 1 error occurs when you reject a true null hypothesis, Type 1 error are also called False Positives.  False Positive implies that we are wrongly predicted a negative as positive, How does it ...
0
votes
0answers
13 views

Using Ordinal (Star Rating) variables to predict outcomes in Log lin regressions + Taking Median significant coefficients of multiple regressions

Framing the regression I am attempting to analyze the effects of several variables on clicks for Google My Business listings. Currently I'm using a Log-Lin regression model to predict the % increase ...
0
votes
1answer
49 views

Is Wilks' Theorem (the LRT is asymptotically chi-squared-distributed) not applicable to logistic regression because of no absolute continuity?

Wilks' Theorem is given in the source below as Theorem 12.4.2, p. 515. It is part of the chapter "Quadratic Mean Differentiable Families" where parametric families $\{P_{\theta}, \theta \in \...
1
vote
1answer
65 views

What to do if partial integration yields $\infty-\infty$?

I have an integral where partial integration yields $$\begin{aligned}\int_{-\infty}^{\infty}\underbrace{f(x)}_{=u}~\underbrace{\log\left(1+\exp\left(x\right)\right)}_{=V}~\mathrm{d}x &= \left[UV\...
1
vote
2answers
52 views

Suppose a pair of random variable is independent from another pair, does it mean that each random variable is independent from the other?

Let $(X_1, Y_1)$, and $(X_2, Y_2)$ be two pairs of random variables, and they are assumed to be independent. Does it mean that: $X_1$ is independent from $X_2$? $X_1$ is independent from $Y_2$? $Y_1$ ...
0
votes
0answers
67 views

What is the probability prediction given only proportional data?

I'm trying to use a simple technique of predicting if someone has a disease given the statistics of the country. I know that Logistic Regression for example is a good model but that only works for ...
1
vote
1answer
73 views

Fenchel conjugate of $\| \cdot \|_1$ and dual of logistic regression

I am trying to replicate some results from Koh, K., Kim, S. J., & Boyd, S. (2007). An interior-point method for large-scale l1-regularized logistic regression. Journal of Machine learning research,...
1
vote
0answers
20 views

Derive the decision rule for a logistic regression function with a Gaussian kernel.

I am trying to derive a decision rule for a logistic regression function that has been localized with a Gaussian kernel. I know that I need to have the probability at one side $P(Y=1\mid X)$ or some ...
0
votes
0answers
71 views

Likelihood ratio tests and pseudo R2 for four-parameter logistic regression model (dose response)

I am using a four-parameter log logistic function to fit curves to dose response data. The underlying equation I use is the following: $$f(x)= c+ \frac{d-c}{1+ ((log(x)-log(e))^b}$$ Where b is the ...
0
votes
0answers
15 views

Options Market: Estimating Actual Probabilities Using Conditional 2-Step Logit Approach

I am working on a problem that requires the use of both fundamental variables and market-generated variables to determine the probability of a stock option finishing in-the-money by the expiration ...
0
votes
0answers
11 views

Analyzing binomial outcome data with a binomial predictor, accounting for differences in probability per sample (preferably in R)

I'm not sure how to go about describing this kind of data, but here goes. I am collecting data that has a binomial outcome/DV (either something occurs or doesn't) The predictor/IV I'm using also ...
0
votes
1answer
31 views

how to find an equation representing a decision boundary in logistic regression

I'm new to machine learning and currently working on logistic regression. but i don't know how to deal this problem. let us consider the logistic regression for a dataset $(x_n,y_n)\ (x_i \in \mathbb ...
0
votes
0answers
18 views

How to interpret Log linear model output?

So for example, let's say I have a Log-linear model with 3 variables: income, age, and health level; where the saturated model is all the individual effects with all the two-way interactions and the 3-...
0
votes
0answers
11 views

Is there any other way to conduct ROC on multiclass logistic regression model?

Suppose I want to conduct a Receiver Operating Characteristics Analysis on multi-class logistic regression model. Suppose there are three classes $X,Y,Z$. One of the ways is proceed by conducting ROC ...

1
2 3 4 5