Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
0answers
5 views

Regression with many discrete and continuous predictors and few rows

I want to do regression on a dataset. It has one continuous dependent variable that I want to predict. It has many categorical and some continuous predictors. It only has a few rows. A simplified ...
1
vote
1answer
30 views

Transposing matrix when differentiating it

Hi so I am trying to understand the solution of linear regression with matrices (found at the following link) and an confused about how on page 10 he says the derivative of $2Y'XB$ with respect to $B$ ...
0
votes
0answers
8 views

Softmax Regression Gradient Derivation

I'm implementing softmax regression and am deriving the max-log-likelihood update for gradient descent by hand first. Coming from the Stanford UFLDL site, they show the gradient of the cost function ...
1
vote
0answers
16 views

LRT and Wald Test in Multivariate Linear Regression significance

I've been researching all afternoon trying to get a better idea of what the Likelihood ratio test and the Wald test are actually doing. I have a bunch of covariates and I'm testing out like 30 ...
0
votes
0answers
17 views

Finding the curvature from a set of datapoints

I have a set of 1. 1-d 2. 2-d data. I want to find the curvature at each single point. Till now I was using difference technique to find out the curvature, i.e, central difference at middle and ...
1
vote
1answer
22 views

Significance of dummy variables in probit regression

I am an undergraduate student working on some projects using probit regression. I have a question on dummy variables that I was hoping someone could help me with (which I think stems from an ...
1
vote
1answer
14 views

Find a distribution for this plot

Please help me find a formula that fits the distribution. It does not need to be exact, a simple approximation would suffice. Bonus points if you can tell me which predefined distribution in the ...
0
votes
1answer
28 views

Least squares with known error in y

so I want to do a linear least squares regression on my data, however I have known experimental error on my data points in $y$ and relatively few numbers of points so I would like to use values ...
0
votes
1answer
30 views

How do I solve for vector $P$ in the matrix equation $s=A'B^{-1}A$?

I would like to rearrange the matrix equation $s=A'B^{-1}A$ into the form $A=f(s,B)$ (i.e., some function of $s$ and $B$), where s is scalar, $A$ is $n\times 1$, $A'$ is the transpose of $A$, and $B$ ...
0
votes
1answer
22 views

finding a question about constrained regression - had a side constraint $x \geq y^*z$

I saw a question that asked about solving a non-negative least squares problem with $3$ unknowns, $(x,y,z)$. But there was an additional constraint, $x \geq y^*z$. Would appreciate getting the ...
0
votes
1answer
24 views

Getting VAR parameters from a research paper.

Many econometrics papers provide the parameters used in their VAR model. If I notate my VAR model as $$z_{t+1} = c + B z_{t} + \Sigma \epsilon_{t+1}$$ where $\epsilon \sim N(0, I)$, then I need to ...
1
vote
3answers
47 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
0
votes
1answer
8 views

In a simple regression model estimated using OLS, the covariance between the estimated errors and regressors is zero by construction

Is this statement true or false? I seem to remember that this relationship does not hold when the regression has no intercept, however my teacher said that this was true regardless of whether we ...
-1
votes
0answers
19 views

(x/y) v (y/x) as predictor in regression

I am trying to predict a variable 'score' with x and y. I believe it is related to (x/y)/z and z/(x/y), but I'm not sure which. Here is some data: ...
0
votes
1answer
25 views

Polynomial least squares fit — restrictions on order?

If we're finding an interpolating polynomial for 10 data pairs, the order of the polynomial has to be 9. In class, my professor said that when doing a polynomial least squares fit, if you have 10 ...
0
votes
2answers
75 views

Non-linear regression fit

I'm trying to fit my data to the following equation: $$ Y = A(1-2e^{bx}) $$ What I tried to do was transform the equation to a linear form via the following steps: \begin{align*} & A-Y = ...
3
votes
0answers
26 views

How to perform nonlinear regression with regressors affected by gaussian error?

I am trying to calibrate a sensor and I have a data set consisting of several observations of a 3-dimensional vector $X_i$, with $X_i=w_i + \epsilon_i$ where $w_i$ is the value that the sensor ...
1
vote
2answers
62 views

Fit exponential with constant

I have data whic would fit to an exponential function with a constant. So y=aexp(bt) + c Now I can solve an exponential without a constant using least square by taking log of y and making the ...
0
votes
1answer
11 views

Least Squares Estimators Derivation (Bi-variate)

To derive least square estimators: We have $SS(\alpha,\beta)= \sum(y_i-\alpha-bx_i)^2$ and find partials for each. The answer I get is: $\beta = \frac{\sum y_i-\bar{y}}{\sum x_i-\bar{x}}$, but the ...
1
vote
1answer
26 views

Multiple Linear Regression in Matrix Form

I am currently studying for my exams and came across the following question: State the multiple linear regression equation in matrix form. Write down the order of each matrix and explain what the ...
2
votes
0answers
21 views

Can I adjust linear growth of a a subpopulation to a linear decay of the general population?

I need to estimate the amount of CF patients in Poland in the next four years. I have: estimations of the Polish population for the future years a CF patients' register for the last couple of years ...
0
votes
2answers
24 views

Calculating the correlation coefficient between least square estimates

PROBLEM STATEMENT: Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean $0$ and variance $1$; $$y_i = α + β(x_i − \bar ...
-1
votes
0answers
16 views

Sample variance estimator OLS without intercept

In the classical OLS regression model the estimator of the variance is $s_n^2=\frac{1}{n-k}\sum_{i=1}^n \hat{e}_i$ where $k$ is the number of regressors without the constant. What happens to this ...
0
votes
1answer
24 views

Why is the SSE <= SST?

I can think of a regression line producing a larger sum of squared errors (SSE) than the total sum of squares (SST). I read that should not be possible, how come? My understanding is that the ...
2
votes
1answer
55 views

Outlier detection with robust multiple regression model

I have a set of features (eg, location, income, budget, education) that I use to predict a continuous variable (say, amount spent per day on the internet). I am interested in detecting outliers. I ...
0
votes
0answers
20 views

Transpose is just the way of generalizing a dot product?

It seems like $a^Tb$ is the same as writing $a \cdot b$ in matrix form. 1) Why is $n \times 1$ and $n \times 1$ matrix multiplication undefined? 2) Is this just a generalization of the dot ...
1
vote
1answer
12 views

Do we only use T distribution for confidence interval of Beta for linear regression or we can also normal distribution?

Do we only use T distribution for confidence interval of Beta for linear regression or we can also normal distribution? Is it that when sample size is less than 30 then we use T distribution else ...
4
votes
5answers
42 views

calculating least squares fit

I read this thread talking about 'why we use least squares' for curve fitting Why do we use a Least Squares fit? One answer by Chris Taylor begins with the assumption that we should look for $$ ...
1
vote
4answers
49 views

How to estimate $\alpha$ in $y=(1-\exp(-\alpha x))/(1+\exp(-\alpha x))$?

I have a function $$y=\dfrac{1-\exp(-\alpha x)}{1+\exp(-\alpha x)}$$ where $y$ is not binary. The range of this function is $[-1,1)$. So this does not fit into either logit or probit models. How ...
0
votes
1answer
42 views

Identical observations in linear regression

I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
0
votes
1answer
18 views

Method for ?not quite? weighted least squares fitting for more realistic results

I need a linear least squares type of fitting algorithm that understands how to weight the probability of a response coming from certain functions over another. To explain, given the standard linear ...
1
vote
1answer
15 views

Assumption of Normal Distribution

I have a problem and I do not know when it is crucial and when it is NOT crucial to assume a normal distribution regarding linear regression, for estimates, t-tests, f-tests, confidence intervals and ...
1
vote
1answer
29 views

Best percentage change for trend

Consider the revenue of a company for the last five year and you want to to know whether there is an upward, downward or no trend. How would you calculate the "optimal" percentage change? I have an ...
-2
votes
0answers
21 views

Repeated measurements for multiple groups with multiple samples

I have 4 groups with 5-6 samples in each, and I have repeated a measurement on them 6 times over a time. This is done during an aging test and the results are evolving over the time roughly linearly. ...
1
vote
1answer
28 views

Is there such a thing as a weighted multiple regression?

I'm new to linear algebra, but I know how multiple linear regressions work. What I want to do is something slightly different. As an example, let's say that I have a list of nutrients I want to get ...
1
vote
0answers
34 views

Confusion with Bayesian Linear Regression

In the book Gaussian Processes for Machine Learning in Chapter 2 p. 11 (see http://www.gaussianprocess.org/gpml/chapters/RW2.pdf), eq. 2.9 states: $p(f_* | X, y) = \int p(f_* | x_*,w) p(w|X, y)dw$ ...
2
votes
1answer
38 views

Calculate trend and represent in text?

First off, I'm terrible at math. I'm writing a script that monitors transactions from clients daily over a 7 day period. Given a set of numbers like below, I would like to calculate a trend and ...
0
votes
0answers
21 views

Relation between the Coefficient of Multiple Correlation and Coefficient of Simple Correlation

Consider the regression model $Y=\beta_1 X_1+\beta_2 X_2+\epsilon$, with a sample of size $n$, $Y_i=\beta_1 X_{i1}+\beta_2 X_{i2}+\epsilon_i$, $\epsilon_i \in N(0,\sigma^2)$. Suppossing ...
1
vote
1answer
22 views

Logistic regression coefficients problem

I'm using logistic regression model to do a multi-class classification (4 classes). I want to look at the logistic regression coefficients to see the importance of different features. I got model ...
0
votes
1answer
28 views

Distribution of e, least squares residuals

I have the model $y=X{\beta}+{\epsilon}$ and $E({\epsilon})=0$ and $Var({\epsilon})={\sigma}^2I_n$ The vector y can be written: $y=X\hat{\beta}+{e}$ If ${\epsilon}$~$N(0, {\sigma}^2I_n)$ how is ...
0
votes
1answer
26 views

Quadratic Form Matrices

How do I know if a matrix in quadratic form, e.g. D'MD is positive or negative (semi)definite? M here is the residual maker matrix for X, so I know that it is symmetric. I know what the definitions ...
2
votes
0answers
27 views

Show $\hat{\beta}$ and $s^2$ are independent?

I have the model: $y=X{\beta}+{\epsilon}$ I know $\hat{\beta}=(X'X)^{-1}X'y$ and that it is an unbiased estimator of ${\beta}$ and that $s^2=\hat{\epsilon}'\hat{\epsilon}/(n-k)$ and is an unbiased ...
0
votes
1answer
35 views

Multiple linear regression inconsistency?

I've got a linear model: $y_i=β_1x_{i1}+β_2x_{i2}+ε_i$ where E($ε_i$)=0 and Var($ε_i$)= $σ^2I_n$ for i=1,...,n Supposed we don't have the data for $x_{i2}$ and we estimate: $y_i=β_1x_{i1}+ε_i$ for ...
1
vote
0answers
36 views

Minimize correlation between input and output of a linear system

I am not sure if "minimize correlation" is the right title for this issue but I could not find a better sentence to describe what I would like to achieve. Let's say that I have a black box with ...
1
vote
1answer
24 views

I would like to know how to do log transformation of hyperparameters in Gaussian Process Classification.

I am using Gaussian Process classification and I want to do log transform of the hyperparameters so that they are all positive. From this www.lce.hut.fi/research/mm/gpstuff/GPstuffDoc.pdf document, I ...
0
votes
0answers
60 views

Solving constrained linear programming problem

For the variable $t$, problem is to find best multipliers $k$ which minimizes the objective function. Time: $t_1$, $t_2$, $t_3$,... given in input Multiplier $k_1$, $k_2$, $k_3$,... (These are ...
0
votes
0answers
16 views

If the null hypothesis is true, how will the test statistic be distributed?

I went with T~(50-6) The question goes.... "A regression is estimated with 50 observations, five explanatory variables and with a constant. Suppose You want to test the following hypothesis $H_0: ...
0
votes
1answer
25 views

If $\ln(y) = 5 - 0.1X $what is the elasticity of $Y$ with respect to $X$, when $X=10$?

So i got the following model $\ln(y) = 5 - 0.1* X$ The elasticity of Y with respect to X, when $X=10$ i said -0.1 but apparently i'm wrong Isn't the coefficient of X the elasticity of y when the ...
1
vote
1answer
14 views

Class or type variables as features in polynomial regression algrorithm

I am new in machine learning area, and trying to use polynomial regression for my problem. I have data - advertisements of the cars from kolesa.kz website. Data contains mark, model, mileage, engine ...
1
vote
1answer
12 views

Linear or Non-Linear Model

I have the following regression equation \begin{align*} y_i = \alpha + \gamma\cdot\beta\cdot x_i+ \varepsilon_i, \end{align*} where $y_i$, $x_i$ and $\varepsilon_i$ are $n\times 1$ vectors, ...