Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
19 views

Least squares with known error in y

so I want to do a linear least squares regression on my data, however I have known experimental error on my data points in $y$ and relatively few numbers of points so I would like to use values ...
0
votes
1answer
25 views

How do I solve for vector $P$ in the matrix equation $s=A'B^{-1}A$?

I would like to rearrange the matrix equation $s=A'B^{-1}A$ into the form $A=f(s,B)$ (i.e., some function of $s$ and $B$), where s is scalar, $A$ is $n\times 1$, $A'$ is the transpose of $A$, and $B$ ...
0
votes
0answers
15 views

finding a question about constrained regression - had a side constraint $x \geq y^*z$

I saw a question that asked about solving a non-negative least squares problem with $3$ unknowns, $(x,y,z)$. But there was an additional constraint, $x \geq y^*z$. Would appreciate getting the ...
0
votes
0answers
20 views

Getting VAR parameters from a research paper.

Many econometrics papers provide the parameters used in their VAR model. If I notate my VAR model as $$z_{t+1} = c + B z_{t} + \Sigma \epsilon_{t+1}$$ where $\epsilon \sim N(0, I)$, then I need to ...
1
vote
3answers
43 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
0
votes
1answer
8 views

In a simple regression model estimated using OLS, the covariance between the estimated errors and regressors is zero by construction

Is this statement true or false? I seem to remember that this relationship does not hold when the regression has no intercept, however my teacher said that this was true regardless of whether we ...
-1
votes
0answers
19 views

(x/y) v (y/x) as predictor in regression

I am trying to predict a variable 'score' with x and y. I believe it is related to (x/y)/z and z/(x/y), but I'm not sure which. Here is some data: ...
0
votes
1answer
23 views

Polynomial least squares fit — restrictions on order?

If we're finding an interpolating polynomial for 10 data pairs, the order of the polynomial has to be 9. In class, my professor said that when doing a polynomial least squares fit, if you have 10 ...
0
votes
2answers
64 views

Non-linear regression fit

I'm trying to fit my data to the following equation: $$ Y = A(1-2e^{bx}) $$ What I tried to do was transform the equation to a linear form via the following steps: \begin{align*} & A-Y = ...
3
votes
0answers
26 views

How to perform nonlinear regression with regressors affected by gaussian error?

I am trying to calibrate a sensor and I have a data set consisting of several observations of a 3-dimensional vector $X_i$, with $X_i=w_i + \epsilon_i$ where $w_i$ is the value that the sensor ...
1
vote
2answers
62 views

Fit exponential with constant

I have data whic would fit to an exponential function with a constant. So y=aexp(bt) + c Now I can solve an exponential without a constant using least square by taking log of y and making the ...
0
votes
1answer
11 views

Least Squares Estimators Derivation (Bi-variate)

To derive least square estimators: We have $SS(\alpha,\beta)= \sum(y_i-\alpha-bx_i)^2$ and find partials for each. The answer I get is: $\beta = \frac{\sum y_i-\bar{y}}{\sum x_i-\bar{x}}$, but the ...
1
vote
1answer
26 views

Multiple Linear Regression in Matrix Form

I am currently studying for my exams and came across the following question: State the multiple linear regression equation in matrix form. Write down the order of each matrix and explain what the ...
2
votes
0answers
18 views

Can I adjust linear growth of a a subpopulation to a linear decay of the general population?

I need to estimate the amount of CF patients in Poland in the next four years. I have: estimations of the Polish population for the future years a CF patients' register for the last couple of years ...
0
votes
2answers
24 views

Calculating the correlation coefficient between least square estimates

PROBLEM STATEMENT: Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean $0$ and variance $1$; $$y_i = α + β(x_i − \bar ...
-1
votes
0answers
14 views

Sample variance estimator OLS without intercept

In the classical OLS regression model the estimator of the variance is $s_n^2=\frac{1}{n-k}\sum_{i=1}^n \hat{e}_i$ where $k$ is the number of regressors without the constant. What happens to this ...
0
votes
1answer
24 views

Why is the SSE <= SST?

I can think of a regression line producing a larger sum of squared errors (SSE) than the total sum of squares (SST). I read that should not be possible, how come? My understanding is that the ...
2
votes
1answer
55 views

Outlier detection with robust multiple regression model

I have a set of features (eg, location, income, budget, education) that I use to predict a continuous variable (say, amount spent per day on the internet). I am interested in detecting outliers. I ...
0
votes
0answers
20 views

Transpose is just the way of generalizing a dot product?

It seems like $a^Tb$ is the same as writing $a \cdot b$ in matrix form. 1) Why is $n \times 1$ and $n \times 1$ matrix multiplication undefined? 2) Is this just a generalization of the dot ...
1
vote
1answer
12 views

Do we only use T distribution for confidence interval of Beta for linear regression or we can also normal distribution?

Do we only use T distribution for confidence interval of Beta for linear regression or we can also normal distribution? Is it that when sample size is less than 30 then we use T distribution else ...
4
votes
5answers
42 views

calculating least squares fit

I read this thread talking about 'why we use least squares' for curve fitting Why do we use a Least Squares fit? One answer by Chris Taylor begins with the assumption that we should look for $$ ...
1
vote
4answers
49 views

How to estimate $\alpha$ in $y=(1-\exp(-\alpha x))/(1+\exp(-\alpha x))$?

I have a function $$y=\dfrac{1-\exp(-\alpha x)}{1+\exp(-\alpha x)}$$ where $y$ is not binary. The range of this function is $[-1,1)$. So this does not fit into either logit or probit models. How ...
0
votes
1answer
42 views

Identical observations in linear regression

I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
0
votes
1answer
17 views

Method for ?not quite? weighted least squares fitting for more realistic results

I need a linear least squares type of fitting algorithm that understands how to weight the probability of a response coming from certain functions over another. To explain, given the standard linear ...
1
vote
1answer
15 views

Assumption of Normal Distribution

I have a problem and I do not know when it is crucial and when it is NOT crucial to assume a normal distribution regarding linear regression, for estimates, t-tests, f-tests, confidence intervals and ...
1
vote
1answer
27 views

Best percentage change for trend

Consider the revenue of a company for the last five year and you want to to know whether there is an upward, downward or no trend. How would you calculate the "optimal" percentage change? I have an ...
-2
votes
0answers
19 views

Repeated measurements for multiple groups with multiple samples

I have 4 groups with 5-6 samples in each, and I have repeated a measurement on them 6 times over a time. This is done during an aging test and the results are evolving over the time roughly linearly. ...
1
vote
1answer
28 views

Is there such a thing as a weighted multiple regression?

I'm new to linear algebra, but I know how multiple linear regressions work. What I want to do is something slightly different. As an example, let's say that I have a list of nutrients I want to get ...
1
vote
0answers
34 views

Confusion with Bayesian Linear Regression

In the book Gaussian Processes for Machine Learning in Chapter 2 p. 11 (see http://www.gaussianprocess.org/gpml/chapters/RW2.pdf), eq. 2.9 states: $p(f_* | X, y) = \int p(f_* | x_*,w) p(w|X, y)dw$ ...
2
votes
1answer
38 views

Calculate trend and represent in text?

First off, I'm terrible at math. I'm writing a script that monitors transactions from clients daily over a 7 day period. Given a set of numbers like below, I would like to calculate a trend and ...
0
votes
0answers
21 views

Relation between the Coefficient of Multiple Correlation and Coefficient of Simple Correlation

Consider the regression model $Y=\beta_1 X_1+\beta_2 X_2+\epsilon$, with a sample of size $n$, $Y_i=\beta_1 X_{i1}+\beta_2 X_{i2}+\epsilon_i$, $\epsilon_i \in N(0,\sigma^2)$. Suppossing ...
1
vote
1answer
21 views

Logistic regression coefficients problem

I'm using logistic regression model to do a multi-class classification (4 classes). I want to look at the logistic regression coefficients to see the importance of different features. I got model ...
0
votes
1answer
28 views

Distribution of e, least squares residuals

I have the model $y=X{\beta}+{\epsilon}$ and $E({\epsilon})=0$ and $Var({\epsilon})={\sigma}^2I_n$ The vector y can be written: $y=X\hat{\beta}+{e}$ If ${\epsilon}$~$N(0, {\sigma}^2I_n)$ how is ...
0
votes
1answer
26 views

Quadratic Form Matrices

How do I know if a matrix in quadratic form, e.g. D'MD is positive or negative (semi)definite? M here is the residual maker matrix for X, so I know that it is symmetric. I know what the definitions ...
2
votes
0answers
27 views

Show $\hat{\beta}$ and $s^2$ are independent?

I have the model: $y=X{\beta}+{\epsilon}$ I know $\hat{\beta}=(X'X)^{-1}X'y$ and that it is an unbiased estimator of ${\beta}$ and that $s^2=\hat{\epsilon}'\hat{\epsilon}/(n-k)$ and is an unbiased ...
0
votes
1answer
34 views

Multiple linear regression inconsistency?

I've got a linear model: $y_i=β_1x_{i1}+β_2x_{i2}+ε_i$ where E($ε_i$)=0 and Var($ε_i$)= $σ^2I_n$ for i=1,...,n Supposed we don't have the data for $x_{i2}$ and we estimate: $y_i=β_1x_{i1}+ε_i$ for ...
1
vote
0answers
34 views

Minimize correlation between input and output of a linear system

I am not sure if "minimize correlation" is the right title for this issue but I could not find a better sentence to describe what I would like to achieve. Let's say that I have a black box with ...
1
vote
1answer
22 views

I would like to know how to do log transformation of hyperparameters in Gaussian Process Classification.

I am using Gaussian Process classification and I want to do log transform of the hyperparameters so that they are all positive. From this www.lce.hut.fi/research/mm/gpstuff/GPstuffDoc.pdf document, I ...
0
votes
0answers
59 views

Solving constrained linear programming problem

For the variable $t$, problem is to find best multipliers $k$ which minimizes the objective function. Time: $t_1$, $t_2$, $t_3$,... given in input Multiplier $k_1$, $k_2$, $k_3$,... (These are ...
0
votes
0answers
16 views

If the null hypothesis is true, how will the test statistic be distributed?

I went with T~(50-6) The question goes.... "A regression is estimated with 50 observations, five explanatory variables and with a constant. Suppose You want to test the following hypothesis $H_0: ...
0
votes
1answer
25 views

If $\ln(y) = 5 - 0.1X $what is the elasticity of $Y$ with respect to $X$, when $X=10$?

So i got the following model $\ln(y) = 5 - 0.1* X$ The elasticity of Y with respect to X, when $X=10$ i said -0.1 but apparently i'm wrong Isn't the coefficient of X the elasticity of y when the ...
1
vote
1answer
14 views

Class or type variables as features in polynomial regression algrorithm

I am new in machine learning area, and trying to use polynomial regression for my problem. I have data - advertisements of the cars from kolesa.kz website. Data contains mark, model, mileage, engine ...
1
vote
1answer
12 views

Linear or Non-Linear Model

I have the following regression equation \begin{align*} y_i = \alpha + \gamma\cdot\beta\cdot x_i+ \varepsilon_i, \end{align*} where $y_i$, $x_i$ and $\varepsilon_i$ are $n\times 1$ vectors, ...
0
votes
0answers
7 views

Where can I find formulas for the multiclass logistic regression with bias term?

In most of the books and web sources and papers the multiclass logistic regression is introduced and discussed without bias terms. I am looking for generalised formulas using bias terms. The standard ...
0
votes
1answer
43 views

Confidence Interval for Nonlinear Regression using F-Test - lmfit

I am trying to understand the implementation for the lmfit confidence interval calculation - in the docs it is stated: "The F-test is used to compare our null model, which is the best fit we have ...
1
vote
1answer
24 views

Gauss-Markov Theorem: How can you show that $\Lambda^T (X^TX)^gX^TX(X^TX)^{g^T} \Lambda$ = $\Lambda^T(X^TX)^g\Lambda$?

I'm stuck on how to go from the first line to the second line in this equation related to the Gauss-Markov model where $\mathbf{y}=X\mathbf{b}+\mathbf{e}$, $E(\mathbf{e})=0$, and ...
0
votes
2answers
41 views

Rational function regression without poles in a interval, or polynomial regression with positivity constraint

I have some sets of experimental data for some functions $f_i$ from $I=[0,1]$ onto itself, which should satisfy the following physical constraints: $f_i(0)=1$ $f_i(x) \in I= [0,1] \; \forall x \in I ...
2
votes
1answer
28 views

Prove a result in multiple linear regression

This arises in multiple linear regression. Given $m, n \in \mathbb{N}$ and matrices $X \in \mathbb{R}^{m \times (n+1)} (m > n + 1), H = X(X'X)^{-1}X' \in \mathbb{R}^{m\times m}, I = I_m$ and $J ...
1
vote
0answers
12 views

How is genetic programming used in Symbolic regression

I am in highschool and have not taken any courses on this. Rather I am working on an outside project. I don't quite understand how Genetic Programming could be used effectively to generate a set of ...
0
votes
0answers
22 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!