Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
2answers
41 views

Expected value and the standard simple regression model

Given the standard simple regression model: $y_i = β_0 + β_1 x_i + u_i$ What is the expected value of the estimator $\hat\beta_1$in terms of $x_i, \beta_0$ and $\beta_1$ when $\hat\beta_1=\sum x_i ...
0
votes
2answers
46 views

Approximate/Find Function

I have got values $x_{i}$ and targets $z_{i}$. Now I want to find a function $f(x)=z$, which approximates the mapping of my value $x_{i}$ to its targetvalue $z_{i}$ as good as possible for every ...
0
votes
0answers
95 views

Simple curve fitting smoothing algorithm

Is there any conditionally simple algorithm to obtain a best fit (equation) for points set? I have a few points for which I want to have a "smooth" function is constructed that approximately fits the ...
3
votes
1answer
42 views

Detrending sine waves accurately

I am doing some data analysis where I look at electricity demand over the course of a day, but need to separate the intra-day (constant and periodic) components from daily changes (assumed linear). At ...
3
votes
1answer
32 views

Quantile Regression - Linear Loss Minimization

I'm currently reading Quantile Regression by Roger Koenker, and for some reason I'm having a lot of trouble deriving one of his equation (sect. 1.3, p. 5-6). He goes on to demonstrate that $\hat{x}$ ...
2
votes
0answers
47 views

GLM for Poisson Regression for Soccer Ratings Not Converging

I have been trying to formulate a model of soccer matches to help me predict the outcomes. The model I'm trying to formulate involves using Poisson regression to assign attack and defence ratings to ...
0
votes
0answers
58 views

How to find a perfect regression fit in R?

I have a set of points, which I know can be described with some equation. How can I find this equation? The scatter plot for this set looks like this: I look at the plot and assume that I can use a ...
0
votes
0answers
62 views

How to calculate relation between Beats (as in BPM) and variance of the energies?

I'm working on an implementation of a Beat Detection algorithm and I can't find a relation between the statistical variance of the energies in 1 second of audio decoded as PCM and the threshold value ...
0
votes
0answers
40 views

Understanding Piecewise Logistic Function

I am trying to understand the use and application of the piecewise logistic function/regression and have recently been shown how to apply it after getting some help at another forum. However, the ...
2
votes
1answer
92 views

Least-squares solution to a matrix equation?

Suppose I have $n$ observations of $m$ dependent variables $y_1,\dots,y_m$, and I believe they follow some model wherein they can all be written as linear combinations of some underlying variables ...
3
votes
0answers
30 views

Regression with many discrete and continuous predictors and few rows

I want to do regression on a dataset. It has one continuous dependent variable that I want to predict. It has many categorical and some continuous predictors. It only has a few rows. A simplified ...
1
vote
3answers
61 views

Transposing matrix when differentiating it

Hi so I am trying to understand the solution of linear regression with matrices (found at the following link) and an confused about how on page 10 he says the derivative of $2Y'XB$ with respect to $B$ ...
1
vote
0answers
25 views

LRT and Wald Test in Multivariate Linear Regression significance

I've been researching all afternoon trying to get a better idea of what the Likelihood ratio test and the Wald test are actually doing. I have a bunch of covariates and I'm testing out like 30 ...
0
votes
0answers
40 views

Finding the curvature from a set of datapoints

I have a set of 1. 1-d 2. 2-d data. I want to find the curvature at each single point. Till now I was using difference technique to find out the curvature, i.e, central difference at middle and ...
1
vote
1answer
50 views

Significance of dummy variables in probit regression

I am an undergraduate student working on some projects using probit regression. I have a question on dummy variables that I was hoping someone could help me with (which I think stems from an ...
1
vote
1answer
14 views

Find a distribution for this plot

Please help me find a formula that fits the distribution. It does not need to be exact, a simple approximation would suffice. Bonus points if you can tell me which predefined distribution in the ...
0
votes
1answer
56 views

Least squares with known error in y

so I want to do a linear least squares regression on my data, however I have known experimental error on my data points in $y$ and relatively few numbers of points so I would like to use values ...
0
votes
1answer
50 views

How do I solve for vector $P$ in the matrix equation $s=A'B^{-1}A$?

I would like to rearrange the matrix equation $s=A'B^{-1}A$ into the form $A=f(s,B)$ (i.e., some function of $s$ and $B$), where s is scalar, $A$ is $n\times 1$, $A'$ is the transpose of $A$, and $B$ ...
0
votes
1answer
26 views

finding a question about constrained regression - had a side constraint $x \geq y^*z$

I saw a question that asked about solving a non-negative least squares problem with $3$ unknowns, $(x,y,z)$. But there was an additional constraint, $x \geq y^*z$. Would appreciate getting the ...
0
votes
1answer
29 views

Getting VAR parameters from a research paper.

Many econometrics papers provide the parameters used in their VAR model. If I notate my VAR model as $$z_{t+1} = c + B z_{t} + \Sigma \epsilon_{t+1}$$ where $\epsilon \sim N(0, I)$, then I need to ...
1
vote
3answers
83 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
0
votes
1answer
15 views

In a simple regression model estimated using OLS, the covariance between the estimated errors and regressors is zero by construction

Is this statement true or false? I seem to remember that this relationship does not hold when the regression has no intercept, however my teacher said that this was true regardless of whether we ...
0
votes
1answer
45 views

Polynomial least squares fit — restrictions on order?

If we're finding an interpolating polynomial for 10 data pairs, the order of the polynomial has to be 9. In class, my professor said that when doing a polynomial least squares fit, if you have 10 ...
0
votes
2answers
95 views

Non-linear regression fit

I'm trying to fit my data to the following equation: $$ Y = A(1-2e^{bx}) $$ What I tried to do was transform the equation to a linear form via the following steps: \begin{align*} & A-Y = ...
3
votes
0answers
39 views

How to perform nonlinear regression with regressors affected by gaussian error?

I am trying to calibrate a sensor and I have a data set consisting of several observations of a 3-dimensional vector $X_i$, with $X_i=w_i + \epsilon_i$ where $w_i$ is the value that the sensor ...
1
vote
2answers
104 views

Fit exponential with constant

I have data whic would fit to an exponential function with a constant. So y=aexp(bt) + c Now I can solve an exponential without a constant using least square by taking log of y and making the ...
0
votes
1answer
16 views

Least Squares Estimators Derivation (Bi-variate)

To derive least square estimators: We have $SS(\alpha,\beta)= \sum(y_i-\alpha-bx_i)^2$ and find partials for each. The answer I get is: $\beta = \frac{\sum y_i-\bar{y}}{\sum x_i-\bar{x}}$, but the ...
1
vote
1answer
61 views

Multiple Linear Regression in Matrix Form

I am currently studying for my exams and came across the following question: State the multiple linear regression equation in matrix form. Write down the order of each matrix and explain what the ...
2
votes
0answers
29 views

Can I adjust linear growth of a a subpopulation to a linear decay of the general population?

I need to estimate the amount of CF patients in Poland in the next four years. I have: estimations of the Polish population for the future years a CF patients' register for the last couple of years ...
0
votes
2answers
38 views

Calculating the correlation coefficient between least square estimates

PROBLEM STATEMENT: Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean $0$ and variance $1$; $$y_i = α + β(x_i − \bar ...
0
votes
1answer
31 views

Why is the SSE <= SST?

I can think of a regression line producing a larger sum of squared errors (SSE) than the total sum of squares (SST). I read that should not be possible, how come? My understanding is that the ...
2
votes
1answer
71 views

Outlier detection with robust multiple regression model

I have a set of features (eg, location, income, budget, education) that I use to predict a continuous variable (say, amount spent per day on the internet). I am interested in detecting outliers. I ...
0
votes
0answers
28 views

Transpose is just the way of generalizing a dot product?

It seems like $a^Tb$ is the same as writing $a \cdot b$ in matrix form. 1) Why is $n \times 1$ and $n \times 1$ matrix multiplication undefined? 2) Is this just a generalization of the dot ...
1
vote
1answer
22 views

Do we only use T distribution for confidence interval of Beta for linear regression or we can also normal distribution?

Do we only use T distribution for confidence interval of Beta for linear regression or we can also normal distribution? Is it that when sample size is less than 30 then we use T distribution else ...
4
votes
5answers
61 views

calculating least squares fit

I read this thread talking about 'why we use least squares' for curve fitting Why do we use a Least Squares fit? One answer by Chris Taylor begins with the assumption that we should look for $$ ...
1
vote
4answers
55 views

How to estimate $\alpha$ in $y=(1-\exp(-\alpha x))/(1+\exp(-\alpha x))$?

I have a function $$y=\dfrac{1-\exp(-\alpha x)}{1+\exp(-\alpha x)}$$ where $y$ is not binary. The range of this function is $[-1,1)$. So this does not fit into either logit or probit models. How ...
0
votes
1answer
73 views

Identical observations in linear regression

I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
0
votes
1answer
36 views

Method for ?not quite? weighted least squares fitting for more realistic results

I need a linear least squares type of fitting algorithm that understands how to weight the probability of a response coming from certain functions over another. To explain, given the standard linear ...
1
vote
1answer
21 views

Assumption of Normal Distribution

I have a problem and I do not know when it is crucial and when it is NOT crucial to assume a normal distribution regarding linear regression, for estimates, t-tests, f-tests, confidence intervals and ...
1
vote
1answer
94 views

Best percentage change for trend

Consider the revenue of a company for the last five year and you want to to know whether there is an upward, downward or no trend. How would you calculate the "optimal" percentage change? I have an ...
1
vote
1answer
31 views

Is there such a thing as a weighted multiple regression?

I'm new to linear algebra, but I know how multiple linear regressions work. What I want to do is something slightly different. As an example, let's say that I have a list of nutrients I want to get ...
1
vote
0answers
48 views

Confusion with Bayesian Linear Regression

In the book Gaussian Processes for Machine Learning in Chapter 2 p. 11 (see http://www.gaussianprocess.org/gpml/chapters/RW2.pdf), eq. 2.9 states: $p(f_* | X, y) = \int p(f_* | x_*,w) p(w|X, y)dw$ ...
3
votes
1answer
57 views

Calculate trend and represent in text?

First off, I'm terrible at math. I'm writing a script that monitors transactions from clients daily over a 7 day period. Given a set of numbers like below, I would like to calculate a trend and ...
0
votes
0answers
32 views

Relation between the Coefficient of Multiple Correlation and Coefficient of Simple Correlation

Consider the regression model $Y=\beta_1 X_1+\beta_2 X_2+\epsilon$, with a sample of size $n$, $Y_i=\beta_1 X_{i1}+\beta_2 X_{i2}+\epsilon_i$, $\epsilon_i \in N(0,\sigma^2)$. Suppossing ...
1
vote
1answer
35 views

Logistic regression coefficients problem

I'm using logistic regression model to do a multi-class classification (4 classes). I want to look at the logistic regression coefficients to see the importance of different features. I got model ...
0
votes
1answer
33 views

Distribution of e, least squares residuals

I have the model $y=X{\beta}+{\epsilon}$ and $E({\epsilon})=0$ and $Var({\epsilon})={\sigma}^2I_n$ The vector y can be written: $y=X\hat{\beta}+{e}$ If ${\epsilon}$~$N(0, {\sigma}^2I_n)$ how is ...
0
votes
1answer
39 views

Quadratic Form Matrices

How do I know if a matrix in quadratic form, e.g. D'MD is positive or negative (semi)definite? M here is the residual maker matrix for X, so I know that it is symmetric. I know what the definitions ...
2
votes
0answers
50 views

Show $\hat{\beta}$ and $s^2$ are independent?

I have the model: $y=X{\beta}+{\epsilon}$ I know $\hat{\beta}=(X'X)^{-1}X'y$ and that it is an unbiased estimator of ${\beta}$ and that $s^2=\hat{\epsilon}'\hat{\epsilon}/(n-k)$ and is an unbiased ...
0
votes
1answer
52 views

Multiple linear regression inconsistency?

I've got a linear model: $y_i=β_1x_{i1}+β_2x_{i2}+ε_i$ where E($ε_i$)=0 and Var($ε_i$)= $σ^2I_n$ for i=1,...,n Supposed we don't have the data for $x_{i2}$ and we estimate: $y_i=β_1x_{i1}+ε_i$ for ...
1
vote
0answers
121 views

Minimize correlation between input and output of a linear system

I am not sure if "minimize correlation" is the right title for this issue but I could not find a better sentence to describe what I would like to achieve. Let's say that I have a black box with ...