Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
162 views

Which machine learning algorithm to use?!

I have a training set which is set of essays written by students for a question. These essays are all scored by human evaluators with labels such as 1, 2 , 3 which is actually marks allotted for those ...
0
votes
0answers
1k views

Prove that the expectation of residual sum of squares (RSS) is equal to $\sigma^2(n-2)$

The assumed regression model is $E(Y_i|x)=\gamma+\beta(x-\bar{x})$ and $Var(Y|x)=\sigma^2$. So I have: $E(RSS)= E(\sum\limits_{i=1}^n (y_i-\hat{y_i})^2)= E(\sum\limits_{i=1}^n ...
0
votes
1answer
373 views

Question about derivatives, Jacobian Matrix and non-linear least square curve fitting

I am working with a software package that performs non-linear least squares curve fitting. The goal of curve fitting is to calculate the coefficients of an equation given x,y points. The software ...
0
votes
2answers
241 views

Regression analysis on temperature/sensor data

Looking for a solution to what I thought should be an easy problem, but has me running in circles somehow... I'm working with two sets of data: he first set is raw values from a sensor ...
0
votes
1answer
39 views

Curve Fitting and Multiple Experiments

Say I do an an experiment 5 times, each of which gives you a list of data points. Do I fit a curve to each one separately and then average the parameters and their uncertainties? Or do I take the ...
0
votes
0answers
166 views

What does the superscript T refer to in the following equations regarding non-linear least squares curve fitting?

In the paper, http://people.duke.edu/~hpgavin/ce281/lm.pdf, equations 1), 2), and 3) refer to the chi-squared error criterion. Equation 2) is reproduced below. $$ \chi^2({\mathbf p}) = (\mathbf y- ...
0
votes
0answers
44 views

Getting formula from known solutions?

Firstly, my first language is not english and I don't really know the technical terms of maths in english, so sorry for my poor expression. I have an equation like $\ y = x * z $. And I know that $\ ...
0
votes
1answer
87 views

Find local min and local max from dataset

I have a collection of data consists of TIME and TEMPERATURE as follow: (time1, temp1), (time2, temp2), (time3, temp3), ... (time-n temp-n). I cannot come up with a good computer algorithm to ...
0
votes
1answer
75 views

Regression vs. Normal Distribution

I have to estimate something using historical data. Should I find the equation of the curve of best fit to estimate? Or use a confidence interval, standard deviation, and a z-score to calculate it? ...
0
votes
1answer
57 views

Questions Regarding Linear Regression

Are the slope and intercept of a simple linear regression model always normally distributed? Is there ever a difference between the distribution of the estimated slope and intercept and the actual ...
0
votes
2answers
440 views

Linear Regression - Proof Sum Adds to Zero

In linear regression, why is $\sum(X_{i} - \mu_{x})$ = $0$? I understand that for ($\sum$ $Y_{i}$ minus the fitted value of Y) = $\sum$ $e_{i}$ this is true but why is this other fact true?
0
votes
0answers
226 views

Stata: “Between and fixed effect estimates” in a linear regression?

I'm working on a paper by B. H. Baltagi and I am trying to replicate the results. It can be found here, the data is here. I'm supposed to do a linear regression - sounds simple. The author uses Stata, ...
0
votes
1answer
26 views

Interpretation of regression formula returned by computer software

I have a dataset consisting of 744 records. Data exploring software generated an equation I don't know how to interpret in simple words. I really appreciate if you could help me about this matter. ...
0
votes
1answer
187 views

Multiple regression problems (restricted regression, dummy variables)

Q1. Model 1: $Y=X_1\beta_1+\varepsilon$ Model 2: $Y=X_1\beta_1+X_2\beta_2+\varepsilon$ (a) Suppose that Model 1 is true. If we estimates OLS estrimator $b_1$ for $\beta_1$ in Model 2, what will happen ...
0
votes
0answers
645 views

Generating an equation from an image I have

I am not exactly sure if this question belongs here but I could not think of a better place to ask. So I recently discovered that various people on the internet have created equations for rather ...
0
votes
1answer
267 views

Fast way of finding RSS of Multiple Linear Regression

Is there any smarter way to compute Residual Sum of Squares(RSS) in Multiple Linear Regression other then fitting the model -> find coefficients -> find fitted values -> find residuals -> find norm of ...
0
votes
0answers
64 views

How does this affect the slope

This is a Linear regression question, where I have a list of values for Sales ($Y$) and Property ($X$), I have calculated the Mean for both $Y$ and $X$, resulting in $Y=52.2$and $X=17$, I have ...
0
votes
1answer
280 views

Determining slope of line relative to a maximum

In the following scientific report (Seismic Q estimation), a mathematical procedure of linear curve-fitting is described in words. The authors state: The stratigraphic effects are minimized by ...
0
votes
2answers
156 views

Non-linear regression - least square regression

I was trying to get some insight into how to solve non-linear regression model problems. Unfortunately, I've never attended a lecture on statistical math. Here is the link: In page number 4, they ...
0
votes
1answer
387 views

How to find autoregressive coefficients of ARMA model

I am searching various sources to find the method of manually calculating coefficients of Auto Regressive Moving Average model. The following is the text I found in a book. My question is how a1 and ...
0
votes
0answers
81 views

Why Local Minimum is calculated for a derivative function instead of actual function?

In Machine learning regression problem, why the local minimum is computed for a derivative function instead of the actual function? Example: http://en.wikipedia.org/wiki/Gradient_descent The ...
0
votes
0answers
41 views

Multivariate linear regression/aproximation

I'm solving a problem which features a function $f: \mathbb{R}^4 \rightarrow \mathbb{R}^4.$ I don't know the function, but I assume it's linear and it can be expressed as $ \mathbf{y} = \mathbf{A} ...
0
votes
1answer
334 views

When to use likelihood ratio test? [closed]

I have a few questions regarding the use of likelihood ratio test in a logistic regression model. Suppose we have a logistic regression model like this: ...
0
votes
0answers
69 views

Linear-Regression Result Accuracy as a Function of Slope, Other Factors

Say I have the following functions $ f(x) = Asin(Bx) $ $ g(x) = M_1x $ $ h(x) = M_2x $; where $M_2 \approx 0$ and $M_1 > 1000 M_2$ $ z(x) = C $ $ e(x) = N(0,\sigma) $ $ m_g(x) = f(x) + z(x) - ...
0
votes
1answer
85 views

Standard error of a statistic

the standard error of a standard deviation is given as : s/n^(1/2). Would the standard error of kurtosis and skewness follow the same idea? For example, se of kurtosis = kurtosis/n^(1/2)?
0
votes
0answers
223 views

How to find the formula of a spiral using least-squares(regression)?

Assume data from a plane which are roughly showing a spiral. I want employ the rationale of regression to find the parameters for the best fit by some spiral. That means, I have to estimate the ...
0
votes
2answers
38 views

Where is the square in the Least square regression method?

I'm having a serious doubt in the least square regression problem. I guess its got to do with the notation of norm. Is the least square formulation $||b - \mathbf{A}x||^2$ or is it $||b - ...
0
votes
1answer
50 views

Plane vertices from normals and centroid

I am trying to visualize a best fit plane for a set of points which is defined by the normals and the centroid. I have to find the boundary vertices of the plane given the extent of the plane. Is ...
0
votes
1answer
253 views

Predict the height of a student whose weight is 60 kilograms.

The average height and weight of a group of students turned out to be 5 ft 6 inches and 65 kilograms respectively. The correlation between heights and weights was found to be 0.6. Using the regression ...
0
votes
1answer
114 views

Linear regression for normal distributions

Basically, I have that $\ Y_i = \alpha +\beta(x_i-x_{bar}) + \epsilon_i $ where $\epsilon_i$ are i.i.d normally distributed with mean 0 variance $\sigma^2$ $\ Y_i ~~has ~a~normal~distribution~as ...
0
votes
1answer
62 views

Expressing Series-Element in Terms of its Index

Consider the following recursion: $$C_{i+1} = a \sum_{j=1}^iC_j + b$$ where $a$ and $b$ are constants. Can series-element $C_i$ be expressed in terms of only its index $i$, $a$ and $b$? In case ...
0
votes
0answers
110 views

Covariance and variance

Within the context of simple linear regression, I came across this: $$\hat{\beta}=\frac{\sum y_nx_n}{\sum x_n^2}=\frac{cov(xy)}{var(x)}$$ where I assume $cov(x,y)$ means the covariance between $x$ ...
0
votes
1answer
90 views

Regression on Linear Model?

I have 50 or so training examples involving a set of 200 or so real numbers (x1,x2,...,x200) (normalized to a 0 mean and std dev 1), and a single output real (y) in the range 0.0..1.0. I want to fit ...
0
votes
1answer
138 views

Explain Least-squares plotting with Ones -matrix

I am stuck to this code and particularly the point F=[ones(size(x)) x x.^2]. What does it do mathematically? I cannot understand what the heck the matrix has to do ...
0
votes
1answer
4k views

How different is Beta computation using Covariance and Linear Regression?

I wanted to compute Beta for a Stock against an Index (Say Stock X against S&P 500). I computed the daily returns for over one year applied the following logic : Beta = COVAR(X, S&P ...
0
votes
2answers
6k views

Unconditional expectation vs conditional expectation in regressions - does it really matter?

I refer here to a simple linear regression whose true representation is given by the equation: $y_i=x_i'\beta+u_i$, where as usual $x_i$ is a $Kx1$ vector of independent explanatory variables, ...
-1
votes
1answer
25 views

Using least squares regression for line of best fit

Use the least square approximation to find the closest line (the line of "Best Fit") to the points: $$(-6,-1), \quad (-2,2), \quad (1,1), \quad (7,6)$$ I'm attempting to use the least squares ...
-1
votes
1answer
15 views

For the three measurements b=0, 3, 12 at times t=0, 1, 2 find the best parabola y=C+Dt+E$t^2$

So I know how to do least squares regression using matrices to solve for Ax=b. I simply do $A^TAx=A^Tb$. However I don't really know how to account for the second power in a typical parabola ...
-1
votes
1answer
43 views

How to proof that least square estimator $\hat{B}$ doesnt exist when $x$ is linearly dependent?

For the linear regression model $Y=xB+e$, prove that if the columns of $X$ are linearly dependent, the least square estimator $\hat{B}$ does not exist I know that since $\hat{B}$ is an unbiased ...
-1
votes
1answer
20 views

Linear Regression question 1

I would really be grateful if someone could let me know how to answer Part (a) of Question 1. I believe i should scatter plot both x and y values separately for year 2000 and year 2001 on same graph ...
-1
votes
2answers
950 views

“taking the derivative of both sides” ln(Y)=ln(x) (to interpret log transformed regression variables or better understand elasticities or % changes)

In a linear regression of the form Y=bX we often have ln transformed Y and X. ie., lnY=b*lnX This is interpreted as a 1% change in X resulting in a b% change in Y (approximately) The derivation of ...
-1
votes
1answer
686 views

How to work out orthogonal polynomials for regression model

I put this question here as it has a pure maths element to it, even though it has a statistical twist. Basically, I have been given the table of data: $$\begin{matrix} i & \mathrm{Response} \, ...
-1
votes
2answers
131 views

Proving that the estimate of a mean is a least squares estimator?

I think this is a really simple question so please bear with me -- I just had my first class in regression and I'm a little confused about nomenclature/labeling. Does anyone recommend some good ...
-4
votes
2answers
188 views

Guess the functional form of a graph

Can you guess the functional form of the following curve y is 0 at x= Infinite ; y is very small ( +ve near to zero) at x=0 Thanks and regards