Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.
2
votes
2answers
192 views
Least squares estimator of mu
The question is:
"Assuming that $y_i = \mu + \epsilon_i $,$i = 1,\ldots,n$ with independent and identically distributed errors $\epsilon_i$ such that $E[\epsilon_i] = 0$ and $Var[\epsilon_i] = ...
3
votes
1answer
111 views
Straight line through data by eye - least squares? [closed]
I heard an interesting fact a while ago about how people draw a line through a cloud of points on a scatter plot. Usually, when calculating lines of best fit, we use the minimal the sum of squares of ...
1
vote
1answer
79 views
Convergence in distribution and OLS in the regression model
Suppose a random variable $X_n$ converges in distribution to X and $E[X]< \infty$,
Is this statement true or is there any condition to satisfy this?
$ plim_{n\to \infty}Var(X_n) = Var(X) $ ...
0
votes
1answer
152 views
Recursive curve fitting
I have a few points for which I have to obtain a best fit. (I tried to use the least squares curve fitting method as Robert says, however, since it seems to be fitting rather poorly, any other ...
1
vote
0answers
18 views
T-Value And Significance (SLR in R)
I am currently trying to figure out the output I got from the summary-command when I do a linear regression in R.
I get 2 values that I do not understand, first: the t-value. I do understand that it ...
1
vote
1answer
183 views
Binary Logistic Regression Model Processing
Thanks for showing interest and wanting to help out.
My aim is to develop a model that - as accurately as possible - predicts how entities in a population will either cooperate or defect, as a % of ...
8
votes
3answers
164 views
Minimize $||Ax-b||$ but for $A$, not $x$
I have a machine learning regression problem. I need to minimize
$$
\sum_i||Ax_i-b_i||_2^2
$$
However I am trying to find matrix $A$, not the usual $x$, and I have lots of example data for $x_i$ and ...
0
votes
1answer
75 views
Why Logistic Regression for Classification Problems?
In a class on machine learning, we covered classification problems. In such a problem, you are studying a property of some object, say malignity of tumors in a patient. You are first given a training ...
0
votes
1answer
48 views
How to find optimal border that defines, who is “friend”
I have the data about usage of several services in the population and the data about interactions among users.
The idea is to determine, who is user's friend and who has interacted just ...
1
vote
2answers
349 views
Exponential Regression Model
I need to model my data ($(x,y)$ pairs) using the following exponential function:
$$f(x) = \exp((x + a)/b) - c$$
So, I need to find $a, b, c$ coefficients that are the best fit for my data. What is ...
1
vote
1answer
56 views
How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?
How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?
Link: http://www.youtube.com/watch?v=YwZYSTQs-Hk
Thanks in advance!
3
votes
4answers
213 views
Method of Least Squares-Why is it preferred? [duplicate]
Possible Duplicate:
Why do we use a Least Squares fit?
To find the normal equations for derivation of the regression line we use the method of least squares, We want to make the error ...
0
votes
2answers
33 views
Where is the square in the Least square regression method?
I'm having a serious doubt in the least square regression problem. I guess its got to do with the notation of norm. Is the least square formulation $||b - \mathbf{A}x||^2$ or is it $||b - ...
1
vote
1answer
130 views
Approximation using Legendre polynomials
my aim is to fit data points by the use of Legendre Polynomials. Has anybody experience with this task? My final aim is to do this automatically with mathematica.
Thanks,
rainer
2
votes
1answer
75 views
Nonlinear regression with correlated errors
it's my first post here and I'm a newbie in statistics, so please forgive me if I'm doing something wrong or explaining myself badly.
Anyway, I have a problem similar to this: How to perform ...
1
vote
1answer
36 views
Interpret overall fitness in prediction
I came through this equation and would like to learn more about how to inteprete it.
d is defined as actual value, and $\hat{d}$ is defined as predicted value.
Why does this equation
(a) divides sum ...
0
votes
2answers
192 views
Least Squares Fitting - Logarithmic
Is there any software that will fit a set of 2D points using a logarithmic curve?
I found this, which looks like the formulas I need, but I don't think I have a fancy enough calculator to solve that ...
0
votes
1answer
29 views
plane vertices from normals and centroid
I am trying to visualize a best fit plane for a set of points which is defined by the normals and the centroid. I have to find the boundary vertices of the plane given the extent of the plane. Is ...
3
votes
2answers
2k views
How to calculate a correction factor for two sets of numbers
Suppose one has a set of numbers. To help understand my question, suppose that these numbers are from two different temperature sensors. In this first example, both sensors are placed in the same ...
1
vote
2answers
116 views
How to do a regression with only integer values and a fixed intercept?
I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this ...
0
votes
0answers
233 views
Use linear regression to make an inverse prediction
I have a problem I am trying to solve using the R programming language and software environment. This isn't a programming question but more how I can use the results however I might describe the ...
1
vote
2answers
73 views
Multiple regression with model $Y = (1 + c_1X_1)(1 + c_2X_2)\ldots(1 + c_nX_n)$
I'm currently working with data contained in $Y, X_1, X_2, \ldots, X_n$ and wish to fit it to the model:
$Y = (1 + c_1X_1)(1 + c_2X_2)\ldots(1 + c_nX_n)$
where the $c_i$ are coefficients to be ...
3
votes
0answers
100 views
How to perform nonlinear regression with correlated errors?
I have a nonlinear least squares problem, but the errors are correlated. I could use R's nls function to do the regression if the errors were independent, but I don't know the right way to handle ...
1
vote
4answers
192 views
Parabola from 4 approximate points
I have calculated four approximate points from a sensors to get information. I would like to deduce the closest parabola to my points. The problem is that I can't solve it to get an appropriate ...
3
votes
2answers
94 views
Can I consider shooting% as an independent variable
First time poster in the math section (a few posts in the stats section) and I am looking for clarification on a variable query that I have. Basically I enjoy sports and enjoy putting a mathematical ...
1
vote
1answer
81 views
Quadratic function as a linear function
I am taking a machine learning course and today we were given an example of regression, with two attributes $x_,x_2$ and $y$ being the real valued outcome.
$y$ is a quadratic function of $x_1,x_2$, ...
2
votes
0answers
188 views
Finding a model for multiple non-linear regression
I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
0
votes
0answers
90 views
Proof that $F$-statistic of dropping single coefficient is square of $z$-score in linear regression
Show that the $F$ statistic $\left(F=\frac{(\text{RSS}_1-\text{RSS}_0)/(p_1-p_0)}{\text{RSS}_1/(N-p_1-1)}\right)$ for dropping a single coefficient from a model is equal to the square of the ...
1
vote
0answers
114 views
Fitting a 3d point cloud with a polynomial surface
I have 3D point cloud and I would like to fit a polynomial surface to it.
Could anybody please explain the step by step process to that.
Thanks a lot.
1
vote
0answers
1k views
Derivation of standard error of beta in simple linear regression
Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example:
...
0
votes
1answer
118 views
Predict the height of a student whose weight is 60 kilograms.
The average height and weight of a group of students turned out to be
5 ft 6 inches and 65 kilograms respectively. The correlation between
heights and weights was found to be 0.6. Using the regression ...
0
votes
2answers
121 views
“Proof” of an Algebraic property of OLS Estimators
I'm having a bit of trouble proving $\sum (x_i - \bar{x})\hat{e_i} = 0$. What I know so far is that the total sum of $\hat{e_i}$'s is zero by property of OLS so when you distribute the $\hat{e_i}$ ...
0
votes
1answer
24 views
Optimization, solving for the 'error' coefficient
Given a modified regression equation:
$\hat Y = \exp(\beta_0 + \sum\beta_ix_i + \varepsilon)*F$
where:
$\hat Y = 11353$
$\beta_0 = 8.693021$
$\sum\beta_ix_i = 5.95487177696$
$F = 0.21829$
what ...
0
votes
2answers
184 views
Overcoming Linear Regression Assumptions
I'm a beginner in econometrics (learning on my own, and not from school) and I'm trying to build an intuition to understanding linear regression. We know that modeling real world data is bound to ...
0
votes
0answers
144 views
Combine dependent variables into index that best matches independent variable?
I'm an undergrad working in a geochemistry lab for the summer. My job requires a good amount of statistical analysis, something that I'm not very familiar with.
I'm looking for an way to blindly ...
1
vote
2answers
70 views
Calculating the MLE for mu(x) in a regression model
Say we have the following regression model:
$$Y_i = \alpha + \beta(x_i - \mathrm{mean}(x)) + R_i$$
where $R_1,\ldots,R_{20} \sim G(0, \sigma)$
If we have $\mu(x) = \alpha + \beta(x - ...
2
votes
1answer
132 views
variance of multiple regression coefficients
If I consider universal kriging (or multiple spatial regression) in matrix form as:
${\bf{V = XA + R }}$
where $\bf{R}$ is the residual and $\bf{A}$ are the trend coefficients, then the estimate of ...
1
vote
0answers
156 views
Confusion regarding confidence interval
I was using matlab's cftool to fit a regression line to my data point x and y. And I could see this
...
2
votes
1answer
52 views
Probabilistic regression on outliers
I have a given data set $D = \{ x_i, y_i \}_{i=1}^n$ for a regression problem. When I plot the data, it looks like there is an underlying parabola (2nd
order linear model) and some outliers.
I want ...
1
vote
1answer
64 views
Orthogonal fitted values
I have two regression models
$$Y=X\beta+\varepsilon,\quad \beta\in\mathbb{R}^k$$
$$Y=Z\alpha+u\quad \alpha\in\mathbb{R}^m$$ it is known that using OLS estimates $\hat{\beta},\hat{\alpha}$ fitted ...
0
votes
1answer
213 views
Multiple Regression over an experimental dataset
I want to do a multiple regression over an experimental result shown as 3D-Plot and heatmap in following Images. Sorry as a new user i am not allowed to post them directly but it is just a link to ...
0
votes
1answer
59 views
Linear regression for normal distributions
Basically, I have that
$\ Y_i = \alpha +\beta(x_i-x_{bar}) + \epsilon_i $ where $\epsilon_i$ are i.i.d normally distributed with mean 0 variance $\sigma^2$
$\ Y_i ~~has ~a~normal~distribution~as ...
1
vote
2answers
148 views
vertical vs. horizontal regression
A horizontal regression is defined as the following:
$$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$
whereas ...
0
votes
1answer
48 views
P-value not shown when there are too many variables in a linear regression
x<-c(1,2,3)
y<-x^1.1+x
summary(lm(y~x+I(x^1.1)))
I have this code in R but it just is for the sake of easier understanding of what I am trying to achieve. ...
3
votes
1answer
220 views
How does one fit the curve $y = ae^{bx} + c$?
How does one fit the curve $y = ae^{bx} + c$?
The method of taking logarithms of both sides does not simplify to allow linear regression.
I can take the three equations derived from setting the ...
0
votes
0answers
139 views
Matlab polyfit boundary condfition
I am trying to perform a polynomal regression of $y(x)$ so with the boundary condition $y(0)=y_0$ and $y(x_max)=0$, and with these constraints perform a least square or other fitting.
It seems that ...
2
votes
1answer
89 views
An intuitive explanation for neural networks as function approximators ?
We use normal linear regression for modelling functions on datasets . But Can someone explain how neural networks help in approximating more complex ,especially non-linear functions ? intuitively , ...
0
votes
0answers
101 views
How to update logistic regression with new data
I am running logistic regression to find the value of products in real time auction and bid accordingly.
At start I bid randomly to collect random data. Then run logistic regression on it.
And ...
2
votes
3answers
149 views
least squares regression in 3space
robjohn is giving me a hand with this, but in case anybody else knows...
I need to do a least-squares regression for linearity on a set of coordinates in 3space. If the dataset is linear, I need to ...
2
votes
2answers
174 views
design matrices
Given a linear model $Y = X\beta + \epsilon$ with three treatments and six subjects where $X$ is the design matrix, suppose $X = \begin{matrix}1 & 1 & 0\\
1 & 1 & 0\\
1 & 0 ...
