Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
66 views

Finding the value which minimises all residuals

I have a series of observations, measurements made at various times $t$. I now need to determine the most likely value of $R$ (distance) using the model below. The guide says I should find the value ...
1
vote
1answer
47 views

Distribute a number in an exponential way in a certain number of times

I have a number $X$ (let say it's 100). I want to 'split' it$N$ times in a 'exponential' looking way. For instance $y = 0,2692x^{-0,925}$ with $x$ the integers between 1 and 16 is working fine. The ...
1
vote
2answers
173 views

Least Squares Derivation

I was reading this to review the derivation of the ordinary least squares estimator but I'm having trouble differentiating (4). Can someone please help explain why $ \dfrac{\partial ...
1
vote
0answers
13 views

What is the meaning of errors of separate variables in a fit?

When doing a least-square fitting of a two-parameter function (e.g. $y=a+bx$) with specialised software like Origin or gnuplot, one gets errors for the resulting $a$ and $b$. What do these errors mean ...
2
votes
2answers
892 views

Least squares estimator of mu

The question is: Assuming that $y_i = \mu + \epsilon_i $,$i = 1,\ldots,n$ with independent and identically distributed errors $\epsilon_i$ such that $E[\epsilon_i] = 0$ and $Var[\epsilon_i] = ...
4
votes
1answer
201 views

Straight line through data by eye - least squares? [closed]

I heard an interesting fact a while ago about how people draw a line through a cloud of points on a scatter plot. Usually, when calculating lines of best fit, we use the minimal the sum of squares of ...
1
vote
1answer
143 views

Convergence in distribution and OLS in the regression model

Suppose a random variable $X_n$ converges in distribution to X and $E[X]< \infty$, Is this statement true or is there any condition to satisfy this? $ plim_{n\to \infty}Var(X_n) = Var(X) $ ...
0
votes
1answer
610 views

Recursive curve fitting

I have a few points for which I have to obtain a best fit. (I tried to use the least squares curve fitting method as Robert says, however, since it seems to be fitting rather poorly, any other ...
1
vote
0answers
25 views

T-Value And Significance (SLR in R)

I am currently trying to figure out the output I got from the summary-command when I do a linear regression in R. I get 2 values that I do not understand, first: the t-value. I do understand that it ...
3
votes
2answers
642 views

Binary Logistic Regression Model Processing

Thanks for showing interest and wanting to help out. My aim is to develop a model that - as accurately as possible - predicts how entities in a population will either cooperate or defect, as a % of ...
9
votes
3answers
271 views

Minimize $||Ax-b||$ but for $A$, not $x$

I have a machine learning regression problem. I need to minimize $$ \sum_i||Ax_i-b_i||_2^2 $$ However I am trying to find matrix $A$, not the usual $x$, and I have lots of example data for $x_i$ and ...
0
votes
2answers
248 views

Why Logistic Regression for Classification Problems?

In a class on machine learning, we covered classification problems. In such a problem, you are studying a property of some object, say malignity of tumors in a patient. You are first given a training ...
0
votes
1answer
65 views

How to find optimal border that defines, who is “friend”

I have the data about usage of several services in the population and the data about interactions among users. The idea is to determine, who is user's friend and who has interacted just ...
1
vote
2answers
577 views

Exponential Regression Model

I need to model my data ($(x,y)$ pairs) using the following exponential function: $$f(x) = \exp((x + a)/b) - c$$ So, I need to find $a, b, c$ coefficients that are the best fit for my data. What is ...
1
vote
1answer
58 views

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method?

How to derive the equations in 3:19-3:30 provide in a MIT opencourse ware lecture about the least square method? Link: http://www.youtube.com/watch?v=YwZYSTQs-Hk Thanks in advance!
4
votes
4answers
675 views

Method of Least Squares-Why is it preferred? [duplicate]

Possible Duplicate: Why do we use a Least Squares fit? To find the normal equations for derivation of the regression line we use the method of least squares, We want to make the error ...
0
votes
2answers
38 views

Where is the square in the Least square regression method?

I'm having a serious doubt in the least square regression problem. I guess its got to do with the notation of norm. Is the least square formulation $||b - \mathbf{A}x||^2$ or is it $||b - ...
1
vote
1answer
250 views

Approximation using Legendre polynomials

my aim is to fit data points by the use of Legendre Polynomials. Has anybody experience with this task? My final aim is to do this automatically with mathematica. Thanks, rainer
2
votes
1answer
101 views

Nonlinear regression with correlated errors

it's my first post here and I'm a newbie in statistics, so please forgive me if I'm doing something wrong or explaining myself badly. Anyway, I have a problem similar to this: How to perform ...
1
vote
1answer
45 views

Interpret overall fitness in prediction

I came through this equation and would like to learn more about how to inteprete it. d is defined as actual value, and $\hat{d}$ is defined as predicted value. Why does this equation (a) divides sum ...
0
votes
2answers
554 views

Least Squares Fitting - Logarithmic

Is there any software that will fit a set of 2D points using a logarithmic curve? I found this, which looks like the formulas I need, but I don't think I have a fancy enough calculator to solve that ...
0
votes
1answer
50 views

Plane vertices from normals and centroid

I am trying to visualize a best fit plane for a set of points which is defined by the normals and the centroid. I have to find the boundary vertices of the plane given the extent of the plane. Is ...
3
votes
2answers
17k views

How to calculate a correction factor for two sets of numbers

Suppose one has a set of numbers. To help understand my question, suppose that these numbers are from two different temperature sensors. In this first example, both sensors are placed in the same ...
1
vote
2answers
715 views

How to do a regression with only integer values and a fixed intercept?

I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this ...
1
vote
2answers
86 views

Multiple regression with model $Y = (1 + c_1X_1)(1 + c_2X_2)\ldots(1 + c_nX_n)$

I'm currently working with data contained in $Y, X_1, X_2, \ldots, X_n$ and wish to fit it to the model: $Y = (1 + c_1X_1)(1 + c_2X_2)\ldots(1 + c_nX_n)$ where the $c_i$ are coefficients to be ...
3
votes
0answers
163 views

How to perform nonlinear regression with correlated errors?

I have a nonlinear least squares problem, but the errors are correlated. I could use R's nls function to do the regression if the errors were independent, but I don't know the right way to handle ...
1
vote
4answers
342 views

Parabola from 4 approximate points

I have calculated four approximate points from a sensors to get information. I would like to deduce the closest parabola to my points. The problem is that I can't solve it to get an appropriate ...
3
votes
2answers
133 views

Can I consider shooting% as an independent variable

First time poster in the math section (a few posts in the stats section) and I am looking for clarification on a variable query that I have. Basically I enjoy sports and enjoy putting a mathematical ...
1
vote
1answer
215 views

Quadratic function as a linear function

I am taking a machine learning course and today we were given an example of regression, with two attributes $x_,x_2$ and $y$ being the real valued outcome. $y$ is a quadratic function of $x_1,x_2$, ...
2
votes
0answers
331 views

Finding a model for multiple non-linear regression

I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
1
vote
0answers
465 views

Fitting a 3d point cloud with a polynomial surface

I have 3D point cloud and I would like to fit a polynomial surface to it. Could anybody please explain the step by step process to that. Thanks a lot.
2
votes
0answers
3k views

Derivation of standard error of beta in simple linear regression

Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example: ...
0
votes
1answer
253 views

Predict the height of a student whose weight is 60 kilograms.

The average height and weight of a group of students turned out to be 5 ft 6 inches and 65 kilograms respectively. The correlation between heights and weights was found to be 0.6. Using the regression ...
0
votes
2answers
753 views

“Proof” of an Algebraic property of OLS Estimators

I'm having a bit of trouble proving $\sum (x_i - \bar{x})\hat{e_i} = 0$. What I know so far is that the total sum of $\hat{e_i}$'s is zero by property of OLS so when you distribute the $\hat{e_i}$ ...
0
votes
1answer
33 views

Optimization, solving for the 'error' coefficient

Given a modified regression equation: $\hat Y = \exp(\beta_0 + \sum\beta_ix_i + \varepsilon)*F$ where: $\hat Y = 11353$ $\beta_0 = 8.693021$ $\sum\beta_ix_i = 5.95487177696$ $F = 0.21829$ what ...
0
votes
2answers
359 views

Overcoming Linear Regression Assumptions

I'm a beginner in econometrics (learning on my own, and not from school) and I'm trying to build an intuition to understanding linear regression. We know that modeling real world data is bound to ...
1
vote
2answers
142 views

Calculating the MLE for mu(x) in a regression model

Say we have the following regression model: $$Y_i = \alpha + \beta(x_i - \mathrm{mean}(x)) + R_i$$ where $R_1,\ldots,R_{20} \sim G(0, \sigma)$ If we have $\mu(x) = \alpha + \beta(x - ...
2
votes
1answer
453 views

variance of multiple regression coefficients

If I consider universal kriging (or multiple spatial regression) in matrix form as: ${\bf{V = XA + R }}$ where $\bf{R}$ is the residual and $\bf{A}$ are the trend coefficients, then the estimate of ...
1
vote
0answers
341 views

Confusion regarding confidence interval

I was using matlab's cftool to fit a regression line to my data point x and y. And I could see this ...
2
votes
1answer
67 views

Probabilistic regression on outliers

I have a given data set $D = \{ x_i, y_i \}_{i=1}^n$ for a regression problem. When I plot the data, it looks like there is an underlying parabola (2nd order linear model) and some outliers. I want ...
1
vote
1answer
101 views

Orthogonal fitted values

I have two regression models $$Y=X\beta+\varepsilon,\quad \beta\in\mathbb{R}^k$$ $$Y=Z\alpha+u\quad \alpha\in\mathbb{R}^m$$ it is known that using OLS estimates $\hat{\beta},\hat{\alpha}$ fitted ...
1
vote
1answer
400 views

Multiple Regression over an experimental dataset

I want to do a multiple regression over an experimental result shown as 3D-Plot and heatmap in following Images. Sorry as a new user i am not allowed to post them directly but it is just a link to ...
0
votes
1answer
114 views

Linear regression for normal distributions

Basically, I have that $\ Y_i = \alpha +\beta(x_i-x_{bar}) + \epsilon_i $ where $\epsilon_i$ are i.i.d normally distributed with mean 0 variance $\sigma^2$ $\ Y_i ~~has ~a~normal~distribution~as ...
1
vote
2answers
551 views

vertical vs. horizontal regression

A horizontal regression is defined as the following: $$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$ whereas ...
0
votes
1answer
67 views

P-value not shown when there are too many variables in a linear regression

x<-c(1,2,3) y<-x^1.1+x summary(lm(y~x+I(x^1.1))) I have this code in R but it just is for the sake of easier understanding of what I am trying to achieve. ...
4
votes
2answers
1k views

How does one fit the curve $y = ae^{bx} + c$?

How does one fit the curve $y = ae^{bx} + c$? The method of taking logarithms of both sides does not simplify to allow linear regression. I can take the three equations derived from setting the ...
2
votes
1answer
360 views

An intuitive explanation for neural networks as function approximators ?

We use normal linear regression for modelling functions on datasets . But Can someone explain how neural networks help in approximating more complex ,especially non-linear functions ? intuitively , ...
2
votes
3answers
190 views

least squares regression in 3space

robjohn is giving me a hand with this, but in case anybody else knows... I need to do a least-squares regression for linearity on a set of coordinates in 3space. If the dataset is linear, I need to ...
2
votes
2answers
302 views

design matrices

Given a linear model $Y = X\beta + \epsilon$ with three treatments and six subjects where $X$ is the design matrix, suppose $X = \begin{matrix}1 & 1 & 0\\ 1 & 1 & 0\\ 1 & 0 ...
0
votes
1answer
62 views

Expressing Series-Element in Terms of its Index

Consider the following recursion: $$C_{i+1} = a \sum_{j=1}^iC_j + b$$ where $a$ and $b$ are constants. Can series-element $C_i$ be expressed in terms of only its index $i$, $a$ and $b$? In case ...