Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
0answers
167 views

A* vs D* vs Dijkstra [closed]

I understand the basis of A* as being a derivative of Dijkstra, however, I recently found out about D*. From wikipedia, I can understand the algorithm. What I do not understand is why I would use D* ...
2
votes
1answer
99 views

How to calculate $\sum(X_i-\bar{X})^2$ in R

I'm trying to figure out how to calculate $\sum(X_i-\bar{X})^2$ in R, specifically identifying it in either the aov function or $\operatorname{lm}(y\sim x)$ function. I am trying to use it to ...
2
votes
1answer
31 views

Aproximate data with this equation (or linearize the equation)

I have found an equation that describes the behaviour of a phisical system: $$ y=a_1e^{-a_2x} + a_3 + a_4x + a_5e^{{-a_6} / {(1-x)}}$$ Now I have data of that physical system and I want to ...
2
votes
1answer
101 views

Best fit line using geometric distance (not vertical distance)

There must be a theory of finding the best fit line to a bunch of points in the plane, where "best fit" is defined by the geometric distance, not vertical distance. In other words, we are trying to ...
2
votes
1answer
58 views

Basic Multilinear regression question for finding examples or counterexamples.

Hello Wise mathematicians! I have few quenstions about Multi linear regresstion. I've been asked from my friend, but I have very weak knowledge background from that field. It seems my friend is in ...
2
votes
1answer
116 views

Why does the regression line of $x$ on $y$ and $y$ on $x$ meet at $\bar{x}$ and $\bar{y}$?

Why does the least squares regression line of $x$ on $y$ and $y$ on $x$ intersect at $\bar{x}$ and $\bar{y}$? Also, why are the form of regression lines as they are? For the general form ...
2
votes
1answer
61 views

Probit model question (regression)

I'm reading a thesis and I need your help to understand the equation below. $$\Pr(\text{failure}=1 \mid X_1,X_3,X_3,X_4)=\int_{-\infty}^z \varphi(k) \, dk\tag{1}$$ $\varphi(k)$ is the standard ...
2
votes
0answers
1k views

Least Squares “analytic expression” for fitting a 2D quadratic function to measurements

I have n scattered elevation measurements: $ \{x_i,y_i,z_i\}_{i=1..n} $ that I want to fit a quadratic function to: $ z = ax^2 + by^2 + cxy + dx + ey + f$. The problem can be written as a vector ...
2
votes
0answers
57 views

Computing evidence for least-squares fit

I'm at a loss trying to implement Bayesian model selection for standard least-squares polynomials fits. I have three polynomials of order $1$, $2$, and $3$, and a sequence of $(x,y)$ data points. ...
2
votes
0answers
793 views

What is the Moore-Penrose pseudoinverse for scaled linear regression?

The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations: $$ ({\bf{X}}^T \bf{X}) \vec{\beta}= ...
2
votes
2answers
32 views

Linear regression. Lowering response maintaining equal independent variable.

I have put some data together and modelled the behaviour of the response ($y$) as function of three independent variables $x_1$, $x_2$ and $x_3$. A simple multi-linear regression. The model looks ...
2
votes
0answers
45 views

Coefficient of determination

$$ \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 = ( \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 - \displaystyle \sum^n_{i = 1} (y_i - \hat{y}_i)^2 ) + \displaystyle \sum^n_{i = 1} (y_i - ...
2
votes
0answers
167 views

Orthonormal Matrix weighted regression

$Q$ is a rectangular matrix with orthonormal columns. A linear system composed of $$Qx= b$$ is really easy to solve as: $$Q'Q=I$$ hence: $$x=Q'b$$ Given that $Q$ is orthonormal can this be used to ...
2
votes
0answers
91 views

Regressing $Y$ back on the residuals

Suppose I have the linear regression model $ \hat{y_i} = a + b x_i $ for $a,b$ obtained via OLS. How does one regress $y$ back on the residuals $\hat{e}_i = y_i - \hat{y}_i$? If we write $ ...
2
votes
0answers
192 views

Effective model for calculating momentum or growth rate for a time series

I have a series of numbers tracking the performance of an entity on any given day. It's nothing but a simple integer for each date. For example, here's a series for Entity "X" ...
2
votes
0answers
421 views

Bare minimum of points in multiple polynomial regression

I have a question on multiple polynomial regression and the absolute minimum amount of points in the different terms. The minimum amount of points required for a second order polynomial would (in one ...
2
votes
0answers
98 views

Find $w$ as the minimizer of regularized logistic regression

Could someone point me to a reference on how to find $w$ as the minimizer of: $$ \frac{1}{2}\sum_{i=1}^{d}q_i(w_i-m_i)^2+\sum_{j=1}^{n}log(1+\exp(-y_jw^Tx_j)) $$ where $q_i$ (initialized with ...
2
votes
0answers
487 views

Logistic regression algorithm in Casio and Texas Instruments calculators

When using logistic regression on a Casio or Texas Instruments calculator, the output is of the form $$f(x) = \frac{c}{1+ae^{-bx}} $$ The problem I have (when teaching in a class where both types of ...
2
votes
2answers
1k views

Least squares estimator of mu

The question is: Assuming that $y_i = \mu + \epsilon_i $,$i = 1,\ldots,n$ with independent and identically distributed errors $\epsilon_i$ such that $E[\epsilon_i] = 0$ and $Var[\epsilon_i] = ...
2
votes
0answers
348 views

Finding a model for multiple non-linear regression

I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
2
votes
0answers
3k views

Derivation of standard error of beta in simple linear regression

Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example: ...
2
votes
1answer
320 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: $y$=value, $x_1$=material, $x_2$=weight, $x_3$=color $x_1$ and $x_2$ are interacting variables but $x_3$ ...
2
votes
0answers
86 views

Polynomial and exponential regression [duplicate]

Possible Duplicate: Determining computational complexity of stochastic processes I have some points $(x_i,y_i)$ generated by a program. These values are not exact, but are random ...
2
votes
0answers
1k views

Correlation and Regression Question

Two separate tests are designed to measure a students ability to solve problems. Several students are randomly selected to take both tests and the results are: $$ \begin{matrix} \text{Test A}(x) ...
2
votes
0answers
3k views

Help with problem: Estimated Standard Deviation of Regression Equation (Simple Linear)

This is a practice problem. I've solved part (a). I have provided verified answers (from the published key) to all parts (a), (b) and & (c). I need help solving (b) and (c). Consider a simple ...
2
votes
0answers
303 views

Surface Function Fitting to Spherical Data

I have a set of geographic (longitude,latitude,value) data to which I would like to fit surface functions, specifically, the set of quadratic surfaces: $f(x,y)=Ax^2+Bx^2+Cxy+Dx+Ey+F$ At the moment, ...
2
votes
0answers
68 views

Accurate computation for Linear Regression case

I am writing a program that inputs a sequence of points $(x_i,y_i)$ based on the user clicking on certain pixels in an image shown. The program should then find the "best -fitting" line in the least ...
1
vote
5answers
645 views

Find square root approximation function (tool)

first I have to apologize for any uncorrect naming or categorisation of my question, as I am an electrical engineer rather than a mathematican. I try to find a simple solution for my problem: I have ...
1
vote
4answers
375 views

Parabola from 4 approximate points

I have calculated four approximate points from a sensors to get information. I would like to deduce the closest parabola to my points. The problem is that I can't solve it to get an appropriate ...
1
vote
3answers
663 views

Simple Least-Squares Regression Question

Given a set of 5 points (i.e. (1, 3), (2, 8) etc...), how can I get just the slope of the best fit line? I've been looking up least squares regression, but I'm rather statistics ignorant and don't ...
1
vote
3answers
47 views

Transpose of $(X'X)^{-1}$

I am taking a Phd class in econometrics, and the following is used constantly, for $X$ a $n\times k$ matrix, $n \neq k$: $$(X'X)^{-1} = ((X'X)^{-1})'$$ with "$'$" standing for transpose. Having a ...
1
vote
2answers
103 views

Fit exponential with constant

I have data whic would fit to an exponential function with a constant. So y=aexp(bt) + c Now I can solve an exponential without a constant using least square by taking log of y and making the ...
1
vote
2answers
76 views

Is it possible to fit any regression line to a set of data points?

If you have a set of data points (x1,y1), (x2,y2),...,(xn,yn) And you know it fits a trend y=f(x) where ...
1
vote
3answers
112 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
1
vote
1answer
300 views

How to find a line of best fit of the form $y=ax$?

We have the following points: $$ (0,0)(1,51.8)(1.9,101.3)(2.8,148.4)(3.7,201.5)(4.7,251.1)(5.6,302.3)(6.6,350.9)(7.5,397.1)(8.5,452.5)(9.3,496.3)$$ How can we find the best fitting line $y=ax$ ...
1
vote
3answers
83 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
1
vote
2answers
70 views

Using inverse of matrix A as approximate inverse of matrix that is very close to A

Say we have two matrices, $A$ and $A'$ so that $A\approx A'$, and we have the inverse of $A$, $B$, where $AB=I$, and the inverse of $A'$ where $A'B'=I$. If we have some guarantee about how big any ...
1
vote
2answers
46 views

Show $E\left(\mathbf{X}_i \otimes \mathbf{u}_i\right)=\mathbf{0}$ implies $E\left(\mathbf{X}_i^{\top}\mathbf{G}\mathbf{u}_i\right)=\mathbf{0}$

Let $\mathbf{X}_i$ be a $G \times K$ random matrix, and let $\mathbf{u}_i$ be a $G \times 1$ random vector, and suppose we have a sample of $i=1,\ldots,N$ of each. Suppose the following condition ...
1
vote
2answers
44 views

Would it be any concern if we find correlation between intercept and other regression coefficients?

During a multiple linear regression analysis, I found correlation between intercept (beta-0) and two of the other regression coefficients. Is there any problem or concern in this case? If no, please ...
1
vote
2answers
171 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
1
vote
3answers
1k views

Exponential extrapolation

Given a set of points on 2D surface $(x_1,y_1),(x_2,y_2),\ldots,(x_n,y_n)$ and a function $f(x)=k+ab^x$, the task is to find values of $k,a$ and $b$ that minimize the following sum: $$\sum_{i=1}^n ...
1
vote
2answers
434 views

Linear trend has to pass through a point

I need to interpolate a linear trend surface through a number of points but with the condition that the surface has to pass exactly through one of them. Can somebody give me any advice?
1
vote
1answer
33 views

Relation between correlation and regression

Let $y\in \mathbb{R}$ be a random variable. Let $y$ be expressed as a linear combination of $x_i$ $i=1,2,\cdots,n$, as follows \begin{equation} y = \sum\limits_{i=1}^nw_ix_i + \epsilon \end{equation} ...
1
vote
1answer
76 views

Statistical property proof

Consider the multiple linear regression model $y = Xβ + ε$, where $X$ is the $n × p$ design matrix, $y$ is the $n × 1$ dimensional vector of response and $ε ∼ N(0,σ^2I)$. The vector consisting of the ...
1
vote
1answer
37 views

Corollary of the Frisch-Waugh Theorem

Consider the following linear regression model: $$y=X \beta + \epsilon = X_1 \beta_1 + X_2 \beta_2 + \epsilon $$ Where we have $n$ obsevations and $k$ variables, and hence $X$ is a matrix $nk$, and ...
1
vote
2answers
44 views

Prove that $E(|(X+Y)(X-Y)|) \leq 2\sqrt{1-\rho²}$, where $\rho$ is correlation.

For two random variables $X,Y$ with mean $0$ and variance $1$, their correlation is $\rho$. We have to prove that $$E(|(X+Y)(X-Y)|) \leq 2\sqrt{1-\rho^2}.$$ But, I can't understand how the $\rho$ ...
1
vote
4answers
406 views

Program to find closest function to fit arbitrary data

I've wanted this for years, but have never come across anything; a program for Windows to find the closest function to fit arbitrary data. The data I feed it is simple: A table with two columns ...
1
vote
2answers
355 views

Is the Square Root of an Inverse Matrix Equal to the Inverse of the Square Root Matrix?

I know in general that if a matrix $A$ is positive definite, then there exists a (unique?) square root matrix $B$, which is also positive definite, such that $BB=A$. Therefore, suppose $A$ is ...
1
vote
1answer
60 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
1
vote
1answer
119 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...