Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
1answer
116 views

Is it possible to have two lines of best fit?

Could you rig a data set to have two lines of equally good (and best) fit? Or is it impossible?
2
votes
1answer
23 views

Three-Perpendicular Theorem for linear regressions

For a random vector $X=(X_1,\ldots,X_p)'$, we define $$ \mathcal{L}(X)=\{b_0+b_1X_1+\cdots+b_pX_p,b_0,\ldots,b_p\in\mathbb{R}\}. $$ The linear regression of the $q$-dimensional random vector $Y$ ...
2
votes
1answer
132 views

Solution of overdetermined polynomial system

Some of you will find this question pretty straightforward to answer, but I desperately need some help in solving a problem involving several equations and 2 unknowns, for an engineering application. ...
2
votes
1answer
62 views

Standard Error in OLS Regression

Assuming I have the following linear regression set-up: $y_i = \alpha + x_i * \beta + \epsilon_i$ for $i = 1,2,..., n$. When I run the regession, I get a $\beta$ and $\alpha$ estimates, along ...
2
votes
1answer
139 views

Prove that $E(\mathbf{u}|\mathbf{X})=\mathbf{0}$ implies $Cov(\mathbf{x},\mathbf{u})=\mathbf{0}$

Let \begin{equation} \mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{u} \end{equation} where $\mathbf{y}=\begin{bmatrix}y_1 \\ \vdots \\ y_n\end{bmatrix}$, $\mathbf{X}=\begin{bmatrix}X_{11} & \...
2
votes
1answer
597 views

Least Squares Regression Matrix for Rational Functions

So first off no, this isn't a homework problem. Second, I'm trying to understand how this works, NOT find a program that will do it for me. Okay so I've known for a while how to use Gaussian-Jordan ...
2
votes
2answers
138 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
2
votes
1answer
40 views

Gaussian prior favors values closest to zero?

I am reading an article on Bayesian Logistic Regression, where they're using Logistic Regression, imposing a Gaussian prior (with mean = 0) on its parameters. They state that a Gaussian prior favors ...
2
votes
1answer
44 views

Is the Inverse of the Vectorised Solid Angle Equation for $n$ Circular Discs Continuous?

I have a continuous function$^{*1}$ that takes in 3 arguments, and returns 24 outputs. I want to know if the inverse of this function is continuous. The 3 input arguments are the x, y, and z position ...
2
votes
2answers
142 views

derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
2
votes
1answer
236 views

Fitting a simple linear regression

A professor in the School of Business in a university polled a dozen colleagues about the number of professional meetings they attended in the past five years $x$ and the number of papers they ...
2
votes
2answers
79 views

Find parameters for curve fitting (simple linear regression involved?)

I would like to fit data in g~t scatterplot, where ...
2
votes
2answers
57 views

Merging Linear Regression

If I have built two linear regression models over sets $A$ and $B$, and now want a linear regression over set $A\cup{}B$. Is there a way to reuse what I already have?
2
votes
2answers
40 views

Regression model when under-estimations costs us more than over-estimations

We have a factory and we are planning how many items produce in 2014. During the learning process we minimize the mean squared error. But, under-estimations costs us more than over-estimations. Let's ...
2
votes
2answers
735 views

How to fit a sinusoidal function through 2 points with known slopes?

I can define my sinusoidal function as $y(x) = A\sin(B x+c) + D$ or as $y(x) = A \sin(B x) + C \cos(B x) + D$ Now, I have two points with known slopes that I must fit this sine wave to, thus my ...
2
votes
1answer
112 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
2
votes
1answer
2k views

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with ...
2
votes
2answers
2k views

Linear Regression: Expectation Proof

I found the following proof in my notes: $E(Y_i) = E[\beta_0 + \beta X_i + \varepsilon_i] =\cdots= \beta_0 + \beta X_i$. This does not seem right to me, however. Why would $E(\beta_1 X_i) = \beta_1 ...
2
votes
1answer
268 views

Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$ y = f(x_1, x_2, x_3,\...
2
votes
1answer
512 views

Finding uncertainty in the slope/intercept for a non-linear least squares fit

I have the following function: $$M = a(\log_{10}W-2.5)+b$$ I also have a set of data with actual measured values of $W$ and $M$ (each have individual $\pm$ errors). Here's a small sampling of the ...
2
votes
1answer
144 views

Least Squares Regression To Half of a Parabola

I have a set of points in two dimensional space, and I know a priori that they approximate half of a parabola. I want to find the coefficients for a quadratic function where all of the points fall on ...
2
votes
2answers
327 views

maximize log determinant subject to a linear constraint

Does anyone know any efficient method to solve the following problem? $ (\alpha,\beta) = \text{argmax} \log \det (\alpha K_1 + \beta K_2)$ s.t. $c_1 \alpha + c_2 \beta = c_3, \alpha\geq0, \beta\geq ...
2
votes
1answer
65 views

Possibility of Unboundedness in Least Squares Minimization

Suppose we have the quadratic minimization problem \begin{equation} \min_x \frac{1}{2} x^TPx + q^Tx +r \end{equation} We know that when $P$ is symmetric positive semi-definite, but the optimality ...
2
votes
1answer
103 views

What am I reinventing? RE: Linear regression modeling for frequency of discrete events

I'm looking to model the frequency of events to quantify how much that frequency is increasing or decreasing. For the sake of concreteness think of the events as web page hits for several low traffic ...
2
votes
1answer
69 views

Biased linear regression

I have a set $S$ of coordinates $(x,y)$, and am estimating $f(x) = ax + b$ where $a > 0$. I also happen to know that $\forall x,y((x,y) \in S \implies y < f(x))$. The question is how I can ...
2
votes
1answer
272 views

How to find a parametric equation?

I want to find an equation for a race track, so I could get the position of a point with respect to time. Let's say I have this track and here are a few points on it: Could it be possible to model ...
2
votes
1answer
106 views

Nonlinear regression with correlated errors

it's my first post here and I'm a newbie in statistics, so please forgive me if I'm doing something wrong or explaining myself badly. Anyway, I have a problem similar to this: How to perform ...
2
votes
1answer
583 views

variance of multiple regression coefficients

If I consider universal kriging (or multiple spatial regression) in matrix form as: ${\bf{V = XA + R }}$ where $\bf{R}$ is the residual and $\bf{A}$ are the trend coefficients, then the estimate of ...
2
votes
1answer
68 views

Probabilistic regression on outliers

I have a given data set $D = \{ x_i, y_i \}_{i=1}^n$ for a regression problem. When I plot the data, it looks like there is an underlying parabola (2nd order linear model) and some outliers. I want ...
2
votes
3answers
2k views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = \sigma(f(...
2
votes
1answer
530 views

Curve fitting with upper and lower bounds for derivatives

I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
2
votes
2answers
227 views

How to fit an equation to a curve with disturbances

For example, I have the following data: $Y = 366$ measured values $X = 366$ measured values $t = [ 1 : 366 ]$, representing the days of the year (index) So at each $t$ (day), we have value of $Y$ ...
2
votes
2answers
155 views

Choosing set of best estimators for linear least squares

I have a measured experimental dataset which is well approximated by the sum of several basis functions in linear combinations. Linear least squares of course gives me the optimal weight for each ...
2
votes
2answers
1k views

Fitting data to a portion of an ellipse or conic section

Is there a straightforward algorithm for fitting data to an ellipse or other conic section? The data generally only approximately fits a portion of the ellipse. I am looking for something that doesn't ...
2
votes
0answers
53 views
+50

Two dimensional (discrete) orthogonal polynomials for regression

This question How to work out orthogonal polynomials for regression model and the answer http://math.stackexchange.com/a/354807/51020 explain how to build orthogonal polynomials for regression. ...
2
votes
0answers
31 views

What does it mean to regress out current features?

First of all, I'd like to say that this is the intro to a homework problem. Please do not post any answers, I am only looking for clarification on some terminology in the setup. I am trying to ...
2
votes
1answer
50 views

How do I calculate regression line using a data set with repeated values indicated as frequencies?

I have a data set that comprises of Independent Variable $(X)$ and Dependent Variable $(Y)$ values with a certain frequency $(F)$. I know that I have to find $x^2$ and $xy$ but how do I factor in the ...
2
votes
0answers
27 views

Ridge Regression Centering Proof

This is a ridge regression problem. The following two problems are equivalent: $(w_t, b_\lambda ) = argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^Tx_i)^2+\lambda w^Tw\} $ $(w_t, b_\lambda )= argmin_{w,b}\{\...
2
votes
1answer
42 views

Add weights to inputs of x-value function to optimize regression [closed]

Say I have $n$ functions (not the regression function) each with $n$ inputs. These functions compute the x-values. The function is a simple summation function where the input is multiplied by a ...
2
votes
0answers
44 views

Does linear regression form a subspace?

The author writes Given a vector of inputs $X^T = (X_1, \dots ,X_p)$, we can predict an output $Y$ via $$ \hat{Y} = \beta_0 + \sum_{j = 1}^p X_j \beta_j$$ He goes on to note that if we include a 1 in ...
2
votes
0answers
23 views

standard deviation for regression

The first slide is the denifition of simple linear regression model, the second slides is an example I have two questions: 1.Did I get the right calculation of the standard deviation? 2.I still ...
2
votes
0answers
26 views

Non linear regression calculator

Are there any really good non linear regression calculators around the web? Or is something like matlab the best solution? I tried using excel and its solver tool, but it's complete garbage lol. ...
2
votes
1answer
34 views

Regression maximum likelihood

Given this regression model: $y_{i}=\beta_{0}+\beta_{1}x_{i}+E_{i}$. All the assumptions are valid except that now: $E_{i}\sim N(0,x_{i}\sigma^{2})$ Find Maximum likelihood parameters for $\beta_0$,$...
2
votes
0answers
26 views

Proof that correlation coefficient squared equals the coefficient of determination

Hi I as the title says I'm looking at the proof that $r^2$ = $R^2$ in the case of simple linear regression, but I don't understand one part. There are different versions of the proof, but in most of ...
2
votes
1answer
17 views

linear modeling question: How can I find variance for vector Y?

Linear models are $$Y_1= 2\theta_1+3\theta_2 +\epsilon_1$$ $$Y_2= -2\theta_1+\theta_2 +\epsilon_2$$ and $\epsilon_1=3z_1-z_2$ and $\epsilon_2=4z_1+z_2$, where $z_1, z_2$ are two random variance such ...
2
votes
1answer
38 views

Kalman filter using regressed model

I'm currently polishing flight control system for KSP, and I'm fightinng high-frequency noise in state vector measurements right now. I want to try to apply Kalman filter to provide more smooth values ...
2
votes
0answers
17 views

Problems with Exchange Procedure in Remez Algorithm

So first off: *** This code is not being used in production software. It is a personal project of mine, trying to understand approximation theory and advanced curve fitting. In other words, I'm ...
2
votes
1answer
45 views

Optimal place to measure for simple linear regression/fitting

Suppose I have a linear model $y_i=a\cdot x_i+b+\epsilon_i$, where $y_i,\epsilon_i,a,b\in\mathbb{R}$, $x_i\in[-1,1]$. I can take n measurements of $y_i$ at $x_i$, where $n\in\mathbb{N}$. $\epsilon_i$ ...
2
votes
0answers
25 views

combining multiple regression outputs

Suppose I have multiple regressions, along with their r-squares, standard-errors, etc.: $y(t) = \alpha_1 + \beta_1 x(t) + e_1$, where $t \in (\tau_0, \tau_1)$ $y(t) = \alpha_2 + \beta_2 x(t) + e_2$,...
2
votes
0answers
62 views

GLM for Poisson Regression for Soccer Ratings Not Converging

I have been trying to formulate a model of soccer matches to help me predict the outcomes. The model I'm trying to formulate involves using Poisson regression to assign attack and defence ratings to ...