Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
2answers
647 views

How to fit a sinusoidal function through 2 points with known slopes?

I can define my sinusoidal function as $y(x) = A\sin(B x+c) + D$ or as $y(x) = A \sin(B x) + C \cos(B x) + D$ Now, I have two points with known slopes that I must fit this sine wave to, thus my ...
2
votes
1answer
97 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
2
votes
2answers
2k views

Linear Regression: Expectation Proof

I found the following proof in my notes: $E(Y_i) = E[\beta_0 + \beta X_i + \varepsilon_i] =\cdots= \beta_0 + \beta X_i$. This does not seem right to me, however. Why would $E(\beta_1 X_i) = \beta_1 ...
2
votes
1answer
241 views

Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$ y = f(x_1, x_2, ...
2
votes
1answer
480 views

Finding uncertainty in the slope/intercept for a non-linear least squares fit

I have the following function: $$M = a(\log_{10}W-2.5)+b$$ I also have a set of data with actual measured values of $W$ and $M$ (each have individual $\pm$ errors). Here's a small sampling of the ...
2
votes
1answer
142 views

Least Squares Regression To Half of a Parabola

I have a set of points in two dimensional space, and I know a priori that they approximate half of a parabola. I want to find the coefficients for a quadratic function where all of the points fall on ...
2
votes
2answers
305 views

maximize log determinant subject to a linear constraint

Does anyone know any efficient method to solve the following problem? $ (\alpha,\beta) = \text{argmax} \log \det (\alpha K_1 + \beta K_2)$ s.t. $c_1 \alpha + c_2 \beta = c_3, \alpha\geq0, \beta\geq ...
2
votes
1answer
65 views

Possibility of Unboundedness in Least Squares Minimization

Suppose we have the quadratic minimization problem \begin{equation} \min_x \frac{1}{2} x^TPx + q^Tx +r \end{equation} We know that when $P$ is symmetric positive semi-definite, but the optimality ...
2
votes
1answer
103 views

What am I reinventing? RE: Linear regression modeling for frequency of discrete events

I'm looking to model the frequency of events to quantify how much that frequency is increasing or decreasing. For the sake of concreteness think of the events as web page hits for several low traffic ...
2
votes
1answer
68 views

Biased linear regression

I have a set $S$ of coordinates $(x,y)$, and am estimating $f(x) = ax + b$ where $a > 0$. I also happen to know that $\forall x,y((x,y) \in S \implies y < f(x))$. The question is how I can ...
2
votes
1answer
241 views

How to find a parametric equation?

I want to find an equation for a race track, so I could get the position of a point with respect to time. Let's say I have this track and here are a few points on it: Could it be possible to model ...
2
votes
1answer
104 views

Nonlinear regression with correlated errors

it's my first post here and I'm a newbie in statistics, so please forgive me if I'm doing something wrong or explaining myself badly. Anyway, I have a problem similar to this: How to perform ...
2
votes
1answer
536 views

variance of multiple regression coefficients

If I consider universal kriging (or multiple spatial regression) in matrix form as: ${\bf{V = XA + R }}$ where $\bf{R}$ is the residual and $\bf{A}$ are the trend coefficients, then the estimate of ...
2
votes
1answer
68 views

Probabilistic regression on outliers

I have a given data set $D = \{ x_i, y_i \}_{i=1}^n$ for a regression problem. When I plot the data, it looks like there is an underlying parabola (2nd order linear model) and some outliers. I want ...
2
votes
1answer
495 views

Curve fitting with upper and lower bounds for derivatives

I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
2
votes
2answers
218 views

How to fit an equation to a curve with disturbances

For example, I have the following data: Y = 366 measured values X = 366 measured values t = [ 1 : 366 ], representing the days of the year (index) So at each t (day), we have value of Y and ...
2
votes
2answers
154 views

Choosing set of best estimators for linear least squares

I have a measured experimental dataset which is well approximated by the sum of several basis functions in linear combinations. Linear least squares of course gives me the optimal weight for each ...
2
votes
2answers
1k views

Fitting data to a portion of an ellipse or conic section

Is there a straightforward algorithm for fitting data to an ellipse or other conic section? The data generally only approximately fits a portion of the ellipse. I am looking for something that doesn't ...
2
votes
0answers
15 views

Ridge Regression Centering Proof

This is a ridge regression problem. The following two problems are equivalent: $(w_t, b_\lambda ) = argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^Tx_i)^2+\lambda w^Tw\} $ $(w_t, b_\lambda )= ...
2
votes
1answer
32 views

Add weights to inputs of x-value function to optimize regression [closed]

Say I have $n$ functions (not the regression function) each with $n$ inputs. These functions compute the x-values. The function is a simple summation function where the input is multiplied by a ...
2
votes
0answers
37 views

Does linear regression form a subspace?

The author writes Given a vector of inputs $X^T = (X_1, \dots ,X_p)$, we can predict an output $Y$ via $$ \hat{Y} = \beta_0 + \sum_{j = 1}^p X_j \beta_j$$ He goes on to note that if we include a 1 in ...
2
votes
0answers
20 views

standard deviation for regression

The first slide is the denifition of simple linear regression model, the second slides is an example I have two questions: 1.Did I get the right calculation of the standard deviation? 2.I still ...
2
votes
0answers
16 views

Non linear regression calculator

Are there any really good non linear regression calculators around the web? Or is something like matlab the best solution? I tried using excel and its solver tool, but it's complete garbage lol. ...
2
votes
1answer
30 views

Regression maximum likelihood

Given this regression model: $y_{i}=\beta_{0}+\beta_{1}x_{i}+E_{i}$. All the assumptions are valid except that now: $E_{i}\sim N(0,x_{i}\sigma^{2})$ Find Maximum likelihood parameters for ...
2
votes
0answers
17 views

Proof that correlation coefficient squared equals the coefficient of determination

Hi I as the title says I'm looking at the proof that $r^2$ = $R^2$ in the case of simple linear regression, but I don't understand one part. There are different versions of the proof, but in most of ...
2
votes
1answer
17 views

linear modeling question: How can I find variance for vector Y?

Linear models are $$Y_1= 2\theta_1+3\theta_2 +\epsilon_1$$ $$Y_2= -2\theta_1+\theta_2 +\epsilon_2$$ and $\epsilon_1=3z_1-z_2$ and $\epsilon_2=4z_1+z_2$, where $z_1, z_2$ are two random variance such ...
2
votes
0answers
22 views

Kalman filter using regressed model

I'm currently polishing flight control system for KSP, and I'm fightinng high-frequency noise in state vector measurements right now. I want to try to apply Kalman filter to provide more smooth values ...
2
votes
0answers
15 views

Problems with Exchange Procedure in Remez Algorithm

So first off: *** This code is not being used in production software. It is a personal project of mine, trying to understand approximation theory and advanced curve fitting. In other words, I'm ...
2
votes
1answer
42 views

Optimal place to measure for simple linear regression/fitting

Suppose I have a linear model $y_i=a\cdot x_i+b+\epsilon_i$, where $y_i,\epsilon_i,a,b\in\mathbb{R}$, $x_i\in[-1,1]$. I can take n measurements of $y_i$ at $x_i$, where $n\in\mathbb{N}$. $\epsilon_i$ ...
2
votes
0answers
25 views

combining multiple regression outputs

Suppose I have multiple regressions, along with their r-squares, standard-errors, etc.: $y(t) = \alpha_1 + \beta_1 x(t) + e_1$, where $t \in (\tau_0, \tau_1)$ $y(t) = \alpha_2 + \beta_2 x(t) + ...
2
votes
0answers
47 views

GLM for Poisson Regression for Soccer Ratings Not Converging

I have been trying to formulate a model of soccer matches to help me predict the outcomes. The model I'm trying to formulate involves using Poisson regression to assign attack and defence ratings to ...
2
votes
0answers
29 views

Can I adjust linear growth of a a subpopulation to a linear decay of the general population?

I need to estimate the amount of CF patients in Poland in the next four years. I have: estimations of the Polish population for the future years a CF patients' register for the last couple of years ...
2
votes
0answers
51 views

Show $\hat{\beta}$ and $s^2$ are independent?

I have the model: $y=X{\beta}+{\epsilon}$ I know $\hat{\beta}=(X'X)^{-1}X'y$ and that it is an unbiased estimator of ${\beta}$ and that $s^2=\hat{\epsilon}'\hat{\epsilon}/(n-k)$ and is an unbiased ...
2
votes
1answer
15 views

Problem with verifying variance of residual

I am supposed to show the following: $$Var(e_{ij}) = \sigma^{2}\left(1-\frac{1}{n_i}\right)$$ Where the follwing is known: $$y_{ij} = \mu + \alpha_{i} + \varepsilon_{ij}$$ $$e_{ij} = y_{ij} - ...
2
votes
0answers
26 views

Proof – OLS estimator regression [closed]

I am having trouble figuring out how I need to form and present an answer to a question. I completely understand the concepts of the math and analysis, I just don't understand how to give an answer ...
2
votes
0answers
132 views

How to reach Moore-Penrose pseudoinverse solution to minimize error function

Edit I'm trying to figure the derivation of the Moore-Penrose pseudoinverse for linear regression. The starting expression is the standard error function. I'm not quite sure how to expand on this ...
2
votes
1answer
160 views

Effects of feature scaling on weight vectors for linear regression

Given that linear regression or polynomial regression can be represented as: $\textbf{w} = (X^{T}X)^{-1}X^{T}Y$ It is standard practice in machine learning to scale each column in their training ...
2
votes
1answer
115 views

Machine Learning: Linear Regression models

I'm currently in a course learning about neural networks and machine learning, and I came across these two formulas in this textbook page on linear regression: 1) $y(x) = a + bx$ and 2) $y(x) = ...
2
votes
1answer
44 views

Predicting the increase/decrease of number

I have these entries in my database that looks like this: ...
2
votes
1answer
27 views

Multiple regression model

I have a multiple regression equation which as four quarters (maybe called them as parameters) ...
2
votes
0answers
66 views

Predicting profit with price variation

I am currently working on a high school project that aims to predict profit from X amount of items to Y amount of profit based off a deviated sale price. For instance: I sale 10 cookies for 10 ...
2
votes
0answers
92 views

Uni-variate Moving Average Theta coefficients

Consider the Uni-variate Moving Average Models (MA models) MA(1) $$x_t = \mu + w_t +\theta_1w_{t-1}$$ or the second order moving average MA(2) $$x_t = \mu + w_t +\theta_1w_{t-1}+\theta_2w_{t-2}$$ ...
2
votes
0answers
260 views

Normal equations for minimization of Frobenius norm least squares error

I'm having a hard time understanding the most efficient sequence of steps for deriving the normal equations for Frobenius norm least squares minimization. Here I want to minimize the norm of a matrix ...
2
votes
0answers
36 views

Best line fit for correlated points

Given in $\mathbb{R}^3$ are $n$ points $\mathbf{y}_i\sim N(\mathbf{y}_i-\mathbf{\hat{y}}_i, \mathbf{C}_i)$, which are normally distributed. I want to determine a best fit line $\mathbf{u}(\lambda) = ...
2
votes
1answer
55 views

Is there a site that will allow me to calculate a best fit for a set of data?

I have a bunch of x's and their corresponding y values, but do not have a Wolfram Pro account. Is there another site where I can input my dataset and have it spit out a best-fit regression (be it ...
2
votes
0answers
31 views

Showing Hat matrix equal specific values

Consider a one way layout model $y_{ij}$ = $\mu_i + e_{ij}$ (1 $\leq$ i $\leq$ a, 1 $\leq$ j $\leq$ $n_i$) where a = 3 and $n_1$ = 2, $n_2$ = 3, $n_3$ = 4. Show that the hat matrix for this design ...
2
votes
1answer
62 views

Trigonometric regression

What methods are performed for regression with trigonometric functions? E.g. : Sequence: $$-1, 0, 1, -1, 0, 1, \text{.....}$$ Regression: ...
2
votes
0answers
106 views

Is it compulsory to make transformation to the econometric model in order to have only diagonal elements on variance-covariance matrix of errors?

I need some sharped and advanced advices for the following issue ... Model and its assumptions I'm working on the methodology of a two-way error component model. Here is the model: $y_{jis} = ...
2
votes
0answers
109 views

What is ${\rm cov}(e_i, \hat y_i)$ in simple linear regression?

The model is $y_i = \beta_0 + \beta_1x_i + \epsilon_i$ What is ${\rm cov}(e_i, \hat y_i)$? What is ${\rm cov}(\epsilon_i, \hat \beta_1)$? What is ${\rm cov}(e_i, \epsilon_i)$? For 1, I am writing ...
2
votes
1answer
798 views

Understanding Regularization parameters in Machine Learning/Statistics

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...