Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
15 views

Is the regression model Identified? (is it possible to obtain a least squares estimator of the parameters?)

$y_t$ is the dependent variable, $x_t$ and $z_t$ are explanatory variables, and $α$, $β$ and $γ$ are unknown parameters. $y_t$ = $α$ + $β$$x^3_t$ + $γ$/$log$($x_t$) + $u_t$ $y_t$ = $α$ + $β$$x_t$ + ...
0
votes
2answers
25 views

Calculate $m_1, m_2 $ for $y = m_1x_1 + m_2x_2$

Given these values: $$x_1 = \left\{1, 3, 6, 8\right\}$$ $$x_2 = \left\{2, 8, 5, 10\right\}$$ $$y = \left\{8.6, 30.8, 34.1, 53.8\right\}$$ And this formula $$y = m_1x_1 + m_2x_2$$ How do I ...
0
votes
0answers
13 views

Can we see the beta coefficients in OLS as mean values?

Can we see the beta coefficients in OLS as mean values? I mean the estimator β alone. y=Xβ+ε
1
vote
1answer
13 views

Find the OLS estimator $β_1$ when a new variable is added to the regression

Suppose $y_t$ = $β$$x_t$ + $u_t$ , where t = 1, 2, ..., n. We know, in this case, the OLS estimator is $\hatβ$ = ∑$x_t$$y_t$ / ∑$x_t^2$ . Now suppose one more observation $x_{n+1}$ is added. At the ...
0
votes
0answers
11 views

Regression model, find $Var(y_i-\hat y_i)$

For the model of $y_i = \beta_0 + \beta_1x_{i1} + e_i$ for $i = 1,...,n$, where $e_i \sim N(0,\sigma^2)$ Find $E(\hat y_i)$ and $Var(\hat y_i)$. Hence or otherwise, find $E(y_i-\hat y_i)$ and ...
2
votes
2answers
47 views

Factoring a multivariate linear polynomial

I'm a computer programmer trying to solve a particular toy problem, and my understanding of linear algebra is far too lacking to solve it! I have a data set that can be modeled using this function: ...
0
votes
1answer
23 views

Multilinear fit vs Polynomial fit

I have a program that generates some physics data in 1D and 2D functions. In this program, the user defines a number of models that are used to compute a 2D function. That 2D function, and it's ...
1
vote
0answers
24 views

Predictive models

Given a set of temperatures of different cities for a month, which prediction model should I use for a two day look ahead prediction? Regression models or Time series?
0
votes
1answer
86 views
+50

The $\alpha$ estimation for the model $x_i = \xi_i \cdot \alpha$

We have $n$ sensors $X_i$ which estimate the scalar value $\alpha$ with different relative accuracies $\delta_i \ll 1$: $$ x_i = X_i(\alpha) = \xi_i \cdot \alpha, \ \ \ \xi_i \sim N(1, \delta_i) $$ ...
0
votes
0answers
17 views

Show that $E[g(X)u]=0$ in a standard regression model where $u = \hat{\beta}-E(\hat{\beta})$

Consider the standard regression model $y = X\beta + \epsilon$ where $y$ and $\epsilon$ are $(n \times 1)$ vectors and $X$ a $(n \times K)$ matrix. Let $\beta$ be any estimator of $\beta$. Let $u = ...
-1
votes
0answers
21 views

Let $Ax=b$ be a linear system where $A$ is of $m\times n$ dimensions and $m>n$, and $x$ and $b$ are both real vectors of $n$ dimension.

I want to prove that in system $A^T Ax_0$=$A^T b$, $x_0$ is the minimum point of the error that is defined by $e(x)=Ax-b$. I know that you have to minimize $\|Ax-b\|$ somehow but I don't know how to ...
1
vote
0answers
14 views

Population versions of multiple correlation coefficients and least squares estimates

I'm reading an old paper (Wold and Faxer (1957)) which considers the theoretical relation $$ y=\beta_1x_1+\cdots+\beta_hx_h+\zeta $$ where $y,x_1,\ldots,x_h,\zeta$ are (scalar) random variables ...
0
votes
1answer
17 views

rotating and exchanging x for y's in regression

I was just wondering what happens generally if i send all my x points to y's and y's to x's (i.e reflect along the y=x line) - if I change the x's and y's will my old error minimizing line still be ...
1
vote
0answers
14 views

var(AB) when A,B not independent

I need to find the variance of $\hat\beta_1 * \bar X_1 / \bar Y$ , where we have the regression equation Y= $\beta_0 + \beta_1* X_1 +…+ \beta_j* X_j$ I initially was thinking the answer is simply ...
0
votes
0answers
32 views

Fitting a equation to a spiral curve

I am completely new to this forum and also to this type of mathematical modeling. I am interested to fit the following equation to the points obtained from experimental data. I am looking for an ...
0
votes
1answer
27 views

Critical points of quadratic forms

Let $A$ be an $n\times n$ symmetric matrix, let $b$ be an $n$-vector, let $c \in \mathbb{R}$ and set $Q(x) = 1/2 x^T Ax-x^T b+c$. Prove that $x_0$, defined as a solution to $Ax_0=b$ is a critical ...
0
votes
0answers
10 views

t test vs f test

For conducting statistical tests concerning the parameter $\beta_1$ (the slope of the estimated linear regression function), why is the $t$ test more versatile than the $F$ test? This is a question ...
0
votes
0answers
9 views

Analysis of block-greedy algorithms for function approximation?

I consider the problem of selecting a final basis set $\{\phi_{c_j}\}_{c_1}^{c_n}$ approximation of function $f \in \cal{H}$ in a Hilbert space that minimizes $L_2$ error. One can use a greedy ...
2
votes
1answer
17 views

Three-Perpendicular Theorem for linear regressions

For a random vector $X=(X_1,\ldots,X_p)'$, we define $$ \mathcal{L}(X)=\{b_0+b_1X_1+\cdots+b_pX_p,b_0,\ldots,b_p\in\mathbb{R}\}. $$ The linear regression of the $q$-dimensional random vector ...
0
votes
0answers
37 views

Searching a function for data

I have a dataset and I am trying to find the appropriate function to fit them. So far, I have fitted the data into a two variable polynomial: $$ y(t,v)=(-525.958 + 4.88502 t - 0.0149025 t^2 + ...
-1
votes
1answer
13 views

For the three measurements b=0, 3, 12 at times t=0, 1, 2 find the best parabola y=C+Dt+E$t^2$

So I know how to do least squares regression using matrices to solve for Ax=b. I simply do $A^TAx=A^Tb$. However I don't really know how to account for the second power in a typical parabola ...
0
votes
1answer
19 views

What is the least squares solution given a line passes through original and following points?

So I am looking for the line y=Dt through the origin that fits the data y=4 at t=1, y=5 at t=2 and y=8 at t=3. This is what I have done so far. I know the three equations that are supposed to be ...
2
votes
0answers
19 views

Proof – OLS estimator regression [closed]

I am having trouble figuring out how I need to form and present an answer to a question. I completely understand the concepts of the math and analysis, I just don't understand how to give an answer ...
1
vote
0answers
17 views

consistency of OLS on misspecified AR(1) process

Suppose the true relationship in data is driven by AR(1) process as follows: $$X_t=\rho X_{t-1}+\epsilon_t\hbox{ , }|\rho|<1$$ and $\epsilon$ is a white noise of $(0,1)$ expectation and variance. ...
0
votes
1answer
16 views

Relation between Regularization and correlation

I was going through Chapter 3 (page 63 bottom) of Elements of Statistical Learning. While explaining regularization in ridge regression authors make the following statements. "When there are many ...
0
votes
0answers
11 views

What is the proper name of a model that takes as input the output of another model?

Thanks in advance for the help. I am writing a paper and for the life of me can't remember the proper term for a model that works as follows. rawData -> model1 -> outputModel1 -> model2 -> ...
0
votes
0answers
28 views

Finding better curved line of best fit

I have a set of hand generated data that follows somewhat closely to an exponential curve: I can come up with an exponential equation to the line that gives the values on the 3rd row, and Someone ...
0
votes
1answer
18 views

Calculating decreased cost with increasing quantity

I have a hand made table I've been using to give customers price per unit on my items, which gives a better price for the more items that they buy. My sample table right now I need to keep the ...
1
vote
1answer
41 views

Solution of overdetermined polynomial system

Some of you will find this question pretty straightforward to answer, but I desperately need some help in solving a problem involving several equations and 2 unknowns, for an engineering application. ...
0
votes
0answers
16 views

Standard deviation errors in log scale

I have a not so common issue with error bars in the log log scale. To be more precise, I have measurements of a quantity Y with an associated standard error Yer that has normal distribution and these ...
1
vote
0answers
65 views

How to reach Moore-Penrose pseudoinverse solution to minimize error function

Edit I'm trying to figure the derivation of the Moore-Penrose pseudoinverse for linear regression. The starting expression is the standard error function. I'm not quite sure how to expand on this ...
0
votes
3answers
159 views

Program to find closest function to fit arbitrary data

I've wanted this for years, but have never come across anything; a program for Windows to find the closest function to fit arbitrary data. The data I feed it is simple: A table with two columns ...
1
vote
1answer
25 views

Effects of feature scaling on weight vectors for linear regression

Given that linear regression or polynomial regression can be represented as: $\textbf{w} = (X^{T}X)^{-1}X^{T}Y$ It is standard practice in machine learning to scale each column in their training ...
0
votes
3answers
44 views

Will someone explain this polynomial regression equation?

I am in high school and I need to write a program that does polynomial regression to any degree on a set of data for a personal project. I think that this Wikipedia Article has the equation that I ...
0
votes
0answers
15 views

How to find the closest integer linear equation to given real linear equation

I am given a set of points in an n-dimensional plane. I want to find the closest (lowest co-variance) integer linear equation that characterizes the points. I find the real linear equation using r^2 ...
0
votes
1answer
21 views

Invertibility of $X^TX$ when sever multicollinearity in regression

I am studying about multicollinearity in regression and in the book it says, "if there is severe (but not perfect) multicollinearity, two or more predictor variables are highly correlated, so $X^TX$ ...
1
vote
0answers
33 views

Notating the components of the $\hat{\beta}$ matrix when $\hat{Y}$ is multidimensional

This is a question on The Elements of Statistical Learning. We have from the linear model $$\hat{Y} = X^{T}\hat{\beta}$$ where $\hat{Y}^{T} = \begin{bmatrix} \hat{Y}_1 & \hat{Y}_2 & \cdots ...
0
votes
1answer
17 views

How to derive this solution to this minimization problem in vector form?

We want to minimize the mean squared error $$ \sum_{t=1}^n (y_t - \theta^T x_t - \theta_0)^2. $$ Letting $X = [x_t, 1]$, we can rewrite the above problem in vector form as $$ \sum_{t=1}^n (y_t - ...
1
vote
1answer
16 views

Correlated explanatory variables in linear regression

Is it any reason to assume that if two strongly correlated explanatory variables have impact on response that regression coefficients for these variables have the same signs ? Could such assumption be ...
0
votes
0answers
17 views

Least squares: Calculus to find residual minimizers?

Reading a section on simple regression in "An Introduction to Statistical Learning with Applications in R" I got a question on residual sum of squares minimization. Quoting from the book: [quote] ... ...
2
votes
1answer
33 views

Machine Learning: Linear Regression models

I'm currently in a course learning about neural networks and machine learning, and I came across these two formulas in this textbook page on linear regression: 1) $y(x) = a + bx$ and 2) $y(x) = ...
0
votes
0answers
21 views

Mutiple Regression, calculating R-squared

If I have two regressors in multiple regression equation y=b0 + b1*X1 + b2*X2, how can I find R-squared for the model?I need to know the written formula(not in excel) for two independent variables as ...
0
votes
0answers
46 views

Fitting an ellipse such that the ratio of its radii is in a range

I need to fit an ellipse to a group of points. However, I have an issue and I appreciate if anyone can help me. The issue is that I need to have the fitted ellipse such that the ratio of its radii is ...
0
votes
1answer
77 views

Example of a real-world situation where multivariate analysis is applicable.

I have searched a lot of site to understand the situation where multivariate analysis is applicable. But not got any easily understandable example. Would you please give me a real-world example where ...
0
votes
0answers
21 views

Application of Multivariate Analysis

The following situation is proven valuable where multivariate analysis can be applied. This example is taken from the book Applied Multivariate Statistical Analysis ...
0
votes
1answer
32 views

Linear regression with constrained weights

I have a set of $n$ linear combinations, each with $m$ parameters and desired value $b$. I want to find the set of weights $w$ which minimizes the total equations distances (e.g. the sum of distances ...
0
votes
0answers
14 views

Chi sqaured table for degrees of freedom 616?

In order to check heteroskedasticity, we use the White's test. I tried to follow this method below, however, could not find a table with df=2016 and 95,5% confidence. I don't understand how we get ...
0
votes
1answer
24 views

estimate coefficients of $y = \alpha x + \beta y + \gamma z + \epsilon$

I know how to find $m$ and $b$ for $y= mx +b$, which is : $m= \frac{\bar{x}\bar{y}- \bar{xy}}{(\bar{x})^2 - \bar{x^2}}$ and $b= \bar{y} - m\bar{x}$ How can we estimate $\alpha, \beta, \gamma$ and ...
0
votes
0answers
32 views

how can I find outliers in a 2 vector data set

I have a two data set $(X,Y)$ where $X$ represents the angles and $Y$ represent the signals. $X$ is always correct because I increment it by coding $x=x+1$. However, $Y$ could be sometimes wrong ...
2
votes
2answers
30 views

Likelyhood function analysis

I've done some calculations on a large number of data, and created the following graph in excel representing the data: How do I go about analysing this regression in order to find the formula that ...