Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
16 views

Understanding polynomial regression

I'm looking for a good tutorial on how to calculate a "line of best fit" for non-linear data. I found this site: http://easycalculation.com/statistics/learn-regression.php which gives a very good ...
0
votes
1answer
21 views

Why is $E(u)=0$ when an intercept is included in OLS Estimation?

I am reading Wooldridge's graduate econometrics text. There he states that when estimating the equation $y=\mathbf{x\beta}+u$ by OLS, if an intercept (constant term) is included in your $\mathbf{x}$ ...
0
votes
0answers
23 views

OLS standard error that corrects for autocorrelation but not heteroskedasticity

Question: By mapping the OLS regression into the GMM framework, write the formula for the standard error of the OLS regression coefficients that corrects for autocorrelation but not ...
0
votes
0answers
11 views

How to find Expected Values of errors in OLS?

I am learning OLS and we are asked to show/explain whether the following statements are true or false. We are given that $\hat{U}= Y-\beta X$ and $E(U|X)=0$. I am able to do this for the first two ...
0
votes
0answers
15 views

Why use the expected value of y, E(y), in simple linear regression

I am learning about linear regression and I have ran into a bit of confusion. I'm trying to relate what I've learned in my probability and mathematical statistics course (in particular, expected ...
2
votes
1answer
22 views

Isn't the Hat Matrix just an identity matrix?

In Linear regression $y = X\beta + \epsilon$ The Hat matrix is defined to be $H = X(X^TX)^{-1} X^T$ . However. If I compute the equation for Hat matrix, I just get an identity matrix. My calculation ...
0
votes
0answers
5 views

Predicted value of polynomial regression models

Suppose that we have a polynomial linear regression as following $$ Y_i = \beta_0 + \beta_1 X_{i} + \beta_2X_{i}^2 + \epsilon_i, \quad i=1,\ldots, n $$ with $\epsilon_i \sim N(0,\sigma^2)$ and ...
0
votes
0answers
14 views

Analysis of variance question ($\sum(\hat{y}-\bar{y})\hat{e}_t=0$)

this is from an econometrics textbook, looking at how to arrive at the analysis of variance table for a linear regression: $\sum (y_{t}-\bar{y})^{2}$ =$\sum [(\hat{y_{t}}-\bar{y})+\hat{e_{t}}]^{2}$ ...
1
vote
1answer
22 views

Linear Regression with independent but non-identical noise

If I have this linear regression equation: $$y=X\beta+\epsilon $$ ($x$ and $\beta$ are vectors) The likelihood function can be written as $$L= \prod_{n=1}^N N(y_n ;x_n ,\beta ,\sigma^2)=(2\pi ...
0
votes
1answer
24 views

Exponential least squares biased to small y?

I'm working on fitting an exponential decay curve to a data set. While searching for techniques I found Wolfram's page describing how to easily accomplish it by taking the natural log of the ...
0
votes
0answers
52 views

weights go to infinity in logistic regression with linearly separable data

I have the loss function of logistic regression $L(W)$ = - $\sum_{i=1}^n {y_i}.log[\sigma(w^Tx)] + {(1-y_i)}.log[1- \sigma(w^Tx)]$ I have derived the Hessian and proven it's positive semi-definite ...
0
votes
0answers
17 views

Likelihood Functions of Nonparametric Simple Regression

I'm trying to find the likelihood function of a nonparametric simple regression model. Nonparametric statistics is new to me however, so I'm having some trouble wrapping my head around some of the ...
0
votes
1answer
32 views

More variables = better fit?

When fitting (let's say) a linear regression model, it is always true, that the more variables we include in our model, the better fit is (in R^2 sense)? I don't want to discuss here overfitting, ...
0
votes
0answers
13 views

Is there any easy way to QR-decompose two related matrices?

Matrix A and B are related in that: $A = W_1X$ and $B = W_2X$ Where $W_1$ and $W_2$ are diagonal matrices with all the same entries (but not in the same order). I am asking for the purpose of ...
1
vote
1answer
32 views

Prove that the the variance estimator $\widehat{\sigma}^2=MSE/(n-2)$ is biased is the simple linear regression model

This is in scope of the simple linear model. Im trying to prove that $\mathbb{E}\left(\widehat{\sigma}^2\right) = \sigma^2$ for $$\widehat{\sigma}^2 = \frac{1}{n-2}\sum^n_{i=1} ...
0
votes
0answers
18 views

How to solve this linear algebra equation / regression?

$Y = X D$ and $Y$ is $n\times 1$ known matrix. $X$ is $n\times k$ unknown matrix. $D$ is $k\times 1$ known matrix. How to solve for $X$?
3
votes
3answers
43 views

linear solution of curve fitting on multiple linear functions differing by a multiplier

I recently posted this question here but I thought this could be of interest also in mathematics, given I found a partially related question here I am facing the following problem. I know nonlinear ...
0
votes
1answer
44 views

Examining the effect of a quantitative factor on response.

To examine the effect of a quantitative factor temperature on yield,the researcher has a plan to use the following model for the analysis: $$y_{ix}=\beta_0+\beta_1 x+\epsilon_{ix}$$ where $y_{ix}$ ...
0
votes
1answer
20 views

Constructing a while loop in R for Newton's method

I'm very, very new to R, and my instructor's example seemed like a special case (or I just don't know how to extrapolate his syntax to my problem). Here's the example we used: Approximate the square ...
1
vote
1answer
23 views

Least Squares Regression Matrix for Rational Functions

So first off no, this isn't a homework problem. Second, I'm trying to understand how this works, NOT find a program that will do it for me. Okay so I've known for a while how to use Gaussian-Jordan ...
0
votes
0answers
12 views

How to find a mapping function from n dimensional space to m dimensional space

What are different methods/approaches to find a function $T:x_1 \to x_2$ that minimizes a cost function defined as: $\text{cost}(T) = | Q(x_1) - P(x_2) | = | Q(x_1) - P(T(x_1) |$ where, $x_1$ has ...
1
vote
1answer
19 views

Regression Model with even function?

Is there any method to test if the mean function, $f(x)$, of a regression model $y=f(x)+\epsilon$ is even or not?
1
vote
0answers
16 views

Orthogonalization of Variables

Let's assume we have three collinear variables/factors, $X_1$, $X_2$, $X_3$. Would there be a method to orthogonalize these variables in a simultaneous way: in other word, orthogonalizing them in such ...
0
votes
0answers
21 views

What is distinction between Functional Linear Regression and Functional Linear Models?

Thanks in advance for the help. I want to make sure that I understand two concepts correctly. A functional linear model is a particular type of linear model while functional linear regression is the ...
0
votes
1answer
23 views

Given a sample of input/output data, predict new outputs

My problem is the following : I have a number of inputs with the corresponding deterministic outputs. There is no error on either input or output. The link between the two is completely unknown to me. ...
1
vote
2answers
31 views

Errors and Residual

Why are errors independent but residuals dependent? As far i know the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also ...
1
vote
2answers
26 views

Simple linear regression seems off

I have the following datapoints: $$p1(52,730)$$ $$p2(53,409)$$ $$p3(52,250)$$ $$p4(52,90)$$ Now I want to find the best fitting line between these points. When I use simple linear regression I get $$y ...
0
votes
0answers
15 views

Random variable variance

I have the model yi=β1+β2Xi+ui where ui∼iid N(0,σ2). I estimate β1 and β2 by drawing a straight line between the first (x1,y1) and last dot (xn,yn). So, β̂ 2 will be the slope of this straight line. ...
3
votes
1answer
43 views

Weighted least squares with angular data

Suppose I have a system whose state is $\Theta=(\theta_1,\theta_2,\ldots,\theta_n)$, where $\theta_i\in[-\pi,\pi)$ (i.e., they are angles). I'd like to determine the most likely estimate of $\Theta$ ...
3
votes
2answers
101 views

Regression with error coming from rounding

I am looking at the following model: $c$ is a fixed vector in $\mathbb{R}_+^n$ and for any $x \in \mathbb{R}_+^n$ we obtain a value $y =[c^Tx]$, i.e. rounding $c^Tx$ to the nearest integer. I want ...
0
votes
0answers
22 views

numerical (script) fit of function with 2 arguments

I would like to find the least-square fit for a 1D-function that takes two arguments. m(x,y) = d * (x-x0)^2 / (y-y0)^2 I would like to write a c++ routine to ...
1
vote
2answers
32 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
2
votes
1answer
46 views

Is there a site that will allow me to calculate a best fit for a set of data?

I have a bunch of x's and their corresponding y values, but do not have a Wolfram Pro account. Is there another site where I can input my dataset and have it spit out a best-fit regression (be it ...
0
votes
1answer
17 views

Minimum number of observations in non-linear regression

Are 5 observations enough to verify the following non-linear regression model in the form: $ Y= C K_0^{\alpha_0}K_1^{\alpha_1}K_2^{\alpha_2}$ And in general how many observations do I need for models ...
0
votes
0answers
15 views

factor models and using cross sectional regression?

I have been doing some reading on factor models. In the literature it mentions that when creating a portfolio that maximises particular attributes it may lead to unwanted bias to other factors. I ...
0
votes
2answers
51 views

Equation going infinitely towards y = 10

I'm programming a site on which I sell services, and the more the customers spends, the more discount they will have. Please have a look at the diagram below. Spending 700 USD will result in 5% ...
1
vote
1answer
63 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...
1
vote
0answers
13 views

Non-Linear Regression involving the maximum function

How do you calculate the regression of this model? I know Minitab and MATLAB, so if you guide me with these software I would totally appreciate it. $$Y=c+\max(X^{-n}, 0.34) $$ Here c and n are ...
1
vote
3answers
51 views

Linear regression throug the origin versus mean?

Assume that I have data that can be described by: $y_i = \beta x_i + \epsilon_i, \epsilon_i \sim (0,\sigma_{\epsilon})$, then the least squares estimator is given by $\hat{\beta_1} = ...
1
vote
2answers
129 views

How do I find equation of this curve?

I need to find equation of the curve as shown below, for which, I need to find equation for upper part. lower part is half circle. upper part is a constant distance from circle with line passing ...
1
vote
0answers
18 views

Good MSE doesnt imply good prediction in logistic regression?

I am writing some code for regularized logistic regression. I observe this interesting phenomena and wonder if it is a normal thing or just my code is wrong. For loss function, I am using the ...
0
votes
0answers
17 views

Nearest points / residuals on a total least squares parabola

Consider fitting a parabola $y = a + bx + cx^2$ to 2d data $X_i, Y_i$ with noise in both X and Y, using the the singular value decomposition as in Total_least_squares (TLS): $\qquad X = [ 1\ \ Xdata\ ...
1
vote
1answer
28 views

time-series regression with missing data

I have a regression as follows for time-series data (e.g. stock prices versus other variables): $$ Y = b \cdot X + b_1 \cdot X_1 + e$$ where $X_1$ will be missing based on pre-determined dates ...
2
votes
0answers
11 views

Showing Hat matrix equal specific values

Consider a one way layout model $y_{ij}$ = $\mu_i + e_{ij}$ (1 $\leq$ i $\leq$ a, 1 $\leq$ j $\leq$ $n_i$) where a = 3 and $n_1$ = 2, $n_2$ = 3, $n_3$ = 4. Show that the hat matrix for this design ...
0
votes
1answer
66 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
0
votes
2answers
44 views

How to do non-linear regression with this function

I have observed that my data matches the function : $ a e^{bx}+c $ I want to get the parameters a ,b and c. I know how to solve this problem if c equals 0. But how to solve it when c involves in?
0
votes
1answer
39 views

Regression with Mean, Standard Deviation, Range and Correlation

A research team collected data on students in a statistics course. Their dependent variable was the student’s score on the final examination, which ranged from 200 to 800 points. The observed average ...
0
votes
0answers
11 views

Finding rounding rules underlying rounded function results

Here's the problem I've been stuck on for two weeks now. I basically have a black-box function, which takes into input an integer and returns an integer, and I'm trying to determine it. Here are the ...
0
votes
1answer
37 views

4 points, how to know if it's growing over time?

I've an array of 4 points, which formula should I use to detect their growth ? ...
0
votes
1answer
19 views

Deriving estimators for the parameters a and b that minimize the random error - setting up linear regression variables?

I'm reviewing old notes, and I know I solved this way back when, but can't remember how to know: Consider the simple linear regression model: $$Y_i = a + bX_i + \epsilon_i$$ where $Y_i$ is the ...