Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
0answers
27 views

what does is mean by 'overfitting' of data?

I have the following equation that is to be estimated: y = a + bA + cB + dC + eF + dG + e and i got 2 other additional variables, fH and gI, that i do not wish to add in.. can i reason this out by ...
2
votes
2answers
48 views

Fitting two parallel lines to a set of points

In two dimension I have a set of points X = $\{x_1,..., x_N\}$. I want to fit two parallel lines to these points like $l_1$ and $l_2$ $$l_1 = p_1 + \lambda n^\perp$$ $$l_2 = p_2 + \lambda n^\perp$$ ...
0
votes
0answers
18 views

distribution of linear regression predicted values

Let $\tilde{y}_*$ the prediction for a new observation at level $X = x_*$. Assume that $\sigma^2$ and $\tilde{y}_*$ are independent. Show that $$\frac{\tilde{y}_* - y_*}{se(\textrm{pred})(\tilde{y}_* ...
0
votes
0answers
23 views

3 variables 'linked' to one outcome (for lack of better words)

Paper route salary formula (reverse engineer...again for lack of a better word;) Ok suppose I have these variables: Pieces of mail : 302 Number of stops: 177 TotalWeight* : 272 Hours allocated: ...
0
votes
0answers
39 views

how to compute joint pdf of (x,y) where y is correlated with x

I want to know how to compute joint pdf of (x,y) where x follows exponential distribution with parameter $\lambda$ and $y=x+\epsilon $, $\epsilon$ follows standard normal distribution. x and ...
1
vote
1answer
37 views

Corollary of the Frisch-Waugh Theorem

Consider the following linear regression model: $$y=X \beta + \epsilon = X_1 \beta_1 + X_2 \beta_2 + \epsilon $$ Where we have $n$ obsevations and $k$ variables, and hence $X$ is a matrix $nk$, and ...
5
votes
0answers
30 views

single variable is significant but overall test is not

I do a multiple regression with 3 independent variables $X_1$, $X_2$ and $X_3$. The correlation between $Y$ and $X_1$, $Y$ and $X_2$, and $Y$ and $X_3$, are each large and statistically significant. ...
1
vote
1answer
22 views

Regression Model with (Y,X) non-random?

In regression, we assume that $(X,Y)$ are random variables following some certain distribution. How would the problem change if we do not assume $(X,Y)$ are randoms. Why can we just have ...
1
vote
1answer
48 views

Calculate power regression

I've searched for quite a while, but google has very little on what I actually need. I have a series of points that form a power function. How do I calculate the what the power function is? I can ...
0
votes
1answer
28 views

Doubt about proving if number of parameters equals to sample size, the error sum of squares is zero

Consider the linear model $Y=X\beta+\varepsilon$, where $Y$ is an $n$ by $1$ vector, $X$ is a known $n$ by $p$ matrix, and $\varepsilon$ is an $n$ by $1$ vector of random errors following normal ...
0
votes
0answers
32 views

Can a supervised learning technique perfectly learn a function?

Obviously the goal of a supervised learning algorithm is to learn some function, $f : x \mapsto y$. In essence, we want $f$ to be a be a good predictor of $y$ when given $x$. I'm curious if a ...
1
vote
2answers
49 views

Calculus derivation of OLS regression formula

Edited to make the question more clear. In deriving the OLS formula from calculus, that is, solving $$\min_{\beta}(Y-X\beta)^T(Y-X\beta).$$ How could be guarantee the solution is actually the ...
5
votes
1answer
213 views

How to prove SSE and SSR are independent

Consider $Y=X\beta+\varepsilon$, where $X$ is n by p, $\beta$ is p by 1 and $\varepsilon$ is n by 1 with covariance matrix = var($\varepsilon$)=$\sigma^2 I$. Give expression for the regression and ...
0
votes
0answers
44 views

Deriving the slope variance of linear regression line through the origin

I have derived the value of the slope of a linear regression line (through the origin) to be to be: I am having difficulty deriving the variance of the slope! Any help is appreciated!
1
vote
1answer
272 views

Linear Regression Model, linearity in parameters/ variables

I am confusing with the wording here. I was reading a book on linear regression. "The primary concern for linear models is that they display linearity in the parameters. Therefore, when we refer to a ...
1
vote
1answer
147 views

Is the sum of residuals in the weighted least squares equal to zero?

So I know that in OLS, the sum of the residuals is equal to zero. This makes sense. I also know that given any slope parameter its possible to rescale the intercept to where the sum of the u will be ...
1
vote
0answers
50 views

LMS Update Rule

I'm starting to study machine learning using Andrew Ng's class notes. I understand conceptually how linear regression works, but am having trouble with this equation: $$ \theta_j := \theta_j + ...
1
vote
0answers
19 views

What to do about Missing values for Multivariate regression analysis

I am required to perform multivariate analysis on all countries in the World bank database regarding digital divide. I am confused becasue when I look at factors such as School enrollment, there are ...
2
votes
0answers
15 views

Problems with Exchange Procedure in Remez Algorithm

So first off: *** This code is not being used in production software. It is a personal project of mine, trying to understand approximation theory and advanced curve fitting. In other words, I'm ...
2
votes
1answer
147 views

Solving for a 3D point in a 5D graph given 3 pairs of 2D points.

I am attempting to solve the values $C$, $D$, and $S$, given three pairs of $[M,R]$. $$R = \frac {M}{C - MDC + DC\left(MS\right)^2}$$ I have been able to solve for a related equation (or rather, ...
5
votes
1answer
53 views

Why do polynomial regressions have larger variance at the end?

In reading the book "An Introduction to Statistical Learning with Applications in R", I came across this graph: It shows that the point-wise variance is larger at the ends of the regression curve. ...
0
votes
0answers
35 views

SSE distribution in simple linear regression

I'm looking at the typical simple linear regression model $Y_i = \beta_0 + \beta_1X_i + \epsilon_i$, where there $\epsilon_i$s are iid $N(0, \sigma^2)$ random variables. We have unbiased estimates ...
2
votes
1answer
42 views

Optimal place to measure for simple linear regression/fitting

Suppose I have a linear model $y_i=a\cdot x_i+b+\epsilon_i$, where $y_i,\epsilon_i,a,b\in\mathbb{R}$, $x_i\in[-1,1]$. I can take n measurements of $y_i$ at $x_i$, where $n\in\mathbb{N}$. $\epsilon_i$ ...
0
votes
1answer
31 views

Multiplicative version of Maclaurin or Talyor series

Is there a multiplicative version of Maclaurin or Talyor series? May be in the format $\ln y = b_0 + b_1 \ln x + b_2 (\ln x)^2 + \cdots $ I want to use that as an approximation in a regression ...
0
votes
0answers
9 views

regression relationship

with data of two variable(say X and Y ),My math teacher gave me two formulas -one for finding Y for given value of X $$ Y- \bar Y= B(X-\bar X)$$ -another for finding X for given value of Y $$ X- ...
0
votes
0answers
58 views

General formula for leverage in a simple regression model using a single binary covariate

Need some help to start derive the general formula for leverage in a simple regression model using a single binary covariate? Consider a data set where the variable X takes only 0 or 1 values (e.g., ...
1
vote
1answer
41 views

Do 'X' and "y' have 'zero' correlation , or can be anything between '-1' and '+1'?

let , we have bi-variate data on X and Y . Now corresponding to the value $x_0$ , y can take any value.but for all other values of x , y takes a constant value. what will be the correlation ...
0
votes
0answers
24 views

What are some possible reasons for a large condition number?

For this question, please assume that I am talking about the condition number with respect to the spectral norm. That is, $\kappa_2(A) = \|A\|_2\|A^{-1}\|_2 = \frac{\sigma_{max}(A)}{\sigma_{min}(A)}$. ...
0
votes
0answers
27 views

Why is $\beta$ a linear combination of $\epsilon$

I have a multiple linear regression question. Why is $\beta$ a linear combination of $\epsilon$? I don't know how one comes to this conclusion. Any help will be greatly appreciated.
0
votes
0answers
21 views

Dummy variables

In the following regression, i want to predict if a new fund depends on their previous old fund returns, y= a + bTYPE + 0.24 Size + 0.2 Leverage + epsilon where TYPE is a dummy variable, 1 for a new ...
1
vote
0answers
26 views

How to compare experimental data with teorethical prediction

I would like to know, what is the method to approximate experimental data to teorethical one. I have heard about polynomial regression. After calculating particular matrices and solving set of ...
1
vote
1answer
29 views

unequal sample size and linear regression

Can i use simple linear regression when there is unequal sample sizes for the dependent and independent variable? Will this be a problem? That is; y=a+bx+epsilon, where y and x have unequal sample ...
1
vote
1answer
48 views

Sum of random error in regression

If you know that $\sum_{i=1}^n e_i=0$.What can you say about $\sum_{i=1}^n\epsilon_i=0$? Where $e_i=Y_i-\hat{Y_i}$ and $\epsilon_i=Y_i-E[Y_i]$. I know that $$Y_i=B_0+B_1X_i+\epsilon_i$$ and ...
0
votes
1answer
24 views

Polynomial Fitting of Circular Data Object

This is a very odd question. I have a one dimensional data set that is graphed on a histogram. I am trying to curve fit this data set (using the class midpoints as the x values, and the frequencies as ...
0
votes
0answers
81 views

I would like to Reverse Engineer a formula, given a set of coordinates.

I'm trying to find the original function without trial and error, also I know that every single function can be represented as a polynomial function but I am not looking for a best fit as much as an ...
0
votes
0answers
21 views

Comparison of performance

i am testing for a sample of fund performance against a benchmark sample of funds performance (i.e. a 'peer' group).each benchmark is calculated one for one that corresponds to each fund strategy and ...
0
votes
0answers
71 views

How to derive one matrix algebra equation from another

I have one matrix equation: s = A' X^-1 A (where s is scalar, A is a vector, A' is its transpose, and X^-1 is the inverse of a square, symmetric matrix) Which can be transformed into another ...
1
vote
2answers
44 views

Prove that $E(|(X+Y)(X-Y)|) \leq 2\sqrt{1-\rho²}$, where $\rho$ is correlation.

For two random variables $X,Y$ with mean $0$ and variance $1$, their correlation is $\rho$. We have to prove that $$E(|(X+Y)(X-Y)|) \leq 2\sqrt{1-\rho^2}.$$ But, I can't understand how the $\rho$ ...
0
votes
0answers
10 views

Error in Joint Modelling in R

I have a dataset which has the following variables: sex (indicator), risk (indicator), and age. I've done cox models with these variables no problem and even basic joint models with them no problem. ...
0
votes
1answer
26 views

Can a dummy variable help me in a linear regression where my slope changes based on that variable

Sorry if that wasn't a particularly helpful title, let me explain the situation. If I have a scatter chart of y ~ x1 and I notice that if I filter the data by tiers of a different variable, my slope ...
1
vote
1answer
29 views

How does measurement error affect the standard error of regression coefficients?

How does inclusion of the measurement error in the model, as $$Y_i + \varDelta_i = bX_i + \varepsilon_i$$ affect the standard error of least square estimators $\hat{b}$ of coefficients $b$? If I ...
0
votes
1answer
97 views

How to propagate uncertainties in the dependent variable when doing linear regression?

Let's say I have an independent variable $\vec x$ and a dependent variable $\vec y$ and measurement errors on my dependent variable that I know to be $\delta y$. For the sake of simplicity, let's say ...
1
vote
3answers
55 views

Is it possible to find initial parameters when fitting triple exponential term function to data?

I'm trying to fit $f(x) = A \exp(Bx) + C \exp(Dx) + E \exp(F x) $ to data. I can finish off the fitting using Levenberg-Marquardt, but I'd like to find a quick way to calculate initial parameters. ...
0
votes
1answer
27 views

Why or how can we say errors(residuals) are independent and they follow the normal probabilties in regression analysis?

While I am studying linear regression analysis and I have encountered a sentence salying "errors are independent and follow normal probabilities". I can only guess what it says but I can't trust my ...
0
votes
1answer
23 views

Simple Regression

The question asks for the slope and intercept, but I don't have a correlation coefficient or the raw data (just sample size, mean, and standard deviation).
1
vote
1answer
25 views

Linear regression relationships

Velocity $= X$, distance to stop $= Y$ $\beta_0= -17.5791$, $\hat{\operatorname{se}}(\beta_0)=6.7584$ $\beta_1 = 3.9324$, $\hat{\operatorname{se}}\beta_1 = 0.41.55$ degrees of freedom $=48$ (a) is ...
0
votes
0answers
34 views

Functon fitting goes wrong

Let's say I got a some function (let's say it named $B_w$) and I make a curve deped on some parameters. As example ...
3
votes
2answers
39 views

Distance between a plane and set of points

Suppose $m$ data points belonging to a class represented by matrix $A$. Therefore, the size of matrix $A$ is $m\times n$. In addition, suppose $w\cdot x + b=0$ be equation of a plane in ...
0
votes
0answers
13 views

Is there a standard formula for calculating a project portfolio elasticity?

My project portfolio has a lot of variable numbers in it which could drastically change the cost, schedule and resource requirements of the portfolio. I was wondering if there is already some kind of ...
2
votes
0answers
25 views

combining multiple regression outputs

Suppose I have multiple regressions, along with their r-squares, standard-errors, etc.: $y(t) = \alpha_1 + \beta_1 x(t) + e_1$, where $t \in (\tau_0, \tau_1)$ $y(t) = \alpha_2 + \beta_2 x(t) + ...