Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
37 views

Wolfram Exponential Fit not match as formulas

I am using WolframAlpha Exponential-Fit formulas to find equation of Exponential Regression http://mathworld.wolfram.com/LeastSquaresFittingExponential.html but after implementation, I tested with a ...
0
votes
0answers
17 views

multicollinearity with intervals

You have multicollinearity when you have 2 variables (X1,X2) that have a relationship, X1=a+X2 where a is constant. My question is: is there still a multicollinearity issue if a is not constant, ...
0
votes
0answers
39 views

Multiple Linear Regression sample problem:

I am currently studying linear regression, but I am not sure I understand everything correctly. I was trying to solve some of the exercises at the end of my book, and I picked a random one below. I am ...
1
vote
1answer
51 views
0
votes
0answers
16 views

Simple Linear Regression variance question

I was wondering if someone could help me clarify the following statement from my lecture notes: The within-sample variance of $Y$ is \begin{equation*} Var(Y) = Var(\hat{\alpha} +\hat{\beta} X + ...
1
vote
1answer
157 views

Intuition and the math behind normalization

What exactly is the purpose of normalization. From what I read, it is to adjust two different sets of values so you can compare them, but I don't understand why, nor the math behind it. Could anyone ...
1
vote
1answer
27 views

Simple linear regression prove variables are uncorrelated:

I am working on the following problem: In a problem of simple linear regression, $$Y = \hat\beta_0 + \hat\beta_1 x(bar),$$ show that the random variables $\hat\beta_1$ and $Y$ are un-correlated (All ...
0
votes
1answer
66 views

Convexity of LASSO

I would like to know if some variables in design matrix are correlated then LASSO is convex or not. If you give me a proof for convexity of LASSO and ADAPTIVE lasso, I will be thankful. LASSO is ...
0
votes
0answers
27 views

Multiple regression - interpretation of coefficients

Assume that one has two input variables (X1, X2) and one output variable (Y). One can approach regression in two ways: One can first run a univariate regression between X1 and X2, have a residual ...
0
votes
0answers
23 views

Variance of regression coefficient

The formula per below gives the regression coefficient under OLS. the textbook that i am using (Elements of statistical learning) subsequently states that the variance of B is as per below (with ...
1
vote
0answers
22 views

Proofs on regression analysis

How can I prove: 1) estimating population variance $\hat\sigma^2={1 \over n-2}[S_{YY}-{S^2_{XY} \over S_{XX}}]$. 2)expected value of error mean square=$E(EMS)=\sigma^2$ To prove (2): I showed that ...
0
votes
2answers
56 views

Is the reference point (x, y) above or below the non-linear equation?

BACKGROUND In short, I have a series of 3 to 10 data points that will be used to represent a curve. For example: $X=0, Y=10$ $X=4, Y=7$ $X=9, Y=12$ $X=16, Y=10$ What I am trying to do is ...
2
votes
0answers
63 views

What is ${\rm cov}(e_i, \hat y_i)$ in simple linear regression?

The model is $y_i = \beta_0 + \beta_1x_i + \epsilon_i$ What is ${\rm cov}(e_i, \hat y_i)$? What is ${\rm cov}(\epsilon_i, \hat \beta_1)$? What is ${\rm cov}(e_i, \epsilon_i)$? For 1, I am writing ...
0
votes
0answers
22 views

Estimating variance in least squares in regression

How to show $\hat\sigma^2=$$1\over n-2$$[S_{yy}-$$S^2_{xy}\over S_{xx}$]. I don't know the matrix form in regression.I don't even understand how to begin this
1
vote
1answer
29 views

properties of least square estimators in regression

$Y_i=\beta_0+\beta_1 X_i+\epsilon_i$ where $\epsilon_i$ is normally distributed with mean $0$ and variance $\sigma^2$ . The least square estimators of this model are $\hat\beta_0$ and $\hat\beta_1$. ...
1
vote
0answers
20 views

Problem on Linear Regression

Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean 0 and variance 1; $y_i = α + β(x_i − \bar x) + e_i , i = ...
1
vote
1answer
10 views

Linear regression or ANOVA with unordered independent variable

I have a set of data, let's say describing a group of people. Let's say we know their income and color of hair: ...
3
votes
1answer
33 views

Logarithmic Functions algebra question

This is my first post and I honestly just want a second opinion on my answer to a question I got incorrect on an exam before I go arguing over it with my professor. Basically, is this mathematically ...
1
vote
1answer
30 views

clustering of singular values

let us consider following graph of singular values i want to make some kind of clustering of these data,namely to seperate main components from non main components,let say signal components ...
0
votes
0answers
41 views

Curve fitting a sinusoidal function

I need to fit the function $y = a\sin(\pi x)$ to these points: $[0,0], [0.5,a], [1,0]$ and another given point $p_0$. Please guide me through tackling this problem, because I don't know what to do. ...
0
votes
3answers
101 views

Reliability of linear regression to predict future

When we have a set of data, where X is the cause, and Y is the effect, we can use linear regression to predict values for Y, based on values of X. I have learned that you may only safely apply this ...
0
votes
1answer
51 views

exponential regression fit error problem

I have the following data and im trying to get an exponential fit. Ive tried a variety of different tools for this, which all seem to give quite a large error margin at the top of the curve. Plotting ...
0
votes
1answer
36 views

reverse a logarithm

I have some data which produces the following logarithmic curve. As you can see, the curve produces the exact opposite of what Im trying to achieve (my data is the line with dots, the logarithm is the ...
0
votes
0answers
63 views

Can't find gradient for MLE for mult-class logistic regression

$$P(k | x_i;w)= \frac{exp(w_k^tx_i)}{\sum_{j=1}^K exp(w_k^tx_i)}$$ $y_i^k$ is a vector that uses 1-of-k encoding. Thus, if $y_i=k$, then the vector $y_i$ has a 1 in the kth spot and a 0 everywhere ...
0
votes
0answers
14 views

Regression Problem Interpretation help

A response Y is a function of three independent variables $x_1,x_2,$ and $x_3$ that are related as follows $Y=\beta_0+\beta_1 x_1+\beta_2 x_2 +\beta_3 x_3 +\epsilon$. a)Fit the model to the $n=7$ ...
0
votes
3answers
27 views

Scatter plot : Are the two observed data related

I have a regression question that ask to draw the scattered plot graph and then conclude if the two data lists are related. The two data lists are years of people and their cholesterol level. I went ...
1
vote
0answers
29 views

Multinomial Logistic Regression

(1) $$P(y^{(i)} =1\mid X,W) = \frac{\exp(W^{(i)^T}X)}{\sum_{j=1}^m \exp(W^{(j)^T}X)}$$ $W$ and $y$ are vectors where the superscript is an index. And there are $m$ classes (that is, there are $m$ ...
0
votes
1answer
36 views

Testing for a random walk with drift

I was wondering if someone could help me clarify something from my lecture notes. It concerns the last step. I was wondering why we test if $\frac{\hat{\beta}}{\textrm{SE}(\hat{\beta}}<0$ and ...
0
votes
0answers
24 views

Identification of linear regression function under $\ell_1$-norm criterion.

Consider a linear system \begin{align*} y = \theta^Tx + e, \end{align*} where $x\in\mathbb{R}^n$ and $e\in\mathbb{R}$ are independent Gaussian random variables with distribution $\mathcal{N}(0,I_n)$ ...
0
votes
0answers
29 views

Coefficient of Determination (Alternative Solution)

Consider the following problem - where I have purposefully omitted the numbers in question since it is of no interest for my question. From $40$ observations on \begin{align} ...
0
votes
1answer
78 views

Linear system of equations and multiple linear regression: Numerical solving

I am currently implementing a test procedure for data, namely a linear form of the Kramers-Kronig relations (paper here: http://jes.ecsdl.org/content/142/6/1885.abstract). This includes solving a ...
1
vote
1answer
50 views

Equations for Cubic Regression

So, I'm making a simple program for drawing graphs, and I'm looking at making some simple best-fit curves using some basic regression analysis. I've happily got linear and quadratic regression working ...
0
votes
1answer
40 views

Regression function - conditional mean

I am trying to understand the statistical fundamentals behind linear regression, and i have never been able to intuitively understand the following; really would appreciate if someone could give an ...
1
vote
0answers
15 views

General 2D taylor surfaces from axial behaviour and discrete points

I have a problem as follows: I have a nonlinear function, f(x,y), for which I (numerically) know the axial behaviours, f(x,y0) and f(x0,y), where x0 and y0 are constants. I can calculate discrete ...
1
vote
2answers
55 views

Is it possible to fit any regression line to a set of data points?

If you have a set of data points (x1,y1), (x2,y2),...,(xn,yn) And you know it fits a trend y=f(x) where ...
0
votes
0answers
27 views

Directional Derivative of a function containing an Indicator function.

I'm trying to understand a passage in Koenker's Quantile regression book (p.33). It says: (note that y,x, are vectors and w is the direction vector) With the first part of the outcome no problem: ...
1
vote
0answers
18 views

Predicting y from a log-linear regression

I was wondering if someone could explain to me the very last step on the right hand slide. Why is do we have a sum rather than a product. Thank you very much.
0
votes
0answers
20 views

Deming estimator

Can someone help me prove that Deming estimator $\hat{\beta_{1}}$ is monotone in $\delta$: (See https://en.wikipedia.org/wiki/Deming_regression). When deriving the estimator w.r.t $\delta$, I find ...
0
votes
2answers
17 views

Naive question re. normal equation for linear regression

The typical normal equation for linear regression is $\theta=(X^TX)^{−1} X^T Y$ such that the gradient of $J(\theta)$ is zero. Why does $X^{-1} Y$ not work? What are the numerical reasons for this?
1
vote
0answers
22 views

Logistic regression eye treacting data (need model)

I have two sets of time course data, they are for an eye-tracking study. The data is 20 100ms chunks, one category being percent fixations for canonical sentences, and the other being percent looks ...
1
vote
2answers
81 views

derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
2
votes
1answer
154 views

Understanding Regularization parameters in Machine Learning/Statistics

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...
2
votes
0answers
32 views

Smallest set of Liner equations, which exactly fit a set of points

I have a set of 2-d points,(it can be of any arbitrary dimension n). I want to find the minimum set of straight lines(linear equations) which exactly passes through the given 2-d points (unlike ...
0
votes
1answer
48 views

Fitting 2nd Order multivariate quadratic with matrices

Hopefully you at least entertain this question as it took forever to construct the below matrix using TeX. Any ways, so I have a list of data points ($X_1$,$X_2$,Y), with the X's being independent ...
0
votes
1answer
72 views

How to interpret coefficients in polynomial regression?

I am working on my thesis (study) about poverty incidence rate and its socio-economic factors using second-order polynomial regression without interaction. The final model in my study is ...
0
votes
0answers
25 views

Asymptotic Mean Square Error for kernel regression estimator

I want to derive the optimal rate of convergence for the kernel-based estimator for $E [Y|X = x]$ based on observations $(X_{1},Y_{1})$,...,$(X_{n},Y_{n})$ (where the $X_{i}$'s are $\mathbb{R}^{d}$ ...
0
votes
0answers
43 views

How do I go from $(X\beta)'Y$ to $Y'(X\beta)$ when solving OLS in matrix form?

When I first learned how to derive the OLS $\beta$ in matrix form, I learned to take the derivative of the summation first, then convert into matrix form. In other words Using calculus, we derive ...
0
votes
2answers
27 views

Prove $\sum_i \frac{\bar{x}(x_i-\bar{x})}{nS_{xx}} =0$

Prove $$\sum_i \frac{\bar{x}(x_i-\bar{x})}{nS_{xx}} =0$$ That is one detail in a proof of the variance of the intercept $\alpha$ in the simple linear regression $Y_i=\alpha+\beta x_i$.
1
vote
2answers
948 views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
1answer
62 views

regression on circular data

How would one design a regression where the dependent variable is measured in degrees on a circle? The dependent variable is on the range [0, 360), and the independent variables are demographic ...