New answers tagged

0

The chi-square distribution can be deduced using a bit of algebra, and then some distribution theory. Algebra: Using the overbar to denote sample mean, we have $\bar Y=\beta_0 +\beta_1\bar X+\bar\varepsilon$ so that $$Y_i-\bar Y = \beta_1(X_i-\bar X) + (\varepsilon_i-\bar\varepsilon).\tag1$$ The least squares estimators of $\beta_0$ and $\beta_1$ are, ...


2

As @Rahul noted in comment, $M_x$ is not invertible since $$ X'M_x=X'(I -X(X'X)^{-1}X') = X' - (X'X)(X'X)^{-1}X' = \mathbb 0 $$ where the result is zero rectangular matrix. It means that all rows of $M_x$ are linearly dependent and $M_x$ is singular.


0

The last equation in your picture says: The $1-\alpha$-th quantile of an F distribution where the first parameter is one and the second is n (F($1-\alpha$, 1, n)) is the same as the square of the $1-\frac{\alpha}{2}$-th quantile of the t distribution with n degrees of freedom($t^2(1-\frac{\alpha}{2},n))$. $\Rightarrow$ In theory this should work like this: ...


0

It essentially allows for the effect of age to be different depending on gender. Definitely try adding one and see if the term is statistically significant. It may help to see the regression equation to understand why an interaction term is useful. Without Interaction Term $$y = \beta_1 x_1 + \beta_2 x_2$$ Let $x_2$ (gender) = 0 $$y = \beta_1 x_1$$ We have ...


1

The model being $$y=ax+bx^2+\frac{c}{x}+\frac{\sin(dx)}{x^2}$$ it is nonlinear because of $d$ and you need at least reasonable estimates for the four parameters. So, fix $d$ at a given value and define $t_i=\frac{\sin(dx_i)}{x_i^2}$. You then face a problem of multilinear regression without intercept; this is easy to solve. So, for a given value of $d$, ...


2

General model fitting by least squares requires a nonlinear minimization algorithm, which will find the parameters that minimize the SSD fitting error $$\epsilon(a,b,c,\cdots)=\sum_{k=1}^n(y_k-f(x_k;a,b,c,\cdots))^2$$ where $f$ is the parametric model. The standard algorithm for this problem is by Levenberg & Marquardt. It requires the Jacobian matrix ...


1

Plug in $\hat{\theta}_0$ into the second derivative. Just don't forget to change signs as the derivative of $-x\theta_1$ w.r.t. $\theta_1$ is $-x$. Then you have $$ \sum x_i y_i + \sum x_i ( \bar{y} - \theta_1 \bar{x} ) + \theta_1\sum x_i ^ 2 = 0 $$ $$ \sum x_i y _i + \bar{y}\sum x_i - \theta_1\bar{x}\sum x_i + \theta_1 \sum x_i ^ 2 = 0 $$ note that $\sum ...


1

Let $t_i=e^{-x_i}$ and the model is just $$y=a-bt$$ which is easy.


1

You have $a$ and $b$ as linear coefficients, meaning that $$ a\cdot x_m+b\cdot e^{-x_m} = y_m $$ should be fulfilled for every $m\in \{1, \cdots, n\}$. Rewriting this as a matrix equation gives \begin{align} \begin{pmatrix}x_1 &e^{-x_1} \\ x_2 &e^{-x_2}\\ \vdots \\ x_n &e^{-x_n}\\ \end{pmatrix} \cdot \begin{pmatrix}a\\b\end{pmatrix} &= \...


1

Yes, your understanding is correct on both of them. Here you can just treat XSQ as another independent variable, i.e. your X as X1 and XSQ as X2. Also, the reason that p-value for X is great than your threshold/cut-off point 0.05 is the existence of XSQ. If you drop XSQ from your equation, X should become significant, i.e. its p-value should fall below 0.05....


1

$$s(t)=a \:2^{-2t}+b\: 2^{-t}$$ Compute the variable : $\quad x_i=2^{-t_i}$ $$s(x)=a \:x^2+b\: x$$ In order to find the approximates of $a,b$ one have to make a linear regression. This means linear with respect to the sought parameters $a,b$, of course not with respect to the functions $x^2,x$. I suppose that all is explain in your textbook on a more ...


2

The case when $X^T X$ is singular is usually called multicollinearity. There are several standart ways to deal with it: You can just get rid of the strongly correlated features. You can use regularization ($l_{1}$ or $l_{2}$ is a common choice: in case of $l_{2}$ regularization the formula for betas will be: $\hat \beta = (X^TX + \lambda I)^{-1}X^T y, \ $ ...


1

Do you know how to do multivariate linear regression? Let one of your variables be $x_1 = 2^{-2t}$ and the other be $x_2 = 2^{-t}$, calculate those on your whole dataset and now your problem is simply optimizing $y(x) = a_1x_1 + a_2x_2$.


0

I will assume you meant $\varepsilon_i$ are intended to be independent, although you omitted that. It is unclear, to say the least, what you meant by $\overline \varepsilon_i.$ If you meant $\overline\varepsilon = (\varepsilon_1+\cdots+\varepsilon_n)/n$ then the subscript $i$ is meaningless. And if that's what you meant, then there's no reason to think it's ...


0

You shouldn't ask so many questions at once. That said: Fit a model on the training set, then see how well this fit performs on the testing set. Your conjectured meaning of residual variance is incorrect. See the explanation here in terms of Bessel's correction. In short, residuals are how wrong the line of best fit is in its estimates, and the residuals ...


2

By numerical integration, assuming $k$ is a integer, $$\lim_{k\rightarrow\infty}\int_0^1~^{2k+1}\left(-\log\left(~^{2k}x\right)\right)\approx0.57951$$ It is very likely that a closed form isn't possible More info on the function: Assuming $k$ is an integer, we see that the graph of $~^{2k+1}\left(-\log\left(~^{2k}x\right)\right)$ have $4$ distinct regions ...


1

With implicit summation over indices,$$(X^TX)_{jk}=X^T_{ji}X_{ik}=(x_i)_j(x_i)_k=(x_ix_i^T)_{jk}.$$


1

Update For $z \ge 0$, let $$f(z, U) = [X^T(V + zI + XUX^T)^{+}X]^{-1} X^T (V + zI + XUX^T)^{+}.$$ Clearly, for $U\ge 0$, $f(z, U)$ is continuous on $z\in (0, \infty)$. If $U$ is not positive definite, $f(z, U)$ is not necessarily right continuous at $z=0$. It is easy to give examples. I guess that for $U>0$, $f(z, U)$ is right continuous at $z=0$. I ...


1

Consider the linear model $Y_i=X_i\theta + ϵ_i$ with $\theta \in \Theta$. The OLS estimator or MLE is $\hat{\theta}=\theta_0+(X′X)^{−1}X′ϵ$. If $lim_{n→\infty}n^{−1}X′X=Q$ where $Q$ is singular, or if the set $\Theta$ is not compact or if $ϵ_i$ is such that the objective function $\mathbb{Q}_0(\theta)$ is not continuous or does not have a unique maximum in $...


0

The basis for two dimensions would be $$ p(x) = \left[\begin{array}{ccccc} x_1 & x_2 & x_1^2 & x_1x_2 & x_2^2 \end{array}\right]^T $$ $f(x) = p(x)^T a$ where $a = [a_0, a_1, ...]$. This would be akin to calculating $$ f(x) = a_0 + a_1 x_1 + a_2 x_2 + a_3 x_1 x_2 + a_4 x_2^2 $$ Then with given data points $$ \left[\begin{array}{ccccc} ...


Top 50 recent answers are included