Tag Info

New answers tagged

0

Your question is best explained in a broader context: What is the difference between "error" and "residual?" In regression, residuals are calculated based on a fitted model for which the underlying parameters are estimated from the data we observed, because those underlying parameters are unknown to us. For this reason, residuals are not independent: a ...


0

Your model can be biased, hence the errors can sum to something other than 0, while the residuals, as you correctly point out, are constrained.


0

A possible solution was to make a linear regression of $x$ on $y$ instead of $y$ on $x$ as proposed in the question. line of regression of Y on X $$y-\bar{y} = \tfrac{cov(X,Y)}{\sigma_{x}^{2}}(x-\bar{x})$$ line of regression of X on Y $$x-\bar{x} = \tfrac{cov(X,Y)}{\sigma_{y}^{2}}(y-\bar{y})$$ By using the points shown above we get Finding the arithmetic ...


1

There are several methods to approach the problem of adjusting estimated regression coefficients for rounding errors in the data. Most of these techniques perform an adjustment to the main diagonal of the sample covariance matrix of the variables. One of the most commonly used method is the so-called Sheppard's correction. This is based on a Taylor expansion ...


0

As there are four points: $$n=4$$ Computing sums: $$S_x=\sum x_i=209$$ $$S_y=\sum y_i=1479$$ $$S_{xx}=\sum x_i^2=10921$$ $$S_{xy}=\sum x_iy_i=77317$$ $$S_{yy}=\sum y_i^2=$$ Computing $\hat\alpha$ and $\hat\beta$ $$\hat \beta=\frac{nS_{xy}-S_xS_y}{nS_{xx}-S_x^2}=52.\bar3$$ $$\hat \alpha=\frac1nS_y-\hat\beta\frac1nS_x=-2364.\bar6$$ Your questions: Yes it ...


-1

for a limited number of levels of y , try using multinomial probit. the idea is that for n-0.5 < y < n+0.5, you see n. Treat this as a categorical dependent variable. hope this helps!


0

EDITED THIS IS INCORRECT Tecnically this is straightforward. You specify the equations $$ z_{ij}=\theta_i-\theta_j + \sigma_{ij} \epsilon_{ij} $$ and try to minimize the error with respect to $\theta$s $$ \sum_{ij}\epsilon_{ij}^2=\sum_{ij}\frac{1}{\sigma_{ij}^2}(z_{ij}-\theta_i+\theta_j)^2 $$ with your contsraints on $\theta$s. EDITED The formula above is ...


0

Two vectors $v,w \in \mathbb{R}^n$ are orthogonal iff $v^t w = 0$ where $t$ indicates the transpose. Really, we're using the dot product given by $\langle v, w \rangle = v^t w$. There is a different notion of orthogonality for matrices. Here's one definition of an orthogonal matrix: $O \in \rm{M}_n(\mathbb{R})$ is orthogonal if $O^t O = I$. Equivalently, ...


0

The definitions I see in your links are unusual to me, but I think you have already answered your own question. In the sentence beginning with "Imagine" sentence in your Link 2, I think this is the definition of "orthogonal matrices" that you want: Two matrices $X_{1}, X_{2}$ are orthogonal provided that "the columns of $X_{1}$ are orthogonal to the columns ...


0

There is such a thing as ordinary least squares. There is such a thing as weight least squares, perhaps used when data points represent groups of different sizes. Some fish grow at a rate proportional to their sizes; the variance of the logarithm of their random growth next year may thus be proportional to their present age; in such a case one would take ...


0

Are all of those variables unknown? If so, then $5$ won't cut it. Take the logarithm of both sides and convert the problem into a linear regression problem. To solve the resulting set of linear equations, you will need at least $7$ data points. I just realized: where is your independent variable, $X$?


1

You want something simple, like $\left(10 - \dfrac{A}{x+B}\right)\%$ for suitable constants $A$, $B$. If you want to go through the points $(700,5\%)$ and $(10000,9.5\%)$, choose $A = \dfrac{15500}{3}, B = \dfrac{1000}{3}$.


0

I would choose sth like $$ f(x)= 10-10a^{-x} $$ for $a>0$. This satisfies $f(0)=0$ and $\lim_{x \rightarrow \infty}f(x)=10$ as well as $f(x)<10$ for all $x>0$. Now choose $a$ such that the graph fits your points best. (for this I need to know the actual values of the points)


1

A very simple equation in polar is coordinates is obtained if the center of the system of axis is not located at the center of circle, but at the "exentric" point :


3

From an engineering and practical point of view, very good models have been proposed. For temperature dependency, Vogel proposed $$\Large\eta=a~ e^{\frac{b}{T+c}}$$ (ignore $c$ if you want to have a model which can be linearized to provide you good estimates for the nonlinera regression - but, in such a case, $T$ must be absolute) and for pressure ...


0

The assumption that $x$ and $\epsilon$ are orthogonal, that is, that $\mathrm{Cov}(\epsilon,x)=0$, does not imply that $E(\epsilon|x)=E(\epsilon)$. Thus the quote in your question is indeed faulty (even assuming that $E(\epsilon)=0$). Example: let $x$ uniform on $(-1,1)$ and $\epsilon=1-3x^2$, then $E(x)=E(\epsilon)=E(x\epsilon)=0$ hence ...


0

In linear models sometimes you want the regressors or other r.v. of interest to be uncorrelated which would imply a linear independence. $E(Y_i|X_j)=0 \Rightarrow E(Y_iX_j)=0 \text{ (orthogonality condition in econometrics) } \Rightarrow Cov(X_j,Y_i)=0$ Check Fumio Hayashi Economtrics book. P.S.: You may have better luck with an answer at ...


1

Let's work with a unit circle, centered at the origin, and suppose the "eccentric" point $E$ is at $(0,a)$, where $a<1$. Let $c$ be the length of the line $L = PQ$, where $c > 1+a$. A little vector reasoning shows that $$ Q = P + \frac{c}{ \| E - P \| }(E - P) $$ If the point $P$ is on the circle, then it can be described by coordinates $P = ...


1

The ordinary least squares estimator, $\widehat \beta_{OLS}=\sum \frac{x_i y_i}{x_i^2}$, as others have mentioned minimizes the sum of squares error $\widehat \beta_{OLS}=argmin_b \sum(y_i -bx_i)^2$. The major reason that it is so widely used is because it is BLUE, best linear unbiased estimator (http://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem) ...


1

The estimator you define also makes sense. But like others said this is the different estimator. Actually it solves the different problem: $$ y_i/x_i = \beta +\epsilon_i $$ and minimizes the different sum of squares $\sum_i(y_i/x_i-\beta)^2$. Of course $x_i$ cannot be zero in such case. So the main idea under choosing the estimator what error exactly you ...


1

It's not 'wrong' to use that estimator, which is the mean of the ratios given by each point. But they are not the same; the first one is the unique value of $\beta$ which minimizes $$ E(\beta) = \sum_i \left( \beta x_i - y_i \right)^2 $$ and will give a different result in the general case. In the case where all the $x_i, y_i$ can be exactly fitted with a ...


1

Generally I know 2 ways to deal with problem like yours. Replace your missing variable by some "nominal one". It might be zero if it makes sense (sometimes it does not if typical $X_1$ is far from zero.) It might be an average of $X_1$ on the past or on say last week. Introduce a categorical variable $Z(X_1)$ wich is one if $X_1$ is not missing and zero ...


1

You can use logistic regression, which is typically used to generate a multivariable model for the prediction of a dichotomous outcome. Independent variables in logistic regression can be continuous or categorical. Using a "stepwise" procedure, most statistical softwares provide a model including only significant predictors. Looking at the odds ratios given ...


0

It is true that polynomial regression makes it far easier to overfit than OLS linear regression, but in certain settings even OLS linear regression can overfit. Suppose that we have 2 observations and 1 variable. Two points determine a line so we will have a perfect fit with all 0 residuals, but we will be overfitting this data and performance will be ...


0

The usual methods to fit the model $y=a +b e^{cx}$ are recursive i.e. with iterative process in order to compute successive impoved approximates. An initial guess for the parameters is required. A lot of publications are available, for example : http://mathworld.wolfram.com/NonlinearLeastSquaresFitting.html An unusual method was recently published. This ...


0

Based on $N$ data points $(X_i,Y_i)$, you need to fit the model $$y=a e^{bx}+c$$ which intrinsically nonlinear with respect to its parameters. This does not make any problem if you have some reasonable guess for the parameters. As you noticed, if parameter $c$ was absent from the model, you could generate estimates of parameters $a$ and $b$ starting with a ...


0

This CANNOT be done with only the means, standard deviations, correlation, and range. The range is not needed, but one other thing is needed that was mentioned in the first paragraph but not in the question in the last paragraph: the sample size, which is $250$. The equation of the least-squares line is: $$ \frac{y - 478}{107.2} = ...


0

Since you don't know exactly what the garbage collector is doing - and obviuosly you don't really know what your program is doing ;) - there is no way to decide what kind of growth you have. But it only seems to be relevant IF your numbers are growing over time or not. So you could assume linear growth and do a linear regression for example with excel/calc. ...


0

Write down an expression for the squared error: $$e=\sum_i (y_i - a x_i - b)^2$$ Then, set $\frac{\partial e}{\partial a} = 0$, $\frac{\partial e}{\partial b} = 0$ and solve the resulting system of equations for $a$ and $b$. Since the expression for $e$ is convex in $a$ and $b$, this minimizes the squared error (or you can use the second derivative test). ...


2

The $\frac{1}{m}$ is to "average" the squared error over the number of components so that the number of components doesn't affect the function (see John's answer). So now the question is why there is an extra $\frac{1}{2}$. In short, it doesn't matter. The solution that minimizes $J$ as you have written it will also minimize $2J=\frac{1}{m} \sum_i ...


1

Dividing by $2m$ ensures that the cost function doesn't depend on the number of elements in the training set. This allows a better comparison across models.



Top 50 recent answers are included