Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
26 views

Most sensitive variable

I have a question in my homework that says identify the most sensitive variable from the regression model, I have 3 dependent variables and one independent. I have found the regression equation and ...
0
votes
0answers
29 views

Combining variances

I have the following problem: I have a data file that contains monthly returns for several companies, returns of a local index and a global index. Now I want to run the following three regressions for ...
0
votes
0answers
10 views

Challenge and Interesting point - GLM for Poisson Regression for Titanic survival data

I have a professor who made a very good point about the data titanic analysis during a lecture this week. I am still however trying to better understand. He argued that it is also possible to have a ...
2
votes
1answer
40 views

Predicting trends of timeseries data with ARIMA

I'm looking for an algorithm that can help identify abnormal trends in time-series metrics. The best I've been able to find so far is ARIMA (a completely new concept for me). We offer several ...
0
votes
2answers
51 views

Formula that describes the movement of a bishop in chess

I'm programming a chess game and I'm trying to validate the movements every player tries to make. Obviously, every piece can move differently and I've had no trouble validating their moves up until ...
1
vote
0answers
14 views

Finding probability of getting into a university based on 3 factors

How would I go about calculating probability of getting into a college (CU Boulder) using a data set containing GPA (0.00-4.00), ACT Composite score (0-36), Class rank percentile (0-100), and whether ...
2
votes
0answers
16 views

Non linear regression calculator

Are there any really good non linear regression calculators around the web? Or is something like matlab the best solution? I tried using excel and its solver tool, but it's complete garbage lol. ...
0
votes
1answer
39 views

Linear Regression Questions - Suspected Typo?

My sister just submitted an assignment and got a few questions marked incorrect (electronically) but I've just checked over them and don't believe this to be the case. Can someone either point out ...
1
vote
3answers
47 views

Transpose of $(X'X)^{-1}$

I am taking a Phd class in econometrics, and the following is used constantly, for $X$ a $n\times k$ matrix, $n \neq k$: $$(X'X)^{-1} = ((X'X)^{-1})'$$ with "$'$" standing for transpose. Having a ...
1
vote
1answer
43 views

Signal processing : future values prediction

Let $f : \mathbb{R}^+ \rightarrow \mathbb{R} $ be a continuous function. Do you have some references (books or online resource) about techniques that allow to predict $f(x_{n+1})$, knowing $f(x_0), ...
0
votes
1answer
35 views

what is the covariance between $\hat Y$ and$\hat \beta_1$?

I'm having a crisis of faith here, I'm trying to prove that $\beta_0$is unbiased. The formula for $\beta_0$(the parameter) is: $$\beta_0=\mu_Y-\beta_1\mu_X$$ The formula for $\hat \beta_0$(the ...
-2
votes
1answer
12 views

How you you add a fitted quartic regression line on MINITAB?

How you you add a fitted quartic regression line on MINITAB? So for the fitted line plot on the regression section you can select a linear, quadratic and cubic fitted regression plots but there is no ...
0
votes
0answers
10 views

Is the weighted mean of residuals over another variable equal to $0$?

I understand how residual errors must sum to zero around in a random sample (e.g. $y$-axis price of diamond predicted by x-axis weight of diamond). However, why must the weighted sum of residuals with ...
1
vote
0answers
14 views

R squared conceptual question with respect to number of observations.

The following statement is true. However, I have difficulties to understand why. I would appreciate if someone could explain it conceptually or perhaps with or without reference to any formula. In a ...
1
vote
1answer
22 views

What is the popular approaches for optimizing boolean function

Optimizing a real value function is a popular field where many optimization algorithms have been proposed such as descent of gradient, levenberg-marquardt,... However, suppose that the function is ...
0
votes
0answers
11 views

LARS algorithm for LASSO

I see that Prof Xiaohui Chen has released his code on his website for LARS in constrained form. Does anyone know that how to solve LASSO using LARS in penalized form? To be frank, I am very new to ...
0
votes
0answers
8 views

Data transformations for optimal regression maximization and minimization

I am looking for a method/algorithm that can determine the percentage at each data-point of a specific depended variable y, that simultaneously maximizes the coefficient of determination of linear ...
2
votes
3answers
43 views

Proof of Gauss-Markov theorem

Theorem: Let $Y=X\beta+\varepsilon$ where $$Y\in\mathcal M_{n\times 1}(\mathbb R),$$ $$X\in \mathcal M_{n\times p}(\mathbb R),$$ $$\beta\in\mathcal M_{n\times 1}(\mathbb R ),$$ and ...
0
votes
0answers
9 views

Auto covariance for AR, MA, ARMA

I think that for AR processes, ACF would alternating signs when we have something like this: $$x_t = -x_{t-1}+z_t$$ but what if we have AR(4): $$x_t = -x_{t-1}+x_{t-2}+x_{t-3}-x_{t-4}+z_t$$ based on ...
0
votes
0answers
43 views

What is the matrix $\boldsymbol X$ in $\boldsymbol X' \cdot \boldsymbol X \cdot \vec{\hat{\beta}} = \boldsymbol X' \vec{y}$?

I was trying to understand multilinear regression, and I was at the part where we use the least squares method to estimate $\vec{\beta}$. In this method, we find partial derivatives, and at the end we ...
0
votes
1answer
29 views

Linear, quadratic and exponential regression

I know the formulas for linear and quadratic regression. Please tell me 1) how to model an equation for exponential regression? 2) if I can use the gradient-point formula for linear regression and ...
1
vote
1answer
57 views

Fitting and Ellipse to a set of data points in Mathcad.

I would like to fit an ellipse to a set of data points in Mathcad and afterwards plot it. Searching the net, I stumbled on to Mike Shaw's post, which answers 75% of my question: See Plotting an ...
0
votes
1answer
38 views

$(Y_i - \hat{Y}_i)(\hat{Y}_i - \bar{Y}_i) = 0$

$(Y_i - \hat{Y}_i)(\hat{Y}_i - \bar{Y}) = 0$ in the image below (third and fourth line of the proof!). Why?
2
votes
1answer
30 views

Regression maximum likelihood

Given this regression model: $y_{i}=\beta_{0}+\beta_{1}x_{i}+E_{i}$. All the assumptions are valid except that now: $E_{i}\sim N(0,x_{i}\sigma^{2})$ Find Maximum likelihood parameters for ...
1
vote
2answers
29 views

The four assumptions on linear regression

It is clear that the four assumptions of a linear regression model are: Linearity, Independence of error, Homoscedasticity and Normality of error distribution. My question is does any of these four ...
2
votes
0answers
17 views

Proof that correlation coefficient squared equals the coefficient of determination

Hi I as the title says I'm looking at the proof that $r^2$ = $R^2$ in the case of simple linear regression, but I don't understand one part. There are different versions of the proof, but in most of ...
0
votes
0answers
10 views

Combining predicted responses from two regressions

I am trying to find info related to combining confidence levels & intervals on predicted response variables. To explain my problem, considering the following: A and B are two different values ...
0
votes
0answers
10 views

Interpretation of PCA

I am wondering if there is a practical interpretation of a principal component analysis: Consider you have a data matrix $X\in\mathbb{R}^{N\times p}$ and you perform a principal component analysis ...
0
votes
0answers
11 views

Logistic Regression with some other Sigmoid function

I have been using Logistic regression with Newton's method where the update rule is the following: $$ W^\intercal = W^\intercal + (X^\intercal P (I - P) X)^{-1} X^\intercal(y - p) $$ Here $P$ is ...
0
votes
0answers
8 views

Is there any correlation between approximation trendline parameters?

Let's say I have two data sets $(x,y)$ and $(p,q)$ and two approximation trendlines: Logarithmic: $y = b·ln(x) + a$ Linear: $y = bx + a$ Let's say I applied logarithmic approximation to both data ...
1
vote
1answer
24 views

Understanding convergence of OLS estimator

From a linear regression with one explanatory variable, $ y = \beta_0 + \beta_1x+e$, the OLS estimator can be written as \begin{equation} \hat{\beta}_1 = \frac{\widehat{cov(y,x)}}{\widehat{var(x)}}. ...
0
votes
1answer
22 views

Variant of linear regression using perpendicular distance instead of vertical

Normally, linear regression asks for a pair of parameters m,b such that for a set of given points $\{x_i,y_i\}$ the variance of $y-m\cdot x-b$ is minimized (this minimizes the distance in y-direction ...
2
votes
1answer
17 views

linear modeling question: How can I find variance for vector Y?

Linear models are $$Y_1= 2\theta_1+3\theta_2 +\epsilon_1$$ $$Y_2= -2\theta_1+\theta_2 +\epsilon_2$$ and $\epsilon_1=3z_1-z_2$ and $\epsilon_2=4z_1+z_2$, where $z_1, z_2$ are two random variance such ...
0
votes
0answers
13 views

Compute mean of posterior distribution given prior

Is there a formula to compute the mean/expectation of a posterior distribution given the prior? In the ridge regression context, for example. I have $Y=X\theta + \epsilon$ where $\epsilon$ ~ $N(0, ...
1
vote
2answers
22 views

Estimator for linear regression where data points have different variances

So in the case where data points have the same variance $\sigma^2$, the estimator (in normal equation form) can be written as $$\theta=(X^TX)^{-1}X^TY$$ I'm not sure how to derive a similar formula ...
0
votes
0answers
7 views

Derivation of log maximum likelihood for multiple target variables

So I'm working through Pattern Recognition and Machine Learning from Bishop. In chapter 3.1.5 he generalizes fitting a function to multiple target variables. Ultimately, we have the function: ...
0
votes
2answers
28 views

How to calculate parameters of a logarithmic approximation trendline?

I have a set of (Y) data $\left\{y_1, y_2, ..., y_n \right\}$ and a set of (X) $\left\{x_1, x_2, ..., x_n \right\}$ which I use to build a graph. I need to place a logarithmic trendline over the ...
0
votes
1answer
29 views

How to find the point after which a discrete function follows a linear and steady trend

I have many discrete functions that follow the same trend. An example of discrete function is shown in the figure below. At each step, represented on x-axis, we reduce a given area, represented on ...
0
votes
0answers
18 views

Linear regression and ANOVA

I'm trying to understand how Linear regression and ANOVA when you have two means only (for anova). I've been reading that linear regression is a special case of ANOVA but do not understand why? Can ...
0
votes
0answers
20 views

Weighted average of slope

I'm trying to find the weighted average of a temperature increase in a vessel. So, essentially the weighted average of a temperature over time. I'd like some input if this equation gets me what I'm ...
0
votes
0answers
9 views

P value graph meaning

What is the meaning of this graph? Why there is a p-value/2 in the right and not in the left?
0
votes
0answers
12 views

Given a confidence interval for B1, determine a confidence interval for the change in mean of Y

I have a simple linear regression model: $Y = B_{0} + B_{1} * X + e$ I determined a 95% confidence interval for $B_{1}$: $[-1.873 * 10^{-5} , -1.095 * 10^{-5}]$ And then the second part of the ...
0
votes
1answer
32 views

Derive estimator for weighted linear regression

I can't figure out to derive estimator for normal equations for weighted linear regression. (Supposed to be similar to normal equations.) I set up problem as $W(y-XB)^T(y-xB)$ My Steps: $W(y^Ty - ...
5
votes
0answers
110 views

Prove the estimator $\hat{B}$ of ridge regression = mean of the posterior distribution under a Gaussian prior

I want to prove that the estimator of ridge regression is the mean of the posterior distribution under Gaussian prior. $$y \sim N(X\beta,\sigma^2I),\quad \text{prior }\beta \sim N(0,\gamma^2 I).$$ ...
0
votes
2answers
51 views

How to apply the method least squares polynomial of single degree?

Now I am making Almon model. Lag is 3, and polynomial of 2 degree, so I have following linear regression equation $y_{t}$ = $a$ + $c_{0}$$z_{0}$+ $c_{1}$$z_{1}$+$c_{2}$$z_{2}$. I have a list of ...
1
vote
1answer
32 views

writing a piecewise regression model as a linear model

lets write the following piecewise regression model $$y= \alpha_0 + \alpha_1 x +\epsilon ;\ \ x\le x_0 $$ $$ y=\beta_0 +\beta_1 x + \epsilon \ \ x\gt x_0$$ according to the variable $x_0$ is ...
2
votes
0answers
22 views

Kalman filter using regressed model

I'm currently polishing flight control system for KSP, and I'm fightinng high-frequency noise in state vector measurements right now. I want to try to apply Kalman filter to provide more smooth values ...
1
vote
0answers
12 views

Estimating required data in Gaussian Process regression

This question concerns determining a function, $f(x)$ say, based on noisy measurements $y = f(x) + \xi$ (where $\xi$ is IID gaussian noise) using Gaussian process machine learning with likelihood ...
1
vote
1answer
17 views

Variance of residuals from simple linear regression

I am trying to compute $Var(e_i)$. So far I have $Var(e_i)=Var(y_i-\hat y_i)=Var(y_i)+Var(\hat y_i)-2cov(y_i,\hat y_i)$ Now, I know that $Cov(y_i,\hat y_i)=var(\hat y_i)$ but how do I prove ...
0
votes
0answers
15 views

Do Monte Carlo perturbations capture all the uncertainty in prediction?

I have a model $M$ that I use to predict a value $y = M(\vec x)$. I have known one-$\sigma$ error bars on each input $x_i \in \vec x$. I want to know the one-$\sigma$ error bar on my prediction $y$. ...