Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
22 views

Rewriting residuals transposed times estimates with Annihilator and Projection: $e'\hat{y} +0 = y'M_xP_xy =0$

I am looking at a proof that says $$e'\hat{y} +0 = y'M_xP_xy =0$$ where $P_x$ is the projection matrix, $M_x$ is annihilator. I don't see how they obtain this result. What I am wondering is they are ...
0
votes
0answers
23 views

How to scale set of numbers

I have set of numbers. Set can have very big numbers (1E+13) and small (1). I need to scale numbers to range from 1 to 500. The scale should save original proportions. Table below shows that formula ...
0
votes
0answers
22 views

Neural network for regression

The way I understand regression for neural networks is weights being added to each x-input from the dataset. I want something slightly different. I want weights ...
0
votes
0answers
15 views

estimating $\sigma$ of regression based on Brownian motion

Suppose I have a regression that looks like this: $x_t = \alpha + \beta t + \sigma W_t$. Also suppose, I want to estimate $\beta$ and $\alpha$ using say $n$ discrete observations by a regression. ...
0
votes
1answer
47 views

How to calculate the linear regression model of function $y=\alpha + \beta k + \beta x$?

I have a linear function $y=\alpha + \beta k + \beta x$ and observation data that consist of pairs of $x$ and $y$. $\alpha$, $\beta$, and $k$ are unknown parameters. I want to estimate the value of ...
0
votes
0answers
34 views

Estimated simple linear regression model

Consider the simple linear regression model $y=50 + 10x + \varepsilon$ where $\varepsilon$ is $NID (0,16)$. Suppose that $n=20$ pairs of observations are used to fit this model. Generate $500$ samples ...
0
votes
1answer
36 views

Computing vector linear regression

In eye tracking we have to compute the linear regression for pupil and gaze. The formula is: $$\begin{bmatrix} gaze_x \\ gaze_y \end{bmatrix} = \begin{bmatrix} \theta_1 \\ \theta_2 \end{bmatrix} + ...
0
votes
0answers
51 views

Implementation of the LOWESS-algorithim (local regression data smoothing)

I need to implement the LOWESS-algorithm in a piece of software I am working on. The LOWESS-algorithm is a type of filter, which applies a locally weighted regression on each data point. In this ...
0
votes
0answers
29 views

Most sensitive variable

I have a question in my homework that says identify the most sensitive variable from the regression model, I have 3 dependent variables and one independent. I have found the regression equation and ...
0
votes
0answers
31 views

Combining variances

I have the following problem: I have a data file that contains monthly returns for several companies, returns of a local index and a global index. Now I want to run the following three regressions for ...
0
votes
0answers
15 views

Challenge and Interesting point - GLM for Poisson Regression for Titanic survival data

I have a professor who made a very good point about the data titanic analysis during a lecture this week. I am still however trying to better understand. He argued that it is also possible to have a ...
0
votes
0answers
11 views

Is the weighted mean of residuals over another variable equal to $0$?

I understand how residual errors must sum to zero around in a random sample (e.g. $y$-axis price of diamond predicted by x-axis weight of diamond). However, why must the weighted sum of residuals with ...
0
votes
0answers
20 views

LARS algorithm for LASSO

I see that Prof Xiaohui Chen has released his code on his website for LARS in constrained form. Does anyone know that how to solve LASSO using LARS in penalized form? To be frank, I am very new to ...
0
votes
0answers
10 views

Data transformations for optimal regression maximization and minimization

I am looking for a method/algorithm that can determine the percentage at each data-point of a specific depended variable y, that simultaneously maximizes the coefficient of determination of linear ...
0
votes
0answers
16 views

Auto covariance for AR, MA, ARMA

I think that for AR processes, ACF would alternating signs when we have something like this: $$x_t = -x_{t-1}+z_t$$ but what if we have AR(4): $$x_t = -x_{t-1}+x_{t-2}+x_{t-3}-x_{t-4}+z_t$$ based on ...
0
votes
0answers
43 views

What is the matrix $\boldsymbol X$ in $\boldsymbol X' \cdot \boldsymbol X \cdot \vec{\hat{\beta}} = \boldsymbol X' \vec{y}$?

I was trying to understand multilinear regression, and I was at the part where we use the least squares method to estimate $\vec{\beta}$. In this method, we find partial derivatives, and at the end we ...
0
votes
1answer
36 views

Linear, quadratic and exponential regression

I know the formulas for linear and quadratic regression. Please tell me 1) how to model an equation for exponential regression? 2) if I can use the gradient-point formula for linear regression and ...
0
votes
0answers
12 views

Combining predicted responses from two regressions

I am trying to find info related to combining confidence levels & intervals on predicted response variables. To explain my problem, considering the following: A and B are two different values ...
0
votes
0answers
16 views

Logistic Regression with some other Sigmoid function

I have been using Logistic regression with Newton's method where the update rule is the following: $$ W^\intercal = W^\intercal + (X^\intercal P (I - P) X)^{-1} X^\intercal(y - p) $$ Here $P$ is ...
0
votes
0answers
8 views

Is there any correlation between approximation trendline parameters?

Let's say I have two data sets $(x,y)$ and $(p,q)$ and two approximation trendlines: Logarithmic: $y = b·ln(x) + a$ Linear: $y = bx + a$ Let's say I applied logarithmic approximation to both data ...
0
votes
1answer
35 views

Variant of linear regression using perpendicular distance instead of vertical

Normally, linear regression asks for a pair of parameters m,b such that for a set of given points $\{x_i,y_i\}$ the variance of $y-m\cdot x-b$ is minimized (this minimizes the distance in y-direction ...
0
votes
0answers
19 views

Compute mean of posterior distribution given prior

Is there a formula to compute the mean/expectation of a posterior distribution given the prior? In the ridge regression context, for example. I have $Y=X\theta + \epsilon$ where $\epsilon$ ~ $N(0, ...
0
votes
0answers
9 views

Derivation of log maximum likelihood for multiple target variables

So I'm working through Pattern Recognition and Machine Learning from Bishop. In chapter 3.1.5 he generalizes fitting a function to multiple target variables. Ultimately, we have the function: ...
0
votes
2answers
41 views

How to calculate parameters of a logarithmic approximation trendline?

I have a set of (Y) data $\left\{y_1, y_2, ..., y_n \right\}$ and a set of (X) $\left\{x_1, x_2, ..., x_n \right\}$ which I use to build a graph. I need to place a logarithmic trendline over the ...
0
votes
0answers
24 views

Weighted average of slope

I'm trying to find the weighted average of a temperature increase in a vessel. So, essentially the weighted average of a temperature over time. I'd like some input if this equation gets me what I'm ...
0
votes
0answers
13 views

P value graph meaning

What is the meaning of this graph? Why there is a p-value/2 in the right and not in the left?
0
votes
0answers
14 views

Given a confidence interval for B1, determine a confidence interval for the change in mean of Y

I have a simple linear regression model: $Y = B_{0} + B_{1} * X + e$ I determined a 95% confidence interval for $B_{1}$: $[-1.873 * 10^{-5} , -1.095 * 10^{-5}]$ And then the second part of the ...
0
votes
2answers
52 views

How to apply the method least squares polynomial of single degree?

Now I am making Almon model. Lag is 3, and polynomial of 2 degree, so I have following linear regression equation $y_{t}$ = $a$ + $c_{0}$$z_{0}$+ $c_{1}$$z_{1}$+$c_{2}$$z_{2}$. I have a list of ...
0
votes
0answers
18 views

Do Monte Carlo perturbations capture all the uncertainty in prediction?

I have a model $M$ that I use to predict a value $y = M(\vec x)$. I have known one-$\sigma$ error bars on each input $x_i \in \vec x$. I want to know the one-$\sigma$ error bar on my prediction $y$. ...
0
votes
0answers
13 views

multiple regression Brown-Forsythe

I have a multiple regression model with three variables: $Y$= Labor Hours $X_1$= Cases Shipped $X_2$= Costs $X_3$= dummy variable which gives a $1$ if it falls on a holiday and $0$ if it does not. ...
0
votes
1answer
27 views

Least Square Estimators of a Linear Regression Model

A linear regression model may be written either: $Y_i$ = $\beta_0$ + $\beta_1X_i$ + $\epsilon_i$ Or $Y_i$ = $\alpha_0$ + $\alpha_1(X_i + \bar x)$ + $\epsilon_i$ Use the method of least square to ...
0
votes
1answer
18 views

How to find the least square estimators for linear regression model.

I have the linear regression model: $Y_i= \alpha_0 + \alpha_1(X_i - \overline{X})$ Anyway I got through the method for find the least square estimator for $\alpha_0$ and end up with $\sum_{i=1}^n ...
0
votes
1answer
68 views

When fitting a polynomial to data points, how to determine the reasonable degree to use?

I have wondered the following: Suppose that there is a set of data points $(x_i,y_i)$. Then I would like to know if it is more reasonable to assume if there is a polynomial relation of degree $m$ ...
0
votes
0answers
14 views

What method to apply to center value in multiple linear regression?

This a multiple linear regression question, an approach for modeling the relationship between a scalar dependent variable $Y$ and several explanatory variables (or independent variables) denoted $X$. ...
0
votes
0answers
29 views

When the predictor variable is so coded that $\bar X = 0$ and the normal error regression model applies, are $b_0$ and $b_1$ independent?

The Statement of the Problem: When the predictor variable is so coded that $\bar X = 0$ and the normal error regression model applies, are $b_0$ and $b_1$ independent? Are the joint confidence ...
0
votes
1answer
26 views

standardised random variable least square regression $X$ against $Y$, $Y$ against $X$

Let $X$ and $Y$ be mean 0 and variance 1 random variables such that we choose $\alpha$ and $\beta$ to minimise $$\mathbb{E}(X-\beta Y)^2$$ and $$\mathbb{E}(Y-\alpha X)^2$$ after not so difficult ...
0
votes
0answers
39 views

Estimating a probability distribution by fitting a function to a frequency histogram

If I want to estimate a probability distribution, is it common practice to simply fit a function to a frequency histogram? So, I am training a classifier, the performance of which is evaluated by its ...
0
votes
0answers
30 views

variances of the slope and intersect of an orthogonal / Deming) linear regression

I am a humble tinkerer who tries to get a rover to run a SLAM (simultaneous localization and mapping) process in his house. I equipped the rover with a laser rangefinder which collect distance and ...
0
votes
0answers
33 views

Hypothesis testing involving regression variable

Consider the regression model $Y = \textbf{X}\beta + \epsilon$, where $\epsilon \sim N(0, \sigma^2I_n)$, $\beta = (\beta_0, \beta_1, \dots, \beta_{10})^{T}$ Construct a test with significance level ...
0
votes
0answers
37 views

Linear regression where explanatory variable of 0 has no meaning

I want to build a predictive model, where given a few numeric explanatory variable n1, n2, ...
0
votes
0answers
34 views

Prove multiple OLS t-test follows t-distribution

I'm trying to prove the multiple regression test has a t distribution, i.e.: $\frac{\hat B_j - Bj}{se(\hat B_j)} \sim t (df=n-k-1)$ I was able to prove $\frac{\hat B_j - B_j}{sd(\hat B_j)} \sim ...
0
votes
0answers
11 views

Problems getting the covariance matrix of the ressiduals

In order to get the variance-covariance matrix of the residuals of a linear regression model, I do the following: Considering that the residual vector $e$ is: $e = Y - \hat{y} = XB+\epsilon - Xb$ ...
0
votes
0answers
24 views

Why is $\hat y_i=\hat\beta_1-\hat\beta_2x_i$?

I'm trying to show that $\hat\sigma^2 =\frac{\sum\hat\epsilon^2}{n-2}$ is an estimator without biais and I started with: $$\hat\epsilon_i=y_i-\hat y_i$$ and my teacher suggested me to use the ...
0
votes
0answers
68 views

Prove that OLS estimator of the intercept has minimum variance

Let $$y_i=B_0+B_1X_i+\epsilon_i$$ where $\epsilon_i\sim > N(0,\sigma^2)$. Find the least squares estimator of $B_0$ and show that it is unbiased and has minimum variance. I will not write in ...
0
votes
1answer
94 views

polynomial curve fitting: higher order models' root mean square error does not decrease

I am trying to fit a curve for 15 data points. I started by creating a linear model and observing the root mean square difference, followed by quadratic, cubic and increasing the degree of polynomial ...
0
votes
0answers
18 views

What is the algorithm for training and testing a logistic regression model using Newton's method and l2 regularization?

I have a spam dataset with 57 features and 3065 data points. I also have test data with around 1500 data points. The classes are spam/non-spam. I have to fit a logistic regression model on MatLab ...
0
votes
0answers
25 views

A property of Piece-wise continuous simple linear regression model

My fellow members I attempted to model the growth of capital of a small business person who does business with the aim of just raising his or her capital, as follows: Assumptions A fixed capital ...
0
votes
0answers
20 views

Piecewise linear regression with a given cost function

If we're given a cluster of $(x, y)$ values that appear non-linear [1](example image), we wish to partition the set of points into $r$ sets of continuous points [2] and then find regression lines on ...
0
votes
0answers
16 views

Proof reviewing: If z¹ and z² two random variables get by two $x_i$ centered and reducted, what may be $ρ_z¹_z²$ in terms of $ρ_x¹_x²$.

Let be z¹ and z² two random variables get by x¹ and x² centered and scaled, give the expression of $ρ_z¹_z²$ in terms of $ρ_x¹_x²$. what I know: the Matrix $R$ of correlation is $$\left\{ ...
0
votes
0answers
21 views

Comparing ridge regressions

I am applying ridge regression to biological problem and using the ridge coefficients as measure of explanatory power of each predictor. I have 2 sets of reponse variables with 2 matching sets of ...