Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
20 views

Comparing a series expansion to polynomial regression

So I don't have a great background in mathematics but I have a quick and hopefully simple question for you guys. I'm a graduate student and I'm doing some polynomial regression on some thermodynamic ...
0
votes
0answers
26 views

Using least squares regression to apply nonlinear function to time series data

If you have a nonlinear function (see example), can you use a least squares regression approach to fit it to time series data ? Is this approach also valid for n variables? How many time points are ...
0
votes
0answers
14 views

Predict the probabilty of a value belonging to a particular class

I have got two classes : BG and FG and a set of values assigned to each of these classes. Given a new value how can I find ...
0
votes
0answers
67 views

Help Determining Gradient for Equation

I am writing an OO program with geometric objects. My Plane object is capable of taking a collection of 3d points and determining the plane of best fit. I'm using this popular document on it from ...
2
votes
1answer
58 views

Estimating landing position for a slowly falling object using latitude, longitude, and altitude.

I have a weather balloon project, in which I intend to use GPS to locate the payload when it finally comes down again. I will make the computer send coordinates to a server every minute or so, as ...
0
votes
1answer
13 views

Statistic test for comparing two regression models

I'm having two linear regression models as follows: $y = a_1x_1 + a_2x_2 + c$ and $y = b_1x_1 + b_2x_3 + c$. I'm looking for a statistical test for proving which model is better. I've obtained the $R^...
2
votes
0answers
27 views

Ridge Regression Centering Proof

This is a ridge regression problem. The following two problems are equivalent: $(w_t, b_\lambda ) = argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^Tx_i)^2+\lambda w^Tw\} $ $(w_t, b_\lambda )= argmin_{w,b}\{\...
1
vote
1answer
50 views

Multivariational regression

I have been given following model $\ln(y)=\beta_0+\beta_1a+\beta_2a^2+\beta_3a^3+\beta_4a^4+\beta_5b$ and a set of observations that describe relation between $y$ and $\{a, b\}$. The goal is to ...
2
votes
1answer
73 views

Relationship between $L^1$ norm and sparsity

I'm doing some research in the field of sparse representation and sparse modeling. I have two variables and their $L^1$ norm is calculated to make comparisons. As I take it the smaller the value of $...
0
votes
0answers
20 views

Categorical vs Numerical covariate linear regression

We have started doing regression at my university, and I would like to model ACT scores of students given some covariates. My problem is I have a covariate grade with values "A", "B", "C",and "F". ...
0
votes
0answers
9 views

FWL Theorem: Regressing $X_1$ on $X_2$ (a matrix on a matrix)

I am confused regarding the FWL theorem. One step is to regress $X_1$ on $X_2$ **to get the residuals from this regression*. Note that $X_i = (Txk_i)$ matrix. $k_1$ not necessarily equal to $k_2$. ...
0
votes
0answers
23 views

Rewriting residuals transposed times estimates with Annihilator and Projection: $e'\hat{y} +0 = y'M_xP_xy =0$

I am looking at a proof that says $$e'\hat{y} +0 = y'M_xP_xy =0$$ where $P_x$ is the projection matrix, $M_x$ is annihilator. I don't see how they obtain this result. What I am wondering is they are ...
0
votes
0answers
24 views

How to scale set of numbers

I have set of numbers. Set can have very big numbers (1E+13) and small (1). I need to scale numbers to range from 1 to 500. The scale should save original proportions. Table below shows that formula <...
0
votes
0answers
24 views

Neural network for regression

The way I understand regression for neural networks is weights being added to each x-input from the dataset. I want something slightly different. I want weights ...
2
votes
1answer
42 views

Add weights to inputs of x-value function to optimize regression [closed]

Say I have $n$ functions (not the regression function) each with $n$ inputs. These functions compute the x-values. The function is a simple summation function where the input is multiplied by a ...
0
votes
0answers
17 views

estimating $\sigma$ of regression based on Brownian motion

Suppose I have a regression that looks like this: $x_t = \alpha + \beta t + \sigma W_t$. Also suppose, I want to estimate $\beta$ and $\alpha$ using say $n$ discrete observations by a regression. ...
1
vote
1answer
36 views

Relation between correlation and regression

Let $y\in \mathbb{R}$ be a random variable. Let $y$ be expressed as a linear combination of $x_i$ $i=1,2,\cdots,n$, as follows \begin{equation} y = \sum\limits_{i=1}^nw_ix_i + \epsilon \end{equation} ...
0
votes
1answer
54 views

How to calculate the linear regression model of function $y=\alpha + \beta k + \beta x$?

I have a linear function $y=\alpha + \beta k + \beta x$ and observation data that consist of pairs of $x$ and $y$. $\alpha$, $\beta$, and $k$ are unknown parameters. I want to estimate the value of $\...
0
votes
0answers
35 views

Estimated simple linear regression model

Consider the simple linear regression model $y=50 + 10x + \varepsilon$ where $\varepsilon$ is $NID (0,16)$. Suppose that $n=20$ pairs of observations are used to fit this model. Generate $500$ samples ...
0
votes
1answer
36 views

Computing vector linear regression

In eye tracking we have to compute the linear regression for pupil and gaze. The formula is: $$\begin{bmatrix} gaze_x \\ gaze_y \end{bmatrix} = \begin{bmatrix} \theta_1 \\ \theta_2 \end{bmatrix} + \...
0
votes
1answer
50 views

Why are the two answers different?

I found exponential regression on Desmos for a few values: I want to find the X value when Y is 40,000. So, when zooming in on 40,000, I see that X is around 160. But, when putting the equation: <...
2
votes
0answers
44 views

Does linear regression form a subspace?

The author writes Given a vector of inputs $X^T = (X_1, \dots ,X_p)$, we can predict an output $Y$ via $$ \hat{Y} = \beta_0 + \sum_{j = 1}^p X_j \beta_j$$ He goes on to note that if we include a 1 in ...
1
vote
1answer
74 views

Multiple Regression Forecast

I'm going through a book called 'Regression Analysis by example by Hadi/Chaterjee and encountered a exercise(3.13) using a regression-output Part C: asks what salary would you forecast for a man ...
0
votes
0answers
66 views

Implementation of the LOWESS-algorithim (local regression data smoothing)

I need to implement the LOWESS-algorithm in a piece of software I am working on. The LOWESS-algorithm is a type of filter, which applies a locally weighted regression on each data point. In this ...
1
vote
0answers
13 views

Why divide regularization factor by size of dataset?

Suppose I'm trying to minimize a cost function: $$ J(\theta) = \frac {1} {2m} \sum _{i = 1}^ m (h_\theta (x^{(i)}) - y^{(i)})^2 $$ Adding regularization, as seen here, we get: $$ J(\theta) = \frac {...
2
votes
0answers
23 views

standard deviation for regression

The first slide is the denifition of simple linear regression model, the second slides is an example I have two questions: 1.Did I get the right calculation of the standard deviation? 2.I still ...
0
votes
3answers
69 views

How can I solve for $x$ values of a cubic function where $x$ intersects a given $y$?

Given the cubic function: y = -3 + 10x - 7x^2 + 1x^3, how can I find a value of x when ...
0
votes
0answers
29 views

Most sensitive variable

I have a question in my homework that says identify the most sensitive variable from the regression model, I have 3 dependent variables and one independent. I have found the regression equation and ...
0
votes
0answers
31 views

Combining variances

I have the following problem: I have a data file that contains monthly returns for several companies, returns of a local index and a global index. Now I want to run the following three regressions for ...
0
votes
0answers
17 views

Challenge and Interesting point - GLM for Poisson Regression for Titanic survival data

I have a professor who made a very good point about the data titanic analysis during a lecture this week. I am still however trying to better understand. He argued that it is also possible to have a ...
2
votes
1answer
46 views

Predicting trends of timeseries data with ARIMA

I'm looking for an algorithm that can help identify abnormal trends in time-series metrics. The best I've been able to find so far is ARIMA (a completely new concept for me). We offer several ...
0
votes
2answers
113 views

Formula that describes the movement of a bishop in chess

I'm programming a chess game and I'm trying to validate the movements every player tries to make. Obviously, every piece can move differently and I've had no trouble validating their moves up until ...
1
vote
0answers
15 views

Finding probability of getting into a university based on 3 factors

How would I go about calculating probability of getting into a college (CU Boulder) using a data set containing GPA (0.00-4.00), ACT Composite score (0-36), Class rank percentile (0-100), and whether ...
2
votes
0answers
26 views

Non linear regression calculator

Are there any really good non linear regression calculators around the web? Or is something like matlab the best solution? I tried using excel and its solver tool, but it's complete garbage lol. ...
0
votes
1answer
42 views

Linear Regression Questions - Suspected Typo?

My sister just submitted an assignment and got a few questions marked incorrect (electronically) but I've just checked over them and don't believe this to be the case. Can someone either point out ...
1
vote
3answers
59 views

Transpose of $(X'X)^{-1}$

I am taking a Phd class in econometrics, and the following is used constantly, for $X$ a $n\times k$ matrix, $n \neq k$: $$(X'X)^{-1} = ((X'X)^{-1})'$$ with "$'$" standing for transpose. Having a ...
1
vote
1answer
70 views

Signal processing : future values prediction

Let $f : \mathbb{R}^+ \rightarrow \mathbb{R} $ be a continuous function. Do you have some references (books or online resource) about techniques that allow to predict $f(x_{n+1})$, knowing $f(x_0), .....
0
votes
1answer
45 views

what is the covariance between $\hat Y$ and$\hat \beta_1$?

I'm having a crisis of faith here, I'm trying to prove that $\beta_0$is unbiased. The formula for $\beta_0$(the parameter) is: $$\beta_0=\mu_Y-\beta_1\mu_X$$ The formula for $\hat \beta_0$(the ...
-1
votes
1answer
16 views

How you you add a fitted quartic regression line on MINITAB?

How you you add a fitted quartic regression line on MINITAB? So for the fitted line plot on the regression section you can select a linear, quadratic and cubic fitted regression plots but there is no ...
0
votes
0answers
12 views

Is the weighted mean of residuals over another variable equal to $0$?

I understand how residual errors must sum to zero around in a random sample (e.g. $y$-axis price of diamond predicted by x-axis weight of diamond). However, why must the weighted sum of residuals with ...
1
vote
0answers
16 views

R squared conceptual question with respect to number of observations.

The following statement is true. However, I have difficulties to understand why. I would appreciate if someone could explain it conceptually or perhaps with or without reference to any formula. In a ...
1
vote
1answer
28 views

What is the popular approaches for optimizing boolean function

Optimizing a real value function is a popular field where many optimization algorithms have been proposed such as descent of gradient, levenberg-marquardt,... However, suppose that the function is ...
0
votes
0answers
23 views

LARS algorithm for LASSO

I see that Prof Xiaohui Chen has released his code on his website for LARS in constrained form. Does anyone know that how to solve LASSO using LARS in penalized form? To be frank, I am very new to ...
0
votes
0answers
10 views

Data transformations for optimal regression maximization and minimization

I am looking for a method/algorithm that can determine the percentage at each data-point of a specific depended variable y, that simultaneously maximizes the coefficient of determination of linear ...
2
votes
3answers
138 views

Proof of Gauss-Markov theorem

Theorem: Let $Y=X\beta+\varepsilon$ where $$Y\in\mathcal M_{n\times 1}(\mathbb R),$$ $$X\in \mathcal M_{n\times p}(\mathbb R),$$ $$\beta\in\mathcal M_{n\times 1}(\mathbb R ),$$ and $$\varepsilon\in\...
0
votes
0answers
18 views

Auto covariance for AR, MA, ARMA

I think that for AR processes, ACF would alternating signs when we have something like this: $$x_t = -x_{t-1}+z_t$$ but what if we have AR(4): $$x_t = -x_{t-1}+x_{t-2}+x_{t-3}-x_{t-4}+z_t$$ based on ...
0
votes
0answers
43 views

What is the matrix $\boldsymbol X$ in $\boldsymbol X' \cdot \boldsymbol X \cdot \vec{\hat{\beta}} = \boldsymbol X' \vec{y}$?

I was trying to understand multilinear regression, and I was at the part where we use the least squares method to estimate $\vec{\beta}$. In this method, we find partial derivatives, and at the end we ...
0
votes
1answer
38 views

Linear, quadratic and exponential regression

I know the formulas for linear and quadratic regression. Please tell me 1) how to model an equation for exponential regression? 2) if I can use the gradient-point formula for linear regression and ...
1
vote
1answer
93 views

Fitting and Ellipse to a set of data points in Mathcad.

I would like to fit an ellipse to a set of data points in Mathcad and afterwards plot it. Searching the net, I stumbled on to Mike Shaw's post, which answers 75% of my question: See Plotting an ...
0
votes
1answer
47 views

$(Y_i - \hat{Y}_i)(\hat{Y}_i - \bar{Y}_i) = 0$

$(Y_i - \hat{Y}_i)(\hat{Y}_i - \bar{Y}) = 0$ in the image below (third and fourth line of the proof!). Why?