Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

3
votes
2answers
54 views

Is minimizing the squared errors optimal?

In least squares regression, we try to minimize the sum of the squares error terms. I was wondering if this would unfairly penalize a model for having terms that are too far away. For example, a term ...
2
votes
0answers
23 views

Impact of spurious regressors on out of sample prediction error

The true DGP is \begin{equation} y=\alpha_0 + \alpha_1 x_1 + \dots + \alpha_k x_k +\epsilon, \quad \epsilon\sim \mathcal{N}(0,1)\label{eq:1} \end{equation} but we instead estimate \begin{equation} y=...
0
votes
0answers
19 views

Find the vanishing point

I have a set of line segments in a plane, seen after a perspective projection. The initial segments are roughly parallel so that their lines of support should converge to a single vanishing point. ...
1
vote
1answer
33 views

LASSO with equivalent quadratic costs

Is there any fundamental difference between the solutions obtained by minimizing following LASSO cost functions, if any? ( $A_{N \times n }$ and $ N >> n$) $ J=\Vert y-Ax \Vert_{2}^{2} + \...
1
vote
0answers
12 views

Nonlinear Multivariate Regression

Assuming I know exactly my forward model, which is represented by $n$ non-linear functions, or some probability models: $\vec{R}=f(x,y,z)$ , $f:\mathbb{R}^3\to\mathbb{R}^n$ Where each item in $R_i$ ...
1
vote
1answer
24 views

How to find an equation that describes a non-linear correlation of multiple parameters

I tried to search for this problem but I don't know what exactly I'm looking for. I found some empirical parameters that correlate but not linearly. An example follows: $Y_1$ and $Y_2$ are the values ...
0
votes
0answers
21 views

Exponential curve fit with MATLAB's fit function does not deliver good fit

I am trying to use MATLAB's fit function to fit a curve through a data set which obviously shows an exponential decay. These are the commands I use: ...
0
votes
1answer
23 views

“Best” solution to incompatible system of linear equations

I'm comparing data to a theory I've developed and right now I have to do some parameter fitting. Say I have two unknown parameters $x$ and $y$ such that $a_{1}x+b_{1}y=c_{1}$, $a_{2}x+b_{2}y=c_{2}$, $...
2
votes
5answers
108 views
+50

Why work with squares of error in regression analysis?

In regression analysis one finds a line that fits best by minimizing the sum of squared errors. But why squared errors? Why not use the absolute value of the error? It seems to me that with squared ...
1
vote
2answers
36 views

Regression without linearity [closed]

Given two independent, standard-normally distributed random variables $x,y\sim \mathcal{N}(0,1).$ I would like to do an univariate linear regression without intercept $Y = X \cdot \beta + \epsilon.$ R ...
0
votes
0answers
23 views

Solving position vector/constants given the acceleration vector is a function of the velocity vector

Recently I've been working on a problem and it's consuming all my free time. I've cobbled together a small model that takes into account lift and drag and now I'm trying to fit it to real data. The ...
1
vote
0answers
13 views

The relationship between simultaneous equations model and seemingly unrelated regression model?

recently I try to solve a equations-system. So after read few pieces of paper, I want to use SUR model. Based on what I read, those paper notes that usually equations-system has two method to be ...
1
vote
0answers
24 views

Probability distribution database

Is there some kind of probability distribution database? Very often I'm faced with a problem of fitting a distribution to data. And it also happens very often that exponential-looking data doesn't fit ...
0
votes
0answers
7 views

What is the relationship between regression line and graph of averages?

Why is the regression line regarded as a smoothed version of the line of averages, and why is it that when the graph of averages falls on a straight line, that line is also the regression line?
1
vote
1answer
32 views

Why is the regression line an estimate of the average value of y for each value of x?

The regression line, passing through the point of averages with a slope equivalent to r, is said to be a good estimate of the average value of y for each value of x. I can see why this is the cases ...
0
votes
0answers
28 views

choose between L1 and L2 normalization in logistic regression regularization

Wondering what are the pros and cons comparing to L1 and L2 normalization in logistic regression regularization part, For example, in below formula, it is use L2 normalization (in squared form of ...
0
votes
1answer
21 views

This is a question about the best type of regression analysis to use in my software

Let me start by saying that I am not a mathematician and I am not very good at math. I am mainly interested in obtaining the best possible results. I am currently doing trial and error with my ...
1
vote
2answers
38 views

Force minimum of quadratic fit to certain data point

I want to fit some data $(x_i, y_i)$ with quadratic function. No problem till there. However, I want the polynomium minimum of the fitted curve to be at certain point $(x_k, y_k)$. If it is possible, ...
1
vote
0answers
25 views

Linear regression from function

I have a function given by its analytical form $f(x,y,z)=0$. Is is possible to calculate linear regression directly from this function on a selected neighborhood of a point? I want to "approximate" ...
1
vote
1answer
22 views

Average Percent Rate of Change

Excuse the png equations, still a MathJax newbie. I am analyzing data I have computed: Alcohol content and Caffeine content retention after a duration of 8 hours for each. I had gotten the data in ...
0
votes
0answers
52 views

Least square method

$4$ arbitrary points $(x_1,y_1)$ $(x_2,y_2)$ $(x_3,y_3)$ $(x_4,y_4)$ are given in the $xy$ plane using the method of least squares. If regression of $y$ upon $x$ give the fitted line $y=ax+b$ , and ...
0
votes
0answers
18 views

Linear regression formula derivation

I always thought linear regression was numerically calculated by bruteforcing solutions and comparing them with some $r^2$ error accuracy. However, I recently stumbled upon this formula for ...
3
votes
2answers
28 views

intuition behind having a unique regression line

I understand this mathematically. we have function of 2 variables represents the sum of square errors. We have to find the $a$ and $b$ that minimize the function. there is only one minimum point. But ...
2
votes
0answers
24 views

What is the relationship between the function $\mathbb{E}(Y \mid X = x)$ and linear regression?

Consider the function $$ r(x) = \mathbb{E}(Y \mid X = x) $$ This has been called the regression function in a textbook I'm using. I'm trying to figure out the relationship between this function ...
0
votes
1answer
30 views

Cubic Polynomial fitting with defined ranges for coefficients

Is there a way, given a set of values $(x,y)$, to find a cubic polynomial $f(x)$ that fits the values? My cubic polynomial is defined as $c_0 + c_1x +\frac {1}{2} c_2 x^2 +\frac {1}{6}c_3 x^3$ ...
2
votes
0answers
14 views

Derive the Hat Matrix to map actual response to estimated resposne

In order to measure the quality of a regression we can calculate the Hat Matrix. Using it we can estimate the response variable as if we used the predictor variables to regress them. For linear ...
0
votes
0answers
20 views

MLE of heteroscedastic model?

Given the regression model where our and are identically and independently distributed. I'm trying to find the MLE B-hat and the unbiased estimator sigma-hat^2. I haven't dealt with any models in ...
0
votes
1answer
33 views

Showing sum of squared residuals is zero?

I have the model $$y_i = B_0+\sum\limits_{i=0}^pB_kX_{ik} + e_i$$ I'm looking to show the sum of squared residuals is zero if $p = (n-1)$. I have tried expanding it quite in depth and I haven't been ...
1
vote
1answer
39 views

Showing Residual Sum of Squares for Multiple Linear Regression is 0

Problem: I have the linear regression model: $y_i=\beta_0+\sum_{k=1}^p \beta_kx_{ik}+\epsilon_i$ where $\epsilon_i\sim N(0,\sigma^2)$, for $i = 1,2,\ldots ,n$. I want to prove that the residual sum ...
1
vote
0answers
15 views

Name of the field of study that details extrapolation of a series based on subset sample data

Apologies if my notation and/or terminology is way off - I'm not well versed in mathematics. I'm looking for the name of the field of mathematics that might help me solve my problem. Here's my problem:...
1
vote
1answer
26 views

Derivative of dot product of Residual Sum Square in matrix notation

I am trying to derive the following expression w.r.t. $\beta$: \begin{equation} RSS(\beta) = (\mathbf{y} - \mathbf{X} \beta)^T (\mathbf{y} - \mathbf{X} \beta) \end{equation} I know that the ...
1
vote
1answer
42 views

Can I ignore multicolinearity problem if all the regression coefficients are highly significant?

Can I ignore multicolinearity problem if all the regression coefficients are highly significant? My data is large enough and all the resulting coefficients are significant enough in less than 0.01 ...
0
votes
0answers
19 views

Is there a way to determine the most most orthagonal variable or the “most powerful variable” in a Logistical Regression in the statstical software R?

I am curently working on a Logistical (Binary) Regression, and I am using R to create ROC curves based on the data. I cannot seem to determine exactly how I can determine variables to change to change ...
1
vote
0answers
17 views

Confused about solution to the piecewise constant regression model

I am confused about the solution to the following solution to fitting piecewise constants: Specifically, are we minimising the sum of squares, that is, finding the vector $\beta = (\beta_1,\beta_2, ...
0
votes
1answer
20 views

error term in time-series Seasonal AR model

I am reading a paper related to timeseries forecasting in which I have a question regarding the seasonal AR model described in equation (1.2) namely: $log(y_t)$~$log(y_{t-1}) + log(y_{t-12}) + x^{(1)}...
0
votes
0answers
15 views

How to get best fitting model decision for data X and Y in e.g. Matlab?

I have two sources of data, X and Y, which are basically counts, from 23 individual origins (3D ROIs in my case). For example: ...
3
votes
0answers
78 views

Two dimensional (discrete) orthogonal polynomials for regression

This question How to work out orthogonal polynomials for regression model and the answer http://math.stackexchange.com/a/354807/51020 explain how to build orthogonal polynomials for regression. ...
0
votes
1answer
34 views

How do I find first two steps of Newton's method?

How to find given by the points $(x_0,y_0)=(0.8,2.1)$ first two steps of newton method ,in order to approximate for $f(x,y)=x^3+14x+x^2y^2-5y$ one result of system of equation $\nabla f(x,y)=(0,0)$ ?
0
votes
0answers
23 views

Software to run non-linear regression

We currently use a very old version of StatGraphics and unfortunately it doesn't run on x64 systems. So I am looking for open-source software that may do the same thing as a replacement. Namely we run ...
1
vote
0answers
104 views

showing SSE of simple regression model is larger than or equal to SSE of multiple regression model

Lets say we have 2 linear regression models: $y_i = B_0 + B_1x_{i1} + \epsilon_i,$ where $\epsilon_i$ follows $N(0,σ_1^2)$ $y_i = B_0 + B_1x_{i1} + B_2x_{i2} + \lambda_i,$ where $ \lambda_i$ follows ...
1
vote
0answers
14 views

Numerical Method for fitting parameters of an explicit integration to actual data

I have a heat transfer system described by, $$\{\dot{T}\} = [C^{-1}]\left([K]\{T\} + \{F\} \right)$$ where ${T}$ is a vector of the nodal temperatures of the system. From initial conditions I am able ...
0
votes
0answers
29 views

A computer test of a very fast primarily test

Fermat's little theorem states that if $p$ is a prime and $a$ is any integer not divisible by $p$, then $a^{p-1} - 1$ is divisible by $p$. $$a^{p-1}\equiv 1\pmod p$$ This can be used to test if a ...
0
votes
0answers
20 views

predicted values combined with LDA

Suppose that we transform the original predictors X to Yˆ by taking the predicted values under linear regression. Show that LDA using Yˆ is identical to using LDA in the original space.
1
vote
1answer
33 views

Logistic regression MLE example. What is this “logistic function”?

This is a basic question (I think). I am trying to grasp the idea behind this example, where we define a "logistic function" and use that to work towards the maximum likelihood estimate (MLE). We ...
0
votes
0answers
15 views

Proof of Random Treatment's Effect on the Causal Regression Function

Consider $C(x)$ to be the outcome a subject would have if they recieve dose $x$ on some real interval. the observed response is given by the random variable relationship $Y = C(X)$. We treat ...
0
votes
1answer
32 views

Unbiased Estimate of Variance

Consider a simple linear regression model for $n$ observations where $$Y_i = \beta_1 X_i + \epsilon_i$$ where $\epsilon_i \sim N(0,\sigma^2).$ I want to show that $$\hat{\sigma}^2 = \frac{1}{n-2} \...
0
votes
0answers
8 views

How to make a covariance matrix from multiple observations of different objects?

I have $N$ objects. From each object, I sample $M$ values $(x,y)$ like so: ...
0
votes
0answers
16 views

Weighted Linear Regression

I am performing linear regression analysis on a time-series of data. Data contains some missing values, My question is, if I impute the missing values using mean of all the values and I want to ...
0
votes
0answers
28 views

Minimize the sum of distances between a sample and two “centers”

Suppose we have a set of readings $\{X_{i}\}$, each of which is a real number. What I want is to find 2 numbers, $a$ and $b$, such that minimize the sum of distances between each $X_{i}$ and ...
0
votes
1answer
27 views

coefficient of determination: absence of cross products [closed]

With regard to the coefficient of determination, why is the total variation equal to the sum of the explained variation and the unexplained variation and there are no cross-products?