# Tagged Questions

Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

2answers
54 views

### Is minimizing the squared errors optimal?

In least squares regression, we try to minimize the sum of the squares error terms. I was wondering if this would unfairly penalize a model for having terms that are too far away. For example, a term ...
0answers
23 views

### Impact of spurious regressors on out of sample prediction error

The true DGP is $$y=\alpha_0 + \alpha_1 x_1 + \dots + \alpha_k x_k +\epsilon, \quad \epsilon\sim \mathcal{N}(0,1)\label{eq:1}$$ but we instead estimate y=...
0answers
19 views

### Find the vanishing point

I have a set of line segments in a plane, seen after a perspective projection. The initial segments are roughly parallel so that their lines of support should converge to a single vanishing point. ...
1answer
33 views

5answers
108 views
+50

### Why work with squares of error in regression analysis?

In regression analysis one finds a line that fits best by minimizing the sum of squared errors. But why squared errors? Why not use the absolute value of the error? It seems to me that with squared ...
2answers
36 views

### Regression without linearity [closed]

Given two independent, standard-normally distributed random variables $x,y\sim \mathcal{N}(0,1).$ I would like to do an univariate linear regression without intercept $Y = X \cdot \beta + \epsilon.$ R ...
0answers
23 views

### Solving position vector/constants given the acceleration vector is a function of the velocity vector

Recently I've been working on a problem and it's consuming all my free time. I've cobbled together a small model that takes into account lift and drag and now I'm trying to fit it to real data. The ...
0answers
13 views

### The relationship between simultaneous equations model and seemingly unrelated regression model?

recently I try to solve a equations-system. So after read few pieces of paper, I want to use SUR model. Based on what I read, those paper notes that usually equations-system has two method to be ...
0answers
24 views

### Probability distribution database

Is there some kind of probability distribution database? Very often I'm faced with a problem of fitting a distribution to data. And it also happens very often that exponential-looking data doesn't fit ...
0answers
7 views

### What is the relationship between regression line and graph of averages?

Why is the regression line regarded as a smoothed version of the line of averages, and why is it that when the graph of averages falls on a straight line, that line is also the regression line?
1answer
32 views

### Why is the regression line an estimate of the average value of y for each value of x?

The regression line, passing through the point of averages with a slope equivalent to r, is said to be a good estimate of the average value of y for each value of x. I can see why this is the cases ...
0answers
28 views

### choose between L1 and L2 normalization in logistic regression regularization

Wondering what are the pros and cons comparing to L1 and L2 normalization in logistic regression regularization part, For example, in below formula, it is use L2 normalization (in squared form of ...
1answer
21 views

### This is a question about the best type of regression analysis to use in my software

Let me start by saying that I am not a mathematician and I am not very good at math. I am mainly interested in obtaining the best possible results. I am currently doing trial and error with my ...
2answers
38 views

### Force minimum of quadratic fit to certain data point

I want to fit some data $(x_i, y_i)$ with quadratic function. No problem till there. However, I want the polynomium minimum of the fitted curve to be at certain point $(x_k, y_k)$. If it is possible, ...
0answers
25 views

### Linear regression from function

I have a function given by its analytical form $f(x,y,z)=0$. Is is possible to calculate linear regression directly from this function on a selected neighborhood of a point? I want to "approximate" ...
1answer
22 views

### Average Percent Rate of Change

Excuse the png equations, still a MathJax newbie. I am analyzing data I have computed: Alcohol content and Caffeine content retention after a duration of 8 hours for each. I had gotten the data in ...
0answers
52 views

### Least square method

$4$ arbitrary points $(x_1,y_1)$ $(x_2,y_2)$ $(x_3,y_3)$ $(x_4,y_4)$ are given in the $xy$ plane using the method of least squares. If regression of $y$ upon $x$ give the fitted line $y=ax+b$ , and ...
0answers
18 views

### Linear regression formula derivation

I always thought linear regression was numerically calculated by bruteforcing solutions and comparing them with some $r^2$ error accuracy. However, I recently stumbled upon this formula for ...
2answers
28 views

### intuition behind having a unique regression line

I understand this mathematically. we have function of 2 variables represents the sum of square errors. We have to find the $a$ and $b$ that minimize the function. there is only one minimum point. But ...
0answers
24 views

### What is the relationship between the function $\mathbb{E}(Y \mid X = x)$ and linear regression?

Consider the function $$r(x) = \mathbb{E}(Y \mid X = x)$$ This has been called the regression function in a textbook I'm using. I'm trying to figure out the relationship between this function ...
1answer
30 views

### Cubic Polynomial fitting with defined ranges for coefficients

Is there a way, given a set of values $(x,y)$, to find a cubic polynomial $f(x)$ that fits the values? My cubic polynomial is defined as $c_0 + c_1x +\frac {1}{2} c_2 x^2 +\frac {1}{6}c_3 x^3$ ...
0answers
14 views

### Derive the Hat Matrix to map actual response to estimated resposne

In order to measure the quality of a regression we can calculate the Hat Matrix. Using it we can estimate the response variable as if we used the predictor variables to regress them. For linear ...
0answers
20 views

### MLE of heteroscedastic model?

Given the regression model where our and are identically and independently distributed. I'm trying to find the MLE B-hat and the unbiased estimator sigma-hat^2. I haven't dealt with any models in ...
1answer
33 views

### Showing sum of squared residuals is zero?

I have the model $$y_i = B_0+\sum\limits_{i=0}^pB_kX_{ik} + e_i$$ I'm looking to show the sum of squared residuals is zero if $p = (n-1)$. I have tried expanding it quite in depth and I haven't been ...
1answer
39 views

### Showing Residual Sum of Squares for Multiple Linear Regression is 0

Problem: I have the linear regression model: $y_i=\beta_0+\sum_{k=1}^p \beta_kx_{ik}+\epsilon_i$ where $\epsilon_i\sim N(0,\sigma^2)$, for $i = 1,2,\ldots ,n$. I want to prove that the residual sum ...
0answers
15 views

### Name of the field of study that details extrapolation of a series based on subset sample data

Apologies if my notation and/or terminology is way off - I'm not well versed in mathematics. I'm looking for the name of the field of mathematics that might help me solve my problem. Here's my problem:...
1answer
26 views

### Derivative of dot product of Residual Sum Square in matrix notation

I am trying to derive the following expression w.r.t. $\beta$: $$RSS(\beta) = (\mathbf{y} - \mathbf{X} \beta)^T (\mathbf{y} - \mathbf{X} \beta)$$ I know that the ...
1answer
42 views

### Can I ignore multicolinearity problem if all the regression coefficients are highly significant?

Can I ignore multicolinearity problem if all the regression coefficients are highly significant? My data is large enough and all the resulting coefficients are significant enough in less than 0.01 ...
0answers
19 views

### Is there a way to determine the most most orthagonal variable or the “most powerful variable” in a Logistical Regression in the statstical software R?

I am curently working on a Logistical (Binary) Regression, and I am using R to create ROC curves based on the data. I cannot seem to determine exactly how I can determine variables to change to change ...
0answers
17 views

0answers
15 views

### How to get best fitting model decision for data X and Y in e.g. Matlab?

I have two sources of data, X and Y, which are basically counts, from 23 individual origins (3D ROIs in my case). For example: ...
0answers
78 views

### Two dimensional (discrete) orthogonal polynomials for regression

This question How to work out orthogonal polynomials for regression model and the answer http://math.stackexchange.com/a/354807/51020 explain how to build orthogonal polynomials for regression. ...
1answer
34 views

### How do I find first two steps of Newton's method?

How to find given by the points $(x_0,y_0)=(0.8,2.1)$ first two steps of newton method ,in order to approximate for $f(x,y)=x^3+14x+x^2y^2-5y$ one result of system of equation $\nabla f(x,y)=(0,0)$ ?
0answers
23 views

### Software to run non-linear regression

We currently use a very old version of StatGraphics and unfortunately it doesn't run on x64 systems. So I am looking for open-source software that may do the same thing as a replacement. Namely we run ...
0answers
104 views

### showing SSE of simple regression model is larger than or equal to SSE of multiple regression model

Lets say we have 2 linear regression models: $y_i = B_0 + B_1x_{i1} + \epsilon_i,$ where $\epsilon_i$ follows $N(0,σ_1^2)$ $y_i = B_0 + B_1x_{i1} + B_2x_{i2} + \lambda_i,$ where $\lambda_i$ follows ...
0answers
14 views

### Numerical Method for fitting parameters of an explicit integration to actual data

I have a heat transfer system described by, $$\{\dot{T}\} = [C^{-1}]\left([K]\{T\} + \{F\} \right)$$ where ${T}$ is a vector of the nodal temperatures of the system. From initial conditions I am able ...
0answers
29 views

### A computer test of a very fast primarily test

Fermat's little theorem states that if $p$ is a prime and $a$ is any integer not divisible by $p$, then $a^{p-1} - 1$ is divisible by $p$. $$a^{p-1}\equiv 1\pmod p$$ This can be used to test if a ...
0answers
20 views

### predicted values combined with LDA

Suppose that we transform the original predictors X to Yˆ by taking the predicted values under linear regression. Show that LDA using Yˆ is identical to using LDA in the original space.
1answer
33 views

### Logistic regression MLE example. What is this “logistic function”?

This is a basic question (I think). I am trying to grasp the idea behind this example, where we define a "logistic function" and use that to work towards the maximum likelihood estimate (MLE). We ...
0answers
15 views

### Proof of Random Treatment's Effect on the Causal Regression Function

Consider $C(x)$ to be the outcome a subject would have if they recieve dose $x$ on some real interval. the observed response is given by the random variable relationship $Y = C(X)$. We treat ...
1answer
32 views

### Unbiased Estimate of Variance

Consider a simple linear regression model for $n$ observations where $$Y_i = \beta_1 X_i + \epsilon_i$$ where $\epsilon_i \sim N(0,\sigma^2).$ I want to show that \hat{\sigma}^2 = \frac{1}{n-2} \...
0answers
8 views

### How to make a covariance matrix from multiple observations of different objects?

I have $N$ objects. From each object, I sample $M$ values $(x,y)$ like so: ...
0answers
16 views

### Weighted Linear Regression

I am performing linear regression analysis on a time-series of data. Data contains some missing values, My question is, if I impute the missing values using mean of all the values and I want to ...
0answers
28 views

### Minimize the sum of distances between a sample and two “centers”

Suppose we have a set of readings $\{X_{i}\}$, each of which is a real number. What I want is to find 2 numbers, $a$ and $b$, such that minimize the sum of distances between each $X_{i}$ and ...
1answer
27 views

### coefficient of determination: absence of cross products [closed]

With regard to the coefficient of determination, why is the total variation equal to the sum of the explained variation and the unexplained variation and there are no cross-products?