Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
2answers
37 views

Regression model when under-estimations costs us more than over-estimations

We have a factory and we are planning how many items produce in 2014. During the learning process we minimize the mean squared error. But, under-estimations costs us more than over-estimations. Let's ...
0
votes
1answer
30 views

Covariance of two random variables in a bivariate normal distribution

http://www.econstreams.com/bivariateproof.jpg Image uploaded to the link above. I'm just not seeing the connection between the 2nd equation on the left handside and the equation on the right. ...
0
votes
0answers
18 views

Principal Component Regression

Suppose that Z1, Z2 and Z3 are the principal components of a data set and Y is a vector of the response variable. The correlation coecients between Y and Z1, Z2 and Z3 are 0.25, -0.4 and 0.7, ...
1
vote
0answers
38 views

Solving a linear matrix equation with respect to the maximum of the euclidian distances between rows.

With $n>m$, real number matrices $A$, $B$, $C$ are shaped like: $$A=\left( \begin{array}{ccc} a_{1,1} & \cdots & a_{1,m} \\ \vdots & \ddots & \vdots \\ a_{n,m} & \cdots ...
1
vote
0answers
27 views

Adjusting regression for small sample bias

I have a set of data points $\{x_i\}$. These data points are grouped so that (say) $i\in\{1,2,3\}$ is group $A$, $i\in\{4,5,6,7\}$ is group $B$, etc. I would like to test the null hypothesis of no ...
0
votes
1answer
33 views

help in multiple linear regression

I am having a question in regression analysis in JMP or any other tool. I have one dependent variable $y$ and $2$ independent variables $x_1$ and $x_2$. For example: time $= y -$ per row time ( ...
1
vote
2answers
47 views

Polynomial best fit line for very large values

not only are the x values large, the difference between them and the y values is huge. My data points: 22353120,720 24448725,671.427053270323 26544330,634.312274868634 28639935,566.291966792026 ...
0
votes
0answers
41 views

standard deviation of residuals: PCR vs MLR

Does anyone know, why the standard deviation of the residuals of a PCR (principal component regression) is greater than the one of a "normal" MLR (multivariate linear regression)? Thanks!
2
votes
1answer
49 views

How to calculate $\sum(X_i-\bar{X})^2$ in R

I'm trying to figure out how to calculate $\sum(X_i-\bar{X})^2$ in R, specifically identifying it in either the aov function or $\operatorname{lm}(y\sim x)$ function. I am trying to use it to ...
1
vote
0answers
26 views

Polynomial regression - differences between algorithms

I know that I can find a polynomial regression's coefficients doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose). This is a way of finding them; now, there is (as far as I know) at least one other ...
2
votes
4answers
203 views

Fitting curve for exponential: $y = A - B\mathrm{e}^{-t/\tau}$

I have some data that follows a saturation or charging profile such as $y = A - B\mathrm{e}^{-t/\tau}$. To begin with, is there a proper name for this function? I have seen it many times, including: ...
1
vote
1answer
29 views

Morphing $\beta_1$ into a different form (OLS question in Statistics)

I am currently studying Simple Linear Regression and I have successfully proven to myself how $\beta_1$ and $\beta_0$ are derived. However, I have been stuck on a seemingly simple problem (I'm sure ...
0
votes
2answers
43 views

Why is regression analysis also statistical test?

According to wikipedia, regression analysis is a statistical process for estimating the relationships among variables. Regression analysis is widely used for prediction and forecasting. So why is ...
1
vote
0answers
36 views

Linear Regression with multiple equations

I am trying to implement a linear regression algorithm to fit a set of "true" points with their "observed" location. The points are specified using spherical coordinates on a unit sphere. I have a ...
1
vote
2answers
116 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
0
votes
2answers
48 views

Error analysis in “linearized” regression

I am currently taking an experimental chemistry course where we need to fit data to an equation of the form $y=a\exp(bx)$. They recommend "linearizing" this equation by taking the logarithms of both ...
3
votes
0answers
105 views

Can the sigmoid function approximate any function (or relation) where 0<y<1

I'm studying Machine Learning and Artificial Neural Networks. Some basic principles of Machine Learning are linear regression, multivariate linear regression, and nonlinear regression. The last of ...
0
votes
2answers
47 views

adapting a function by a linear regression

I am wondering if it is possible to adapt the function $$y=a\cdot \ln(x)+\frac{b}{x}+x$$ by a linear regression to fit experimental data? If yes, how could this be done? Thank you!
-1
votes
2answers
81 views

Proving that the estimate of a mean is a least squares estimator?

I think this is a really simple question so please bear with me -- I just had my first class in regression and I'm a little confused about nomenclature/labeling. Does anyone recommend some good ...
0
votes
1answer
39 views

Probability, Linear Models, Expectation

I'm trying to find a way of predicting various models from one "perfect model", using EXCEL. i.e. If I assume that all models should behave like the original one, for which I have all the ...
2
votes
1answer
23 views

Aproximate data with this equation (or linearize the equation)

I have found an equation that describes the behaviour of a phisical system: $$ y=a_1e^{-a_2x} + a_3 + a_4x + a_5e^{{-a_6} / {(1-x)}}$$ Now I have data of that physical system and I want to ...
0
votes
1answer
68 views

Contrary interpretations of Least Squares for Regression

According to the original thought, our goal is to minimize the quadratic error $$\min\{\frac{1}{2}(Ax-b)^2 \}$$ Then, we search the extremum by the derivation of $x$ $$A^T(Ax-b)=0$$ $$A^TAx=A^Tb$$ ...
1
vote
0answers
2k views

Linear regression: degrees of freedom of SST, SSR, and RSS

I'm trying to understand the concept of degrees of freedom in the specific case of the three quantities involved in a linear regression solution, i.e. $SST=SSR+SSE, $ i.e. Total sum of squares = ...
0
votes
2answers
82 views

How to fit a power function to data with noise

I have multiple data-sets from a Fourier series of a function $f(t,x)$ (the data-sets where obtained by varying $x$), so $A_n=\frac{2}{T}\int_0^T{f(t,x)\sin{\frac{2\pi nt}{T}}dt}$, which seems to ...
0
votes
0answers
24 views

MSE of weighted PCA estimator

I need to calculate the variance of this estimator which is a generalisation of the OLS estimator: OLS: $Y=X\beta+e$ where Y is n*1 vector of responses X is n*p pbserved matrix of regressor ...
3
votes
1answer
70 views

Minimum required data for cosine fit

With a minimum amount of (noisy) data-points, I need to find the amplitude of a simple cosine y=A*cos(x), where x is an angle from 0:2pi. I know how to fit data to the function, and I know how to ...
2
votes
2answers
329 views

How to fit a sinusoidal function through 2 points with known slopes?

I can define my sinusoidal function as $y(x) = A\sin(B x+c) + D$ or as $y(x) = A \sin(B x) + C \cos(B x) + D$ Now, I have two points with known slopes that I must fit this sine wave to, thus my ...
0
votes
0answers
55 views

Bias in Principal Components Regression

Assume we have the well known OLS model $Y=X\beta+e$ where Y is n*1 vector of responses X is n*p pbserved matrix of regressor variables $\beta$ is a p*1 vector of unknown parameters e is a n*1 ...
0
votes
2answers
57 views

Regression Analysis

When I have a table of values like \begin{array}{c|ccccc} x & 1 & 2 & 3 & 4 & 5 \\ y & 3 & 6 & 8 & 9 & 0 \\ y & 4 & 6 & 1 & 2 & 4 ...
1
vote
2answers
45 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
0
votes
1answer
34 views

regression on principal component analysis

I have done a PCA to get my principal components and now do a principal component regression. In the PCR the 1., 2. and 4. component are significant and the 3. component is insignificant. Can anyone ...
0
votes
0answers
54 views

Calculating the “likelihood of progressive fit”

I am faced with the following least squares model fitting problem: I have a process that generates time series data. This time-series data have a specific structure (i.e. i can fit a model with ...
0
votes
0answers
31 views

Sigmoid curve fit

I'm trying to use a Sigmoid function to distribute a Y quantity on a curve. I have only this constraint: The lower asymptote is (0,100) and the upper asymptote is (6,150.000). The saturation point is ...
1
vote
1answer
27 views

Unexplainable determination coëfficiënt

I have a series of data (more specific, they are coördinates of a package attached to the end of a mini level-luffing crane. The "flightpath" is linear and horizontal.) Now when I plot the data: ...
1
vote
3answers
52 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
2
votes
1answer
34 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
1
vote
0answers
156 views

Formulize / eureqa any replacements?

Greets Now that Formulize / Eureqa now charge $30.00 a month for use and have crippled the trial version does anyone know of any replacements that can do similar things like find an equation given ...
0
votes
1answer
50 views

Regression with equally spaced set

I'm working on an algorithm (written in Python/Cython, but it reads like pseudo-code) that estimates the gradient of each point in noisy data, using a variable window size. It's working very well, but ...
0
votes
1answer
82 views

Jacobian in Levenberg-Marquardt for 4-Parameter equation

I am trying to fully understand how I can use Levenberg-Marquardt to minimise a 4 parameter equation. There are lots of fancy programs to do this but the documentation about the mathematics is ...
0
votes
1answer
37 views

How to check linear independence

How can I check the linear independence of my variables? I have this system $Ax=b$ where $A$ is a $N \times 4$ matrix. I want to check the linear independence between the 4 variables in matrix $A$.
0
votes
1answer
49 views

Large Regression dataset

For a project I need a large regression (least squares) dataset: If $n$ is the number of samples and $p$ the number of features, then I need $p < n$ and $p,n$ both very large. For example ...
1
vote
2answers
58 views

Multiple linear regression

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other ...
1
vote
0answers
43 views

Linear Regression with limited information

You have grades ($Y $) for men ($D = 0$) and women ($D = 1$). The mean grades (out of total possible score of 100) are 65 for men and 72 for women. Regression of $Y$ on $D$ yields: $Y_i = b_0 + ...
1
vote
1answer
629 views

Convert nonlinear regression equation to a linear regression equation

The question is: "Show how the nonlinear regression equation y=aX^B can be converted to a linear regression equation solvable by the method of least ...
1
vote
1answer
52 views

Best fit line using geometric distance (not vertical distance)

There must be a theory of finding the best fit line to a bunch of points in the plane, where "best fit" is defined by the geometric distance, not vertical distance. In other words, we are trying to ...
1
vote
1answer
66 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
27 views

Non-close-form Regression Research

As I try to process some physic experiment data that I don't have the closed form formula with unknown parameters, I have to use some regression models like polynomials or normal distributions . The ...
0
votes
1answer
44 views

Formulating regression model in matrix notation

The observations $y_1, y_2, y_3$ were taken on the random variables $Y_1, Y_2, Y_3$ where $Y_1=\theta+e_1$ $Y_2=2\theta - \phi+e_2$ $Y_3=\theta +2 \phi+e_3$ and $E(e_i)=0, var(e_i)=\sigma^2 ...
3
votes
1answer
82 views

Solution to linear system of equations

Notation. Let $y$, $a$, and $b$ be $n\times 1$, $p\times 1$, and $q\times1$ real vectors. Let also $X$ and $Z$ be $n\times p$ and $n \times q$ real matrices. Suppose that there is no solution, $a$, ...
2
votes
1answer
51 views

Choosing the right regression

I'm trying to analyze my sleep using regression analysis. Each night is rated (dependent variable). I'm trying to explain this rating with, for example, my sleep duration and each night's bed time's ...