Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
0answers
23 views

Polynomial regression - differences between algorithms

I know that I can find a polynomial regression's coefficients doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose). This is a way of finding them; now, there is (as far as I know) at least one other ...
2
votes
4answers
112 views

Fitting curve for exponential: $y = A - B\mathrm{e}^{-t/\tau}$

I have some data that follows a saturation or charging profile such as $y = A - B\mathrm{e}^{-t/\tau}$. To begin with, is there a proper name for this function? I have seen it many times, including: ...
1
vote
1answer
25 views

Morphing $\beta_1$ into a different form (OLS question in Statistics)

I am currently studying Simple Linear Regression and I have successfully proven to myself how $\beta_1$ and $\beta_0$ are derived. However, I have been stuck on a seemingly simple problem (I'm sure ...
0
votes
2answers
39 views

Why is regression analysis also statistical test?

According to wikipedia, regression analysis is a statistical process for estimating the relationships among variables. Regression analysis is widely used for prediction and forecasting. So why is ...
1
vote
0answers
28 views

Linear Regression with multiple equations

I am trying to implement a linear regression algorithm to fit a set of "true" points with their "observed" location. The points are specified using spherical coordinates on a unit sphere. I have a ...
1
vote
2answers
101 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
0
votes
2answers
28 views

Error analysis in “linearized” regression

I am currently taking an experimental chemistry course where we need to fit data to an equation of the form $y=a\exp(bx)$. They recommend "linearizing" this equation by taking the logarithms of both ...
3
votes
0answers
60 views

Can the sigmoid function approximate any function (or relation) where 0<y<1

I'm studying Machine Learning and Artificial Neural Networks. Some basic principles of Machine Learning are linear regression, multivariate linear regression, and nonlinear regression. The last of ...
0
votes
2answers
44 views

adapting a function by a linear regression

I am wondering if it is possible to adapt the function $$y=a\cdot \ln(x)+\frac{b}{x}+x$$ by a linear regression to fit experimental data? If yes, how could this be done? Thank you!
-1
votes
2answers
55 views

Proving that the estimate of a mean is a least squares estimator?

I think this is a really simple question so please bear with me -- I just had my first class in regression and I'm a little confused about nomenclature/labeling. Does anyone recommend some good ...
0
votes
1answer
37 views

Probability, Linear Models, Expectation

I'm trying to find a way of predicting various models from one "perfect model", using EXCEL. i.e. If I assume that all models should behave like the original one, for which I have all the ...
2
votes
1answer
21 views

Aproximate data with this equation (or linearize the equation)

I have found an equation that describes the behaviour of a phisical system: $$ y=a_1e^{-a_2x} + a_3 + a_4x + a_5e^{{-a_6} / {(1-x)}}$$ Now I have data of that physical system and I want to ...
0
votes
1answer
62 views

Contrary interpretations of Least Squares for Regression

According to the original thought, our goal is to minimize the quadratic error $$\min\{\frac{1}{2}(Ax-b)^2 \}$$ Then, we search the extremum by the derivation of $x$ $$A^T(Ax-b)=0$$ $$A^TAx=A^Tb$$ ...
0
votes
0answers
377 views

Linear regression: degrees of freedom of SST, SSR, and RSS

I'm trying to understand the concept of degrees of freedom in the specific case of the three quantities involved in a linear regression solution, i.e. $SST=SSR+SSE, $ i.e. Total sum of squares = ...
0
votes
2answers
73 views

How to fit a power function to data with noise

I have multiple data-sets from a Fourier series of a function $f(t,x)$ (the data-sets where obtained by varying $x$), so $A_n=\frac{2}{T}\int_0^T{f(t,x)\sin{\frac{2\pi nt}{T}}dt}$, which seems to ...
0
votes
0answers
20 views

MSE of weighted PCA estimator

I need to calculate the variance of this estimator which is a generalisation of the OLS estimator: OLS: $Y=X\beta+e$ where Y is n*1 vector of responses X is n*p pbserved matrix of regressor ...
3
votes
1answer
54 views

Minimum required data for cosine fit

With a minimum amount of (noisy) data-points, I need to find the amplitude of a simple cosine y=A*cos(x), where x is an angle from 0:2pi. I know how to fit data to the function, and I know how to ...
2
votes
2answers
177 views

How to fit a sinusoidal function through 2 points with known slopes?

I can define my sinusoidal function as $y(x) = A\sin(B x+c) + D$ or as $y(x) = A \sin(B x) + C \cos(B x) + D$ Now, I have two points with known slopes that I must fit this sine wave to, thus my ...
0
votes
0answers
48 views

Bias in Principal Components Regression

Assume we have the well known OLS model $Y=X\beta+e$ where Y is n*1 vector of responses X is n*p pbserved matrix of regressor variables $\beta$ is a p*1 vector of unknown parameters e is a n*1 ...
0
votes
2answers
55 views

Regression Analysis

When I have a table of values like \begin{array}{c|ccccc} x & 1 & 2 & 3 & 4 & 5 \\ y & 3 & 6 & 8 & 9 & 0 \\ y & 4 & 6 & 1 & 2 & 4 ...
1
vote
2answers
45 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
0
votes
1answer
33 views

regression on principal component analysis

I have done a PCA to get my principal components and now do a principal component regression. In the PCR the 1., 2. and 4. component are significant and the 3. component is insignificant. Can anyone ...
0
votes
0answers
51 views

Calculating the “likelihood of progressive fit”

I am faced with the following least squares model fitting problem: I have a process that generates time series data. This time-series data have a specific structure (i.e. i can fit a model with ...
0
votes
0answers
25 views

Sigmoid curve fit

I'm trying to use a Sigmoid function to distribute a Y quantity on a curve. I have only this constraint: The lower asymptote is (0,100) and the upper asymptote is (6,150.000). The saturation point is ...
1
vote
1answer
25 views

Unexplainable determination coëfficiënt

I have a series of data (more specific, they are coördinates of a package attached to the end of a mini level-luffing crane. The "flightpath" is linear and horizontal.) Now when I plot the data: ...
1
vote
3answers
36 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
2
votes
1answer
28 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
1
vote
0answers
90 views

Formulize / eureqa any replacements?

Greets Now that Formulize / Eureqa now charge $30.00 a month for use and have crippled the trial version does anyone know of any replacements that can do similar things like find an equation given ...
0
votes
1answer
39 views

Regression with equally spaced set

I'm working on an algorithm (written in Python/Cython, but it reads like pseudo-code) that estimates the gradient of each point in noisy data, using a variable window size. It's working very well, but ...
0
votes
1answer
63 views

Jacobian in Levenberg-Marquardt for 4-Parameter equation

I am trying to fully understand how I can use Levenberg-Marquardt to minimise a 4 parameter equation. There are lots of fancy programs to do this but the documentation about the mathematics is ...
0
votes
1answer
32 views

How to check linear independence

How can I check the linear independence of my variables? I have this system $Ax=b$ where $A$ is a $N \times 4$ matrix. I want to check the linear independence between the 4 variables in matrix $A$.
0
votes
1answer
45 views

Large Regression dataset

For a project I need a large regression (least squares) dataset: If $n$ is the number of samples and $p$ the number of features, then I need $p < n$ and $p,n$ both very large. For example ...
1
vote
2answers
52 views

Multiple linear regression

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other ...
1
vote
0answers
42 views

Linear Regression with limited information

You have grades ($Y $) for men ($D = 0$) and women ($D = 1$). The mean grades (out of total possible score of 100) are 65 for men and 72 for women. Regression of $Y$ on $D$ yields: $Y_i = b_0 + ...
1
vote
1answer
321 views

Convert nonlinear regression equation to a linear regression equation

The question is: "Show how the nonlinear regression equation y=aX^B can be converted to a linear regression equation solvable by the method of least ...
1
vote
1answer
42 views

Best fit line using geometric distance (not vertical distance)

There must be a theory of finding the best fit line to a bunch of points in the plane, where "best fit" is defined by the geometric distance, not vertical distance. In other words, we are trying to ...
1
vote
1answer
52 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
25 views

Non-close-form Regression Research

As I try to process some physic experiment data that I don't have the closed form formula with unknown parameters, I have to use some regression models like polynomials or normal distributions . The ...
0
votes
1answer
32 views

Formulating regression model in matrix notation

The observations $y_1, y_2, y_3$ were taken on the random variables $Y_1, Y_2, Y_3$ where $Y_1=\theta+e_1$ $Y_2=2\theta - \phi+e_2$ $Y_3=\theta +2 \phi+e_3$ and $E(e_i)=0, var(e_i)=\sigma^2 ...
3
votes
1answer
73 views

Solution to linear system of equations

Notation. Let $y$, $a$, and $b$ be $n\times 1$, $p\times 1$, and $q\times1$ real vectors. Let also $X$ and $Z$ be $n\times p$ and $n \times q$ real matrices. Suppose that there is no solution, $a$, ...
2
votes
1answer
46 views

Choosing the right regression

I'm trying to analyze my sleep using regression analysis. Each night is rated (dependent variable). I'm trying to explain this rating with, for example, my sleep duration and each night's bed time's ...
0
votes
0answers
73 views

Ordinal Regression: Parallel lines test

I'm analyzing survey results with many responses coming in the form of Likert scales. In this case, I actually have two Likert scale predictors (in the model as nominal variables) predicting my ...
0
votes
1answer
39 views

Derive a formula of a specific curve

I have this curve And I know that the first point is $$A(0,5)$$ and the last point is $$C(1650,9.5)$$ The point almost at the center where the curve changes (if you look close, you can see a ...
1
vote
1answer
101 views

Simple linear regression - understanding given

The question is to fill out the missing numbers (A-L) of a simple linear regression model. I am having problems with converting and interpreting the given table in terms of variables. Would it be ...
1
vote
0answers
46 views

Multiple regression and hypothesis test $H_0$:$\beta_2=0$

Multiple regression model $H_0$:$\beta_2=0$, $H_1$:$\beta_2 \neq 0$ where $\beta_2$ is the vector of elements ($\beta_2, \beta_3, \dots, \beta_k$) and $\beta$ is slope of regression line. Why it is ...
0
votes
1answer
53 views

Is a traditional Multi Layer Perceptron Network capable of non-linear regression? Which activation function should be used for that purpose?

I need to use a Multi Layer Perceptron Network in order to perform some non-linear regression. Any ideas if it's possible to perform a task like that and how? Which activation function should be used ...
0
votes
3answers
47 views

Finding a function given a set of points, without knowing what type of function

If I have a set of points, but am not sure what type of function they could fit, is there a way to find that function? Here are a few of the points... ...
0
votes
0answers
126 views

Which machine learning algorithm to use?!

I have a training set which is set of essays written by students for a question. These essays are all scored by human evaluators with labels such as 1, 2 , 3 which is actually marks allotted for those ...
1
vote
1answer
40 views

Implicit Curve Fitting

I have 100 points scattered in the 3D space along the $z$ coordinate axis. The points appear to lie on a curve. Is it possible to find an (implicit) curve that fit these points and option to insert ...
1
vote
0answers
33 views

Sampling data prior to nonlinear regression

As my question shows it, I am not a statistician. My problem is that I have too many data points to be used in a nonlinear fit (I have millions of them, automatically acquired). Is there a methodology ...