Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
0answers
50 views

Linear Regression with multiple equations

I am trying to implement a linear regression algorithm to fit a set of "true" points with their "observed" location. The points are specified using spherical coordinates on a unit sphere. I have a ...
2
votes
2answers
193 views

Polynomial regression - correctness and accuracy

I have just finished a code that performs polynomial regression, doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose) to estimate the vector of coefficients. Now I'd like to add some check procedures ...
0
votes
2answers
159 views

Error analysis in “linearized” regression

I am currently taking an experimental chemistry course where we need to fit data to an equation of the form $y=a\exp(bx)$. They recommend "linearizing" this equation by taking the logarithms of both ...
3
votes
0answers
387 views

Can the sigmoid function approximate any function (or relation) where 0<y<1

I'm studying Machine Learning and Artificial Neural Networks. Some basic principles of Machine Learning are linear regression, multivariate linear regression, and nonlinear regression. The last of ...
1
vote
2answers
52 views

adapting a function by a linear regression

I am wondering if it is possible to adapt the function $$y=a\cdot \ln(x)+\frac{b}{x}+x$$ by a linear regression to fit experimental data? If yes, how could this be done? Thank you!
-1
votes
2answers
178 views

Proving that the estimate of a mean is a least squares estimator?

I think this is a really simple question so please bear with me -- I just had my first class in regression and I'm a little confused about nomenclature/labeling. Does anyone recommend some good ...
0
votes
1answer
51 views

Probability, Linear Models, Expectation

I'm trying to find a way of predicting various models from one "perfect model", using EXCEL. i.e. If I assume that all models should behave like the original one, for which I have all the ...
2
votes
1answer
31 views

Aproximate data with this equation (or linearize the equation)

I have found an equation that describes the behaviour of a phisical system: $$ y=a_1e^{-a_2x} + a_3 + a_4x + a_5e^{{-a_6} / {(1-x)}}$$ Now I have data of that physical system and I want to ...
0
votes
1answer
87 views

Contrary interpretations of Least Squares for Regression

According to the original thought, our goal is to minimize the quadratic error $$\min\{\frac{1}{2}(Ax-b)^2 \}$$ Then, we search the extremum by the derivation of $x$ $$A^T(Ax-b)=0$$ $$A^TAx=A^Tb$$ ...
6
votes
2answers
35k views

Linear regression: degrees of freedom of SST, SSR, and RSS

I'm trying to understand the concept of degrees of freedom in the specific case of the three quantities involved in a linear regression solution, i.e. $SST=SSR+SSE, $ i.e. Total sum of squares = ...
0
votes
2answers
111 views

How to fit a power function to data with noise

I have multiple data-sets from a Fourier series of a function $f(t,x)$ (the data-sets where obtained by varying $x$), so $A_n=\frac{2}{T}\int_0^T{f(t,x)\sin{\frac{2\pi nt}{T}}dt}$, which seems to ...
3
votes
1answer
156 views

Minimum required data for cosine fit

With a minimum amount of (noisy) data-points, I need to find the amplitude of a simple cosine y=A*cos(x), where x is an angle from 0:2pi. I know how to fit data to the function, and I know how to ...
2
votes
2answers
721 views

How to fit a sinusoidal function through 2 points with known slopes?

I can define my sinusoidal function as $y(x) = A\sin(B x+c) + D$ or as $y(x) = A \sin(B x) + C \cos(B x) + D$ Now, I have two points with known slopes that I must fit this sine wave to, thus my ...
0
votes
2answers
81 views

Regression Analysis

When I have a table of values like \begin{array}{c|ccccc} x & 1 & 2 & 3 & 4 & 5 \\ y & 3 & 6 & 8 & 9 & 0 \\ y & 4 & 6 & 1 & 2 & 4 ...
1
vote
2answers
49 views

Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
0
votes
1answer
43 views

regression on principal component analysis

I have done a PCA to get my principal components and now do a principal component regression. In the PCR the 1., 2. and 4. component are significant and the 3. component is insignificant. Can anyone ...
1
vote
1answer
35 views

Unexplainable determination coëfficiënt

I have a series of data (more specific, they are coördinates of a package attached to the end of a mini level-luffing crane. The "flightpath" is linear and horizontal.) Now when I plot the data: ...
1
vote
3answers
174 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
2
votes
1answer
111 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
0
votes
1answer
186 views

Regression with equally spaced set

I'm working on an algorithm (written in Python/Cython, but it reads like pseudo-code) that estimates the gradient of each point in noisy data, using a variable window size. It's working very well, but ...
0
votes
1answer
140 views

Jacobian in Levenberg-Marquardt for 4-Parameter equation

I am trying to fully understand how I can use Levenberg-Marquardt to minimise a 4 parameter equation. There are lots of fancy programs to do this but the documentation about the mathematics is ...
0
votes
1answer
68 views

How to check linear independence

How can I check the linear independence of my variables? I have this system $Ax=b$ where $A$ is a $N \times 4$ matrix. I want to check the linear independence between the 4 variables in matrix $A$.
0
votes
1answer
57 views

Large Regression dataset

For a project I need a large regression (least squares) dataset: If $n$ is the number of samples and $p$ the number of features, then I need $p < n$ and $p,n$ both very large. For example ...
1
vote
2answers
78 views

Multiple linear regression

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other ...
1
vote
0answers
53 views

Linear Regression with limited information

You have grades ($Y $) for men ($D = 0$) and women ($D = 1$). The mean grades (out of total possible score of 100) are 65 for men and 72 for women. Regression of $Y$ on $D$ yields: $Y_i = b_0 + ...
2
votes
1answer
2k views

Convert nonlinear regression equation to a linear regression equation

The question is: "Show how the nonlinear regression equation y=aX^B can be converted to a linear regression equation solvable by the method of least ...
2
votes
1answer
109 views

Best fit line using geometric distance (not vertical distance)

There must be a theory of finding the best fit line to a bunch of points in the plane, where "best fit" is defined by the geometric distance, not vertical distance. In other words, we are trying to ...
1
vote
1answer
128 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
32 views

Non-close-form Regression Research

As I try to process some physic experiment data that I don't have the closed form formula with unknown parameters, I have to use some regression models like polynomials or normal distributions . The ...
0
votes
1answer
102 views

Formulating regression model in matrix notation

The observations $y_1, y_2, y_3$ were taken on the random variables $Y_1, Y_2, Y_3$ where $Y_1=\theta+e_1$ $Y_2=2\theta - \phi+e_2$ $Y_3=\theta +2 \phi+e_3$ and $E(e_i)=0, var(e_i)=\sigma^2 ...
3
votes
1answer
95 views

Solution to linear system of equations

Notation. Let $y$, $a$, and $b$ be $n\times 1$, $p\times 1$, and $q\times1$ real vectors. Let also $X$ and $Z$ be $n\times p$ and $n \times q$ real matrices. Suppose that there is no solution, $a$, ...
2
votes
1answer
66 views

Choosing the right regression

I'm trying to analyze my sleep using regression analysis. Each night is rated (dependent variable). I'm trying to explain this rating with, for example, my sleep duration and each night's bed time's ...
0
votes
1answer
47 views

Derive a formula of a specific curve

I have this curve And I know that the first point is $$A(0,5)$$ and the last point is $$C(1650,9.5)$$ The point almost at the center where the curve changes (if you look close, you can see a ...
1
vote
1answer
520 views

Simple linear regression - understanding given

The question is to fill out the missing numbers (A-L) of a simple linear regression model. I am having problems with converting and interpreting the given table in terms of variables. Would it be ...
1
vote
1answer
58 views

Multiple regression and hypothesis test $H_0$:$\beta_2=0$

Multiple regression model $H_0$:$\beta_2=0$, $H_1$:$\beta_2 \neq 0$ where $\beta_2$ is the vector of elements ($\beta_2, \beta_3, \dots, \beta_k$) and $\beta$ is slope of regression line. Why it is ...
0
votes
1answer
211 views

Is a traditional Multi Layer Perceptron Network capable of non-linear regression? Which activation function should be used for that purpose?

I need to use a Multi Layer Perceptron Network in order to perform some non-linear regression. Any ideas if it's possible to perform a task like that and how? Which activation function should be used ...
0
votes
3answers
69 views

Finding a function given a set of points, without knowing what type of function

If I have a set of points, but am not sure what type of function they could fit, is there a way to find that function? Here are a few of the points... ...
0
votes
0answers
172 views

Which machine learning algorithm to use?!

I have a training set which is set of essays written by students for a question. These essays are all scored by human evaluators with labels such as 1, 2 , 3 which is actually marks allotted for those ...
1
vote
1answer
72 views

Implicit Curve Fitting

I have 100 points scattered in the 3D space along the $z$ coordinate axis. The points appear to lie on a curve. Is it possible to find an (implicit) curve that fit these points and option to insert ...
2
votes
0answers
40 views

Sampling data prior to nonlinear regression

As my question shows it, I am not a statistician. My problem is that I have too many data points to be used in a nonlinear fit (I have millions of them, automatically acquired). Is there a methodology ...
1
vote
1answer
99 views

Logistic regression with interactions

I'm supposed to do a model using logistic regression. So I have a series of $N$ observed data points each of which consists of $m$ explanatory variables $x = (x_{1,i}, ... , x_{m,i})$ and associated ...
1
vote
0answers
2k views

Prove that the expectation of residual sum of squares (RSS) is equal to $\sigma^2(n-2)$

The assumed regression model is $E(Y_i|x)=\gamma+\beta(x-\bar{x})$ and $Var(Y|x)=\sigma^2$. So I have: $E(RSS)= E(\sum\limits_{i=1}^n (y_i-\hat{y_i})^2)= E(\sum\limits_{i=1}^n ...
0
votes
2answers
91 views

Autocorrelation problem, regression analysis

Bit stuck on my econometrics course (old exam q), not big on mathematical statistics, anyway this is the problem: Given some model $y_{it}=\beta_0+\beta_1x_{it}+u_{it}$ and suppose that the ...
0
votes
1answer
62 views

Statistical inference and t-stats?

I have this linear regression model with an intercept(b0) and 3 variables(b1,b2,b3). Then they drop b2 and b3 and they give a new regression line with a new b0 and b1 and consequently new standard ...
0
votes
1answer
521 views

Calculate the covariance matrix of $\hat{\beta}$

$y=X\beta+u$ where $u \sim N(0,\Sigma)$ and $\Sigma$ is symmetric & idempotent. $X: n*k$, $y:n*1$, $\beta=k*1$, $u:n*1$ vector. Suppose you apply LS(least square) to the model. Calculate the ...
0
votes
1answer
507 views

Question about derivatives, Jacobian Matrix and non-linear least square curve fitting

I am working with a software package that performs non-linear least squares curve fitting. The goal of curve fitting is to calculate the coefficients of an equation given x,y points. The software ...
0
votes
2answers
320 views

Regression analysis on temperature/sensor data

Looking for a solution to what I thought should be an easy problem, but has me running in circles somehow... I'm working with two sets of data: he first set is raw values from a sensor ...
2
votes
1answer
60 views

Basic Multilinear regression question for finding examples or counterexamples.

Hello Wise mathematicians! I have few quenstions about Multi linear regresstion. I've been asked from my friend, but I have very weak knowledge background from that field. It seems my friend is in ...
0
votes
1answer
47 views

Curve Fitting and Multiple Experiments

Say I do an an experiment 5 times, each of which gives you a list of data points. Do I fit a curve to each one separately and then average the parameters and their uncertainties? Or do I take the ...
0
votes
1answer
706 views

Show posterior probability takes the form of the logistic function

Suppose you have a D-dimensional data vector $x$ = ($x_1$, ..., $x_n$) and associated class variable $y \in \{0, 1\}$, which is Bernoulli with parameter $\alpha$. Assume the dimensions of $x$ are ...