Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
388 views

Multiple Regression over an experimental dataset

I want to do a multiple regression over an experimental result shown as 3D-Plot and heatmap in following Images. Sorry as a new user i am not allowed to post them directly but it is just a link to ...
0
votes
1answer
110 views

Linear regression for normal distributions

Basically, I have that $\ Y_i = \alpha +\beta(x_i-x_{bar}) + \epsilon_i $ where $\epsilon_i$ are i.i.d normally distributed with mean 0 variance $\sigma^2$ $\ Y_i ~~has ~a~normal~distribution~as ...
1
vote
2answers
535 views

vertical vs. horizontal regression

A horizontal regression is defined as the following: $$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$ whereas ...
0
votes
1answer
66 views

P-value not shown when there are too many variables in a linear regression

x<-c(1,2,3) y<-x^1.1+x summary(lm(y~x+I(x^1.1))) I have this code in R but it just is for the sake of easier understanding of what I am trying to achieve. ...
4
votes
2answers
984 views

How does one fit the curve $y = ae^{bx} + c$?

How does one fit the curve $y = ae^{bx} + c$? The method of taking logarithms of both sides does not simplify to allow linear regression. I can take the three equations derived from setting the ...
2
votes
1answer
333 views

An intuitive explanation for neural networks as function approximators ?

We use normal linear regression for modelling functions on datasets . But Can someone explain how neural networks help in approximating more complex ,especially non-linear functions ? intuitively , ...
2
votes
3answers
187 views

least squares regression in 3space

robjohn is giving me a hand with this, but in case anybody else knows... I need to do a least-squares regression for linearity on a set of coordinates in 3space. If the dataset is linear, I need to ...
2
votes
2answers
297 views

design matrices

Given a linear model $Y = X\beta + \epsilon$ with three treatments and six subjects where $X$ is the design matrix, suppose $X = \begin{matrix}1 & 1 & 0\\ 1 & 1 & 0\\ 1 & 0 ...
0
votes
1answer
62 views

Expressing Series-Element in Terms of its Index

Consider the following recursion: $$C_{i+1} = a \sum_{j=1}^iC_j + b$$ where $a$ and $b$ are constants. Can series-element $C_i$ be expressed in terms of only its index $i$, $a$ and $b$? In case ...
6
votes
1answer
100 views

Formula for straight part of a slightly bumpy line

Given a straight line that deviates from the horizontal by at most 15 degrees. On this straight line there are bumps on top at random places on the line. The combined width of the bumps is at most ...
25
votes
6answers
24k views

Regression vs Classification

This is more machine learning questions, but perhaps someone will be able to help. I would like to know what is the difference between regression and classification when we try to generate output for ...
0
votes
0answers
105 views

Covariance and variance

Within the context of simple linear regression, I came across this: $$\hat{\beta}=\frac{\sum y_nx_n}{\sum x_n^2}=\frac{cov(xy)}{var(x)}$$ where I assume $cov(x,y)$ means the covariance between $x$ ...
1
vote
2answers
1k views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = ...
1
vote
1answer
126 views

Calculating the regression equations

I have four data points $(1,2), (2,4), (3,5), (5,7)$ and Im looking for the least squares regression line that best fits them. I use the normal equation $A^tAx=A^tb$ in this form - ...
1
vote
0answers
110 views

Jacobian approximation at given point without explicit derivatives expression

after solving a NLproblem with optimization method, I would like to compute confidence intervals, prediction bounds and standard deviation for these optimal parameters. Explicit formulas I have read ...
1
vote
0answers
193 views

Fitting a Cylinder Around a Line

Assuming data like the following: ...
5
votes
3answers
2k views

Linear Regression?

How was the formula for Ordinary Least Squares Linear Regression arrived at? Note I am not only looking for the proof, but also the derivation. Where did the formula come from?
2
votes
4answers
89 views

Seeking a function based on its level set

I'm trying to create a function for a research project, but I fear my math knowledge is insufficient to derive it from the attached diagram I've created showing its desired behavior. I'm hoping ...
2
votes
1answer
311 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: $y$=value, $x_1$=material, $x_2$=weight, $x_3$=color $x_1$ and $x_2$ are interacting variables but $x_3$ ...
0
votes
2answers
308 views

linear regression analysis

i am given data for analysis following data:relationship between height and weight,question is :is relationship between them linear?like $y$=$a$+$b$*$x$+$e$ where e is error,or quadratic?or ...
6
votes
3answers
3k views

Correlation Coefficient and Determination Coefficient

I'm really new to linear regression and am trying to teach myself. In my textbook there's a problem that asks why $R^{2}$ in the regression of $Y$ on $X =$ the sample correlation between X and Y the ...
0
votes
1answer
103 views

How to convert the constants in a regression equation to constants in a linear equation

Hello dear mathematicians, I'm not entirely sure what to tag this question with since I'm new here but I hope some more experienced user can guide me. Here is my problem: I'm using an internal ...
1
vote
1answer
125 views

Econometrics OLS estimates

I cant seem to use the formula to calculate B1 without knowing xi and yi. Is it possible to calculate using just the variances and covariance? Please help! The classical linear regression model ...
1
vote
1answer
41 views

Getting the formula of a live counter

I'm looking to replicate this greenhouse gases counter in my website. Poking around i found the initial data for the formula. The counter use the following information: Beginnig date: 2012/03/01 ...
2
votes
1answer
3k views

Construct / find the simplest function based on data

Let's say I have these 7 natural numbers (all between 0 and 255): 255, 23, 45, 32, 87, 52, 146 How can I find a function F(x) that, once computed, gives me back ...
5
votes
1answer
26k views

How to find curve equation from data?

How do I find the formula when I only know some data points ? Usually I just use the Trendline option for diagrams in Excel, but this one eludes me. I expect it to be something like : ...
1
vote
2answers
395 views

Simple Least Squares Regression?

I have a vector X of 50 real numbers and a vector Y of 50 real numbers. I want to model them as y = ax + b How do I determine a and b such that it minimizes the ...
1
vote
0answers
76 views

Books on Function approximation and Regression

Can you suggest books/articles on Function approximation Let me quote from the above wiki: Second, the target function, call it g, may be unknown; instead of an explicit formula, only a set of ...
0
votes
1answer
89 views

Regression on Linear Model?

I have 50 or so training examples involving a set of 200 or so real numbers (x1,x2,...,x200) (normalized to a 0 mean and std dev 1), and a single output real (y) in the range 0.0..1.0. I want to fit ...
1
vote
5answers
589 views

Find square root approximation function (tool)

first I have to apologize for any uncorrect naming or categorisation of my question, as I am an electrical engineer rather than a mathematican. I try to find a simple solution for my problem: I have ...
0
votes
1answer
2k views

Multiple vs Single Linear Regression

I'm having trouble understanding the relationship between multiple and single linear regression. I have six variables $(x_1, \dots, x_6)$ I'm using in my model. If I check each one individually ...
2
votes
3answers
465 views

How to deal with Linear Regression model with some data aggregated

Lets say I am trying to find a linear regression between Weight and Height of a person. $W=b_0+b_1 H+e$ The data I have gathered from 8 people is like this: ...
1
vote
3answers
760 views

Find parameters for exponential function fitting to datapoints

I have a set of datapoints, in this case the temperature of an object adjusting to the environment temperature over time. Because I know these kind of processes take the form of $$f(x)=Ae^{x/B}+C$$ I ...
3
votes
3answers
609 views

Correlation between variables

I asked this question on stats SE but did not find a suitable answer so far. Maybe someone can help. Given n random variables x1,...,xn (one-dimensional). The following is known (corr() = Pearson ...
2
votes
0answers
85 views

Polynomial and exponential regression [duplicate]

Possible Duplicate: Determining computational complexity of stochastic processes I have some points $(x_i,y_i)$ generated by a program. These values are not exact, but are random ...
2
votes
3answers
380 views

Residuals of regression model

Let's suppose I do a regression between earnings and age (and suppose I do not know the distribution of earnings). Would it be possible for the residuals to be normally distributed? I am thinking it ...
0
votes
1answer
399 views

Strange behaviour with ode45 in matlab, probably some numerical error.

I'm having a problem with a parameter estimation in a non-linear model. I think the culprit is that ode45 (an ode solver in matlab) is not properly solving my ode. It's the in red highlighted part, ...
1
vote
0answers
410 views

Moore–Penrose pseudoinverse reference

Given the eigendecompositions $AA^{\top}=Q \Lambda Q^{\top}$ and $A^{\top}A=P \Lambda P^{\top}$, where $\Lambda$ is a diagonal matrix (of eigenvalues) and $P$ and $Q$ are unitary eigenvectors matrices ...
1
vote
4answers
247 views

How to fit a curve to my data

I have a datasheet. It looks like an hyperbola. How can I fit a curve to it? And how can I plot a curve of the first derivative? ...
3
votes
0answers
303 views

Least Square Method with Positive Parameters

this is my first post here in the Stack Exchange. A friend told me about this forum and I'm giving it a try. I searched a bit past threads, but couldn't find what I wanted, so I'm posting the problem ...
0
votes
1answer
138 views

Explain Least-squares plotting with Ones -matrix

I am stuck to this code and particularly the point F=[ones(size(x)) x x.^2]. What does it do mathematically? I cannot understand what the heck the matrix has to do ...
0
votes
1answer
120 views

Linear algebra with a linear model (Matlab)

Given the equation $$r = B + e(r\cos(\theta))$$ and the corresponding data: $\theta: 0.88; 1.1; 1.42; 1.77; 2.14$ and $r: 3; 2.4; 1.65; 1.25; 1.01$ How do you input these data for matlab to ...
3
votes
0answers
1k views

Correlation and Regression Question

Two separate tests are designed to measure a students ability to solve problems. Several students are randomly selected to take both tests and the results are: $$ \begin{matrix} \text{Test A}(x) ...
3
votes
2answers
586 views

Fitting a sine function to data

I have a sequence of $n$ points $(x_i,y_i)$, for $i=1,\dots,n$. I would like to find the function, of the form $y=V\sin(x+\phi)$, which best fits the points. Which numerical method could I use? I have ...
2
votes
1answer
433 views

Curve fitting with upper and lower bounds for derivatives

I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
10
votes
1answer
6k views

Computational complexity of least square regression operation

In a least square regression algorithm, I have to do the following operations to compute regression coefficients: Matrix multiplication, complexity: $O(C^2N)$ Matrix inversion, complexity: ...
1
vote
3answers
612 views

Simple Least-Squares Regression Question

Given a set of 5 points (i.e. (1, 3), (2, 8) etc...), how can I get just the slope of the best fit line? I've been looking up least squares regression, but I'm rather statistics ignorant and don't ...
1
vote
1answer
909 views

Simple non linear fitting question(Least Squares Fitting--Exponential) [duplicate]

Possible Duplicate: easy to implement method to fit a power function (regression) I have the following simple function: $h = cV^n$ h and V being the variables and $c$ and $n$ are ...
1
vote
0answers
908 views

Best-fitting plane

I need to implement the algorithm described below. Everything is fine until the eigenvalues computation. I'm completely new to them and I found a lot of very complicated paper on the net. Is it ...
3
votes
2answers
565 views

Confidence interval of a random variable for an ordinary linear regression

I have a small problem. With my limited stats background I am not sure I am getting this one right. After fitting an ordinary linear regression model I get ...