Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
2answers
690 views

How to do a regression with only integer values and a fixed intercept?

I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this ...
1
vote
2answers
86 views

Multiple regression with model $Y = (1 + c_1X_1)(1 + c_2X_2)\ldots(1 + c_nX_n)$

I'm currently working with data contained in $Y, X_1, X_2, \ldots, X_n$ and wish to fit it to the model: $Y = (1 + c_1X_1)(1 + c_2X_2)\ldots(1 + c_nX_n)$ where the $c_i$ are coefficients to be ...
3
votes
0answers
162 views

How to perform nonlinear regression with correlated errors?

I have a nonlinear least squares problem, but the errors are correlated. I could use R's nls function to do the regression if the errors were independent, but I don't know the right way to handle ...
1
vote
4answers
335 views

Parabola from 4 approximate points

I have calculated four approximate points from a sensors to get information. I would like to deduce the closest parabola to my points. The problem is that I can't solve it to get an appropriate ...
3
votes
2answers
132 views

Can I consider shooting% as an independent variable

First time poster in the math section (a few posts in the stats section) and I am looking for clarification on a variable query that I have. Basically I enjoy sports and enjoy putting a mathematical ...
1
vote
1answer
209 views

Quadratic function as a linear function

I am taking a machine learning course and today we were given an example of regression, with two attributes $x_,x_2$ and $y$ being the real valued outcome. $y$ is a quadratic function of $x_1,x_2$, ...
2
votes
0answers
325 views

Finding a model for multiple non-linear regression

I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
1
vote
0answers
455 views

Fitting a 3d point cloud with a polynomial surface

I have 3D point cloud and I would like to fit a polynomial surface to it. Could anybody please explain the step by step process to that. Thanks a lot.
2
votes
0answers
3k views

Derivation of standard error of beta in simple linear regression

Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example: ...
0
votes
1answer
251 views

Predict the height of a student whose weight is 60 kilograms.

The average height and weight of a group of students turned out to be 5 ft 6 inches and 65 kilograms respectively. The correlation between heights and weights was found to be 0.6. Using the regression ...
0
votes
2answers
738 views

“Proof” of an Algebraic property of OLS Estimators

I'm having a bit of trouble proving $\sum (x_i - \bar{x})\hat{e_i} = 0$. What I know so far is that the total sum of $\hat{e_i}$'s is zero by property of OLS so when you distribute the $\hat{e_i}$ ...
0
votes
1answer
33 views

Optimization, solving for the 'error' coefficient

Given a modified regression equation: $\hat Y = \exp(\beta_0 + \sum\beta_ix_i + \varepsilon)*F$ where: $\hat Y = 11353$ $\beta_0 = 8.693021$ $\sum\beta_ix_i = 5.95487177696$ $F = 0.21829$ what ...
0
votes
2answers
349 views

Overcoming Linear Regression Assumptions

I'm a beginner in econometrics (learning on my own, and not from school) and I'm trying to build an intuition to understanding linear regression. We know that modeling real world data is bound to ...
1
vote
2answers
141 views

Calculating the MLE for mu(x) in a regression model

Say we have the following regression model: $$Y_i = \alpha + \beta(x_i - \mathrm{mean}(x)) + R_i$$ where $R_1,\ldots,R_{20} \sim G(0, \sigma)$ If we have $\mu(x) = \alpha + \beta(x - ...
2
votes
1answer
438 views

variance of multiple regression coefficients

If I consider universal kriging (or multiple spatial regression) in matrix form as: ${\bf{V = XA + R }}$ where $\bf{R}$ is the residual and $\bf{A}$ are the trend coefficients, then the estimate of ...
1
vote
0answers
333 views

Confusion regarding confidence interval

I was using matlab's cftool to fit a regression line to my data point x and y. And I could see this ...
2
votes
1answer
67 views

Probabilistic regression on outliers

I have a given data set $D = \{ x_i, y_i \}_{i=1}^n$ for a regression problem. When I plot the data, it looks like there is an underlying parabola (2nd order linear model) and some outliers. I want ...
1
vote
1answer
101 views

Orthogonal fitted values

I have two regression models $$Y=X\beta+\varepsilon,\quad \beta\in\mathbb{R}^k$$ $$Y=Z\alpha+u\quad \alpha\in\mathbb{R}^m$$ it is known that using OLS estimates $\hat{\beta},\hat{\alpha}$ fitted ...
1
vote
1answer
393 views

Multiple Regression over an experimental dataset

I want to do a multiple regression over an experimental result shown as 3D-Plot and heatmap in following Images. Sorry as a new user i am not allowed to post them directly but it is just a link to ...
0
votes
1answer
112 views

Linear regression for normal distributions

Basically, I have that $\ Y_i = \alpha +\beta(x_i-x_{bar}) + \epsilon_i $ where $\epsilon_i$ are i.i.d normally distributed with mean 0 variance $\sigma^2$ $\ Y_i ~~has ~a~normal~distribution~as ...
1
vote
2answers
543 views

vertical vs. horizontal regression

A horizontal regression is defined as the following: $$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$ whereas ...
0
votes
1answer
66 views

P-value not shown when there are too many variables in a linear regression

x<-c(1,2,3) y<-x^1.1+x summary(lm(y~x+I(x^1.1))) I have this code in R but it just is for the sake of easier understanding of what I am trying to achieve. ...
4
votes
2answers
1k views

How does one fit the curve $y = ae^{bx} + c$?

How does one fit the curve $y = ae^{bx} + c$? The method of taking logarithms of both sides does not simplify to allow linear regression. I can take the three equations derived from setting the ...
2
votes
1answer
349 views

An intuitive explanation for neural networks as function approximators ?

We use normal linear regression for modelling functions on datasets . But Can someone explain how neural networks help in approximating more complex ,especially non-linear functions ? intuitively , ...
2
votes
3answers
188 views

least squares regression in 3space

robjohn is giving me a hand with this, but in case anybody else knows... I need to do a least-squares regression for linearity on a set of coordinates in 3space. If the dataset is linear, I need to ...
2
votes
2answers
299 views

design matrices

Given a linear model $Y = X\beta + \epsilon$ with three treatments and six subjects where $X$ is the design matrix, suppose $X = \begin{matrix}1 & 1 & 0\\ 1 & 1 & 0\\ 1 & 0 ...
0
votes
1answer
62 views

Expressing Series-Element in Terms of its Index

Consider the following recursion: $$C_{i+1} = a \sum_{j=1}^iC_j + b$$ where $a$ and $b$ are constants. Can series-element $C_i$ be expressed in terms of only its index $i$, $a$ and $b$? In case ...
6
votes
1answer
100 views

Formula for straight part of a slightly bumpy line

Given a straight line that deviates from the horizontal by at most 15 degrees. On this straight line there are bumps on top at random places on the line. The combined width of the bumps is at most ...
25
votes
6answers
26k views

Regression vs Classification

This is more machine learning questions, but perhaps someone will be able to help. I would like to know what is the difference between regression and classification when we try to generate output for ...
0
votes
0answers
110 views

Covariance and variance

Within the context of simple linear regression, I came across this: $$\hat{\beta}=\frac{\sum y_nx_n}{\sum x_n^2}=\frac{cov(xy)}{var(x)}$$ where I assume $cov(x,y)$ means the covariance between $x$ ...
1
vote
2answers
1k views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = ...
1
vote
1answer
127 views

Calculating the regression equations

I have four data points $(1,2), (2,4), (3,5), (5,7)$ and Im looking for the least squares regression line that best fits them. I use the normal equation $A^tAx=A^tb$ in this form - ...
1
vote
0answers
110 views

Jacobian approximation at given point without explicit derivatives expression

after solving a NLproblem with optimization method, I would like to compute confidence intervals, prediction bounds and standard deviation for these optimal parameters. Explicit formulas I have read ...
1
vote
0answers
193 views

Fitting a Cylinder Around a Line

Assuming data like the following: ...
5
votes
3answers
2k views

Linear Regression?

How was the formula for Ordinary Least Squares Linear Regression arrived at? Note I am not only looking for the proof, but also the derivation. Where did the formula come from?
2
votes
4answers
89 views

Seeking a function based on its level set

I'm trying to create a function for a research project, but I fear my math knowledge is insufficient to derive it from the attached diagram I've created showing its desired behavior. I'm hoping ...
2
votes
1answer
311 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: $y$=value, $x_1$=material, $x_2$=weight, $x_3$=color $x_1$ and $x_2$ are interacting variables but $x_3$ ...
0
votes
2answers
309 views

linear regression analysis

i am given data for analysis following data:relationship between height and weight,question is :is relationship between them linear?like $y$=$a$+$b$*$x$+$e$ where e is error,or quadratic?or ...
6
votes
3answers
3k views

Correlation Coefficient and Determination Coefficient

I'm really new to linear regression and am trying to teach myself. In my textbook there's a problem that asks why $R^{2}$ in the regression of $Y$ on $X =$ the sample correlation between X and Y the ...
0
votes
1answer
104 views

How to convert the constants in a regression equation to constants in a linear equation

Hello dear mathematicians, I'm not entirely sure what to tag this question with since I'm new here but I hope some more experienced user can guide me. Here is my problem: I'm using an internal ...
1
vote
1answer
125 views

Econometrics OLS estimates

I cant seem to use the formula to calculate B1 without knowing xi and yi. Is it possible to calculate using just the variances and covariance? Please help! The classical linear regression model ...
1
vote
1answer
41 views

Getting the formula of a live counter

I'm looking to replicate this greenhouse gases counter in my website. Poking around i found the initial data for the formula. The counter use the following information: Beginnig date: 2012/03/01 ...
2
votes
1answer
3k views

Construct / find the simplest function based on data

Let's say I have these 7 natural numbers (all between 0 and 255): 255, 23, 45, 32, 87, 52, 146 How can I find a function F(x) that, once computed, gives me back ...
5
votes
1answer
27k views

How to find curve equation from data?

How do I find the formula when I only know some data points ? Usually I just use the Trendline option for diagrams in Excel, but this one eludes me. I expect it to be something like : ...
1
vote
2answers
398 views

Simple Least Squares Regression?

I have a vector X of 50 real numbers and a vector Y of 50 real numbers. I want to model them as y = ax + b How do I determine a and b such that it minimizes the ...
1
vote
0answers
78 views

Books on Function approximation and Regression

Can you suggest books/articles on Function approximation Let me quote from the above wiki: Second, the target function, call it g, may be unknown; instead of an explicit formula, only a set of ...
0
votes
1answer
90 views

Regression on Linear Model?

I have 50 or so training examples involving a set of 200 or so real numbers (x1,x2,...,x200) (normalized to a 0 mean and std dev 1), and a single output real (y) in the range 0.0..1.0. I want to fit ...
1
vote
5answers
595 views

Find square root approximation function (tool)

first I have to apologize for any uncorrect naming or categorisation of my question, as I am an electrical engineer rather than a mathematican. I try to find a simple solution for my problem: I have ...
0
votes
1answer
2k views

Multiple vs Single Linear Regression

I'm having trouble understanding the relationship between multiple and single linear regression. I have six variables $(x_1, \dots, x_6)$ I'm using in my model. If I check each one individually ...
2
votes
3answers
473 views

How to deal with Linear Regression model with some data aggregated

Lets say I am trying to find a linear regression between Weight and Height of a person. $W=b_0+b_1 H+e$ The data I have gathered from 8 people is like this: ...