Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

4
votes
0answers
130 views

How to compare the similarity between functions?

I'm designing a web service that finds the regression function of a pattern within an image. I analyzed three images and found the following three regressions: 1) $f(x) = 74.7602 + 0.2005x - ...
3
votes
0answers
60 views

Can the sigmoid function approximate any function (or relation) where 0<y<1

I'm studying Machine Learning and Artificial Neural Networks. Some basic principles of Machine Learning are linear regression, multivariate linear regression, and nonlinear regression. The last of ...
3
votes
0answers
130 views

How to perform nonlinear regression with correlated errors?

I have a nonlinear least squares problem, but the errors are correlated. I could use R's nls function to do the regression if the errors were independent, but I don't know the right way to handle ...
3
votes
0answers
235 views

Least Square Method with Positive Parameters

this is my first post here in the Stack Exchange. A friend told me about this forum and I'm giving it a try. I searched a bit past threads, but couldn't find what I wanted, so I'm posting the problem ...
3
votes
0answers
776 views

Correlation and Regression Question

Two separate tests are designed to measure a students ability to solve problems. Several students are randomly selected to take both tests and the results are: $$ \begin{matrix} \text{Test A}(x) ...
2
votes
0answers
29 views

Smallest set of Liner equations, which exactly fit a set of points

I have a set of 2-d points,(it can be of any arbitrary dimension n). I want to find the minimum set of straight lines(linear equations) which exactly passes through the given 2-d points (unlike ...
2
votes
0answers
80 views

Multilinear or Tensor Regression?

Given input data $x_t\in \mathbb{R}^n$ and output data $y_t\in\mathbb{R}^m$, the closed form solution to $\min_A \sum_t \|y_t - Ax_t\|^2_2$ is given by $A = (XX^T)^{-1}XY^T$ where $x_t$ form the ...
2
votes
0answers
42 views

Computing evidence for least-squares fit

I'm at a loss trying to implement Bayesian model selection for standard least-squares polynomials fits. I have three polynomials of order $1$, $2$, and $3$, and a sequence of $(x,y)$ data points. ...
2
votes
0answers
32 views

Coefficient of determination

$$ \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 = ( \displaystyle \sum^n_{i = 1} (y_i - \bar{y})^2 - \displaystyle \sum^n_{i = 1} (y_i - \hat{y}_i)^2 ) + \displaystyle \sum^n_{i = 1} (y_i - ...
2
votes
0answers
106 views

Minimizing L4/ L6/ L2N norm for linear regression

OLS regression minimizes the sum of the squared errors. The normal equation for an OLS for $L_2$ minimization is as follows: $$b= (A'A)^{-1}A'y$$ What would be the equation to minimize the $L_4$ norm ...
2
votes
0answers
114 views

Orthonormal Matrix weighted regression

$Q$ is a rectangular matrix with orthonormal columns. A linear system composed of $$Qx= b$$ is really easy to solve as: $$Q'Q=I$$ hence: $$x=Q'b$$ Given that $Q$ is orthonormal can this be used to ...
2
votes
0answers
77 views

Regressing $Y$ back on the residuals

Suppose I have the linear regression model $ \hat{y_i} = a + b x_i $ for $a,b$ obtained via OLS. How does one regress $y$ back on the residuals $\hat{e}_i = y_i - \hat{y}_i$? If we write $ ...
2
votes
0answers
118 views

Effective model for calculating momentum or growth rate for a time series

I have a series of numbers tracking the performance of an entity on any given day. It's nothing but a simple integer for each date. For example, here's a series for Entity "X" ...
2
votes
0answers
191 views

Bare minimum of points in multiple polynomial regression

I have a question on multiple polynomial regression and the absolute minimum amount of points in the different terms. The minimum amount of points required for a second order polynomial would (in one ...
2
votes
0answers
78 views

Find $w$ as the minimizer of regularized logistic regression

Could someone point me to a reference on how to find $w$ as the minimizer of: $$ \frac{1}{2}\sum_{i=1}^{d}q_i(w_i-m_i)^2+\sum_{j=1}^{n}log(1+\exp(-y_jw^Tx_j)) $$ where $q_i$ (initialized with ...
2
votes
0answers
278 views

Logistic regression algorithm in Casio and Texas Instruments calculators

When using logistic regression on a Casio or Texas Instruments calculator, the output is of the form $$f(x) = \frac{c}{1+ae^{-bx}} $$ The problem I have (when teaching in a class where both types of ...
2
votes
0answers
275 views

Finding a model for multiple non-linear regression

I want to implement a regression analysis, but I have problems with finding a model for the given data. There are $149$ $(x,y,z)$-values. $y$ values are all positive, $x$ is between $[-10, 10]$ and ...
2
votes
0answers
2k views

Derivation of standard error of beta in simple linear regression

Countless web pages show the equation for the standard error of the slope in a simple linear regression. For example: ...
2
votes
0answers
2k views

Help with problem: Estimated Standard Deviation of Regression Equation (Simple Linear)

This is a practice problem. I've solved part (a). I have provided verified answers (from the published key) to all parts (a), (b) and & (c). I need help solving (b) and (c). Consider a simple ...
2
votes
0answers
224 views

Surface Function Fitting to Spherical Data

I have a set of geographic (longitude,latitude,value) data to which I would like to fit surface functions, specifically, the set of quadratic surfaces: $f(x,y)=Ax^2+Bx^2+Cxy+Dx+Ey+F$ At the moment, ...
2
votes
0answers
62 views

Accurate computation for Linear Regression case

I am writing a program that inputs a sequence of points $(x_i,y_i)$ based on the user clicking on certain pixels in an image shown. The program should then find the "best -fitting" line in the least ...
1
vote
0answers
11 views

Problem on Linear Regression

Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean 0 and variance 1; $y_i = α + β(x_i − \bar x) + e_i , i = ...
1
vote
0answers
4 views

Linear regression or ANOVA with unordered independent variable

I have a set of data, let's say describing a group of people. Let's say we know their income and color of hair: ...
1
vote
0answers
11 views

Multinomial Logistic Regression

(1) $$P(y^{(i)} =1\mid X,W) = \frac{\exp(W^{(i)^T}X)}{\sum_{j=1}^m \exp(W^{(j)^T}X)}$$ $W$ and $y$ are vectors where the superscript is an index. And there are $m$ classes (that is, there are $m$ ...
1
vote
0answers
12 views

General 2D taylor surfaces from axial behaviour and discrete points

I have a problem as follows: I have a nonlinear function, f(x,y), for which I (numerically) know the axial behaviours, f(x,y0) and f(x0,y), where x0 and y0 are constants. I can calculate discrete ...
1
vote
0answers
17 views

Predicting y from a log-linear regression

I was wondering if someone could explain to me the very last step on the right hand slide. Why is do we have a sum rather than a product. Thank you very much.
1
vote
0answers
16 views

Logistic regression eye treacting data (need model)

I have two sets of time course data, they are for an eye-tracking study. The data is 20 100ms chunks, one category being percent fixations for canonical sentences, and the other being percent looks ...
1
vote
0answers
12 views

Understanding the regularization parameter in polynomial regression

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...
1
vote
0answers
13 views

Regression factors and covariance matrix

I am trying to follow some notes. They have two matrices. One is called comfact (company factors). This is a 580 x 5 matrix. The 580 rows represent 580 different companies. The 5 columns represent 5 ...
1
vote
0answers
31 views

Linear Regression Question (Linear Algebra) Help!!

Hey guys, I have a quick question. I am trying to prove that the squared sample correlation between fitted and observed values is equal to $R^2$ (coefficient of determination). I am having a lot of ...
1
vote
0answers
19 views

How to find a function that can approximate another blackbox function programmaticly?

This question has been posted on http://stackoverflow.com/questions/21758016/how-to-find-a-function-that-can-approximate-another-blackbox-function-programmat I was suggested to post it here. I ...
1
vote
0answers
35 views

Solving a linear matrix equation with respect to the maximum of the euclidian distances between rows.

With $n>m$, real number matrices $A$, $B$, $C$ are shaped like: $$A=\left( \begin{array}{ccc} a_{1,1} & \cdots & a_{1,m} \\ \vdots & \ddots & \vdots \\ a_{n,m} & \cdots ...
1
vote
0answers
25 views

Adjusting regression for small sample bias

I have a set of data points $\{x_i\}$. These data points are grouped so that (say) $i\in\{1,2,3\}$ is group $A$, $i\in\{4,5,6,7\}$ is group $B$, etc. I would like to test the null hypothesis of no ...
1
vote
0answers
23 views

Polynomial regression - differences between algorithms

I know that I can find a polynomial regression's coefficients doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose). This is a way of finding them; now, there is (as far as I know) at least one other ...
1
vote
0answers
28 views

Linear Regression with multiple equations

I am trying to implement a linear regression algorithm to fit a set of "true" points with their "observed" location. The points are specified using spherical coordinates on a unit sphere. I have a ...
1
vote
0answers
90 views

Formulize / eureqa any replacements?

Greets Now that Formulize / Eureqa now charge $30.00 a month for use and have crippled the trial version does anyone know of any replacements that can do similar things like find an equation given ...
1
vote
0answers
42 views

Linear Regression with limited information

You have grades ($Y $) for men ($D = 0$) and women ($D = 1$). The mean grades (out of total possible score of 100) are 65 for men and 72 for women. Regression of $Y$ on $D$ yields: $Y_i = b_0 + ...
1
vote
0answers
46 views

Multiple regression and hypothesis test $H_0$:$\beta_2=0$

Multiple regression model $H_0$:$\beta_2=0$, $H_1$:$\beta_2 \neq 0$ where $\beta_2$ is the vector of elements ($\beta_2, \beta_3, \dots, \beta_k$) and $\beta$ is slope of regression line. Why it is ...
1
vote
0answers
33 views

Sampling data prior to nonlinear regression

As my question shows it, I am not a statistician. My problem is that I have too many data points to be used in a nonlinear fit (I have millions of them, automatically acquired). Is there a methodology ...
1
vote
0answers
83 views

Multivariate regression model

Supposing I have a graph involving time (x axis time in seconds) and log of nubers of bacteria: I would like to adjust a model given by a formula involving 5 coefficients like: $Number = Log\left ...
1
vote
0answers
355 views

Least Squares “analytic expression” for fitting a 2D quadratic function to measurements

I have n scattered elevation measurements: $ \{x_i,y_i,z_i\}_{i=1..n} $ that I want to fit a quadratic function to: $ z = ax^2 + by^2 + cxy + dx + ey + f$. The problem can be written as a vector ...
1
vote
0answers
35 views

Determining t-values in multiple regression without STDERR(parameter)

If one of the t-values (and its SE Coef) was erased from the output below, how could you still determine its value, from other output shown? Output: The regression equation is Cons = 29.6 + ...
1
vote
0answers
270 views

What is the Moore-Penrose pseudoinverse for scaled linear regression?

The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations: $$ ({\bf{X}}^T \bf{X}) \vec{\beta}= ...
1
vote
0answers
50 views

Conditioning on $X$ equal to premultiplying by $X'$?

I am coming across similar thing in many problems in econometrics and I have not been able to figure out whether it is some general notion or only a "coincidence". To take two examples: Deriving ...
1
vote
0answers
97 views

Broken Line Regression

$X = $Lot & $Y = $Cost Give a broken line linear model with a breakpoint at $250$: $$Y = B_0 + B_1X_1 + B_2X_2 + B_3X_3 + e$$ where $X_2 = 0$ or $1$ depending on whether the lot size is $\geq ...
1
vote
0answers
61 views

Residual Plot Analysis

I'm working on building a regression model for a large set of data (n>54000). Clearly a ton of assumptions are being violated that I have to try and adjust for. I'm all for transformations of data and ...
1
vote
0answers
61 views

Proving $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$

I have asked in another question how $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$. Note that $\hat{y}_h$ = $b_0 + b_1X_h$ which is a regression line ...
1
vote
0answers
224 views

How to calculate probability with sigmoid output in feedforward neural network?

first of all I'm sorry for my not very skilled English, but I will do my best to explain my problem. I'm trying to create a feedforward neural network with one hidden layer (with probably arctan ...
1
vote
0answers
45 views

regression coefficient

Consider observations on three variables X1;X2 and X3: Suppose that X1 is regressed on X2: When the residual of the above regression is regressed on X3; the regression coefficient of X3 is b3: When X1 ...
1
vote
0answers
48 views

About the weights assigned in the linear regression

I have this confusion related to linear regression. Lets say I have two predictors $x_1$ and $x_2$ and the target is $y$. I learn a linear regression with $y \sim x_1,x_1 \cdot x_2,x_2$ with $x_1 ...