Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
314 views

Recursive coefficient of determination (R2)

Is there a way to compute the coefficient of determination $R^2$ in a recursive way? $R^2$ is defined as following: $$R^2 \equiv 1 - \frac{SS_{\rm err} }{ SS_{\rm tot}} = 1 - \frac{\sum_i (y_i - ...
1
vote
1answer
122 views

can an artificial neural network with only one hidden layer fit all purposes/applications/functions?

I have heard that only a single layer is needed for an ANN to fit any possible function (input to output). Is this true and where is this investigated/state/found? Then what is the advantage of having ...
1
vote
1answer
20 views

properties of least square estimators in regression

$Y_i=\beta_0+\beta_1 X_i+\epsilon_i$ where $\epsilon_i$ is normally distributed with mean $0$ and variance $\sigma^2$ . The least square estimators of this model are $\hat\beta_0$ and $\hat\beta_1$. ...
1
vote
0answers
14 views

Problem on Linear Regression

Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean 0 and variance 1; $y_i = α + β(x_i − \bar x) + e_i , i = ...
1
vote
0answers
4 views

Linear regression or ANOVA with unordered independent variable

I have a set of data, let's say describing a group of people. Let's say we know their income and color of hair: ...
1
vote
0answers
26 views

What statistical tests for headache journal? [migrated]

I track my pain levels in an online spreadsheet along with daily habits and trigger events. I want to test whether changes in my pain over time follow a trend (not concerned whether it is linear or ...
1
vote
0answers
13 views

Multinomial Logistic Regression

(1) $$P(y^{(i)} =1\mid X,W) = \frac{\exp(W^{(i)^T}X)}{\sum_{j=1}^m \exp(W^{(j)^T}X)}$$ $W$ and $y$ are vectors where the superscript is an index. And there are $m$ classes (that is, there are $m$ ...
1
vote
0answers
12 views

General 2D taylor surfaces from axial behaviour and discrete points

I have a problem as follows: I have a nonlinear function, f(x,y), for which I (numerically) know the axial behaviours, f(x,y0) and f(x0,y), where x0 and y0 are constants. I can calculate discrete ...
1
vote
0answers
17 views

Predicting y from a log-linear regression

I was wondering if someone could explain to me the very last step on the right hand slide. Why is do we have a sum rather than a product. Thank you very much.
1
vote
0answers
16 views

Logistic regression eye treacting data (need model)

I have two sets of time course data, they are for an eye-tracking study. The data is 20 100ms chunks, one category being percent fixations for canonical sentences, and the other being percent looks ...
1
vote
2answers
54 views

derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
1
vote
0answers
23 views
+50

Understanding Regularization parameters in Machine Learning/Statistics

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...
1
vote
0answers
13 views

Regression factors and covariance matrix

I am trying to follow some notes. They have two matrices. One is called comfact (company factors). This is a 580 x 5 matrix. The 580 rows represent 580 different companies. The 5 columns represent 5 ...
1
vote
1answer
15 views

Regression analysis question

I do writing that involves correlation studies, but I am not a mathematician. I am considering extending my research to golf, but wonder whether the nature of golf scoring makes that unfeasible. In ...
1
vote
0answers
31 views

Linear Regression Question (Linear Algebra) Help!!

Hey guys, I have a quick question. I am trying to prove that the squared sample correlation between fitted and observed values is equal to $R^2$ (coefficient of determination). I am having a lot of ...
1
vote
2answers
47 views

Find parameters for curve fitting (simple linear regression involved?)

I would like to fit data in g~t scatterplot, where ...
1
vote
0answers
20 views

How to find a function that can approximate another blackbox function programmaticly?

This question has been posted on http://stackoverflow.com/questions/21758016/how-to-find-a-function-that-can-approximate-another-blackbox-function-programmat I was suggested to post it here. I ...
1
vote
3answers
29 views

Different Regression Lines?

Hi quick question with regression. If the coefficients of a simple regression line, B0 and B1, are the same then why are the regression lines of y on x and x on y different given the condition r^2 ...
1
vote
1answer
35 views

Calculating variance of estimated intercept parameter, $\hat\beta_0$

I have the following sample : $$ \begin{array}{c|lr} X&80&100&120&140&160&180&200&220&240&260\\ \hline Y & 70 ...
1
vote
0answers
35 views

Solving a linear matrix equation with respect to the maximum of the euclidian distances between rows.

With $n>m$, real number matrices $A$, $B$, $C$ are shaped like: $$A=\left( \begin{array}{ccc} a_{1,1} & \cdots & a_{1,m} \\ \vdots & \ddots & \vdots \\ a_{n,m} & \cdots ...
1
vote
0answers
26 views

Adjusting regression for small sample bias

I have a set of data points $\{x_i\}$. These data points are grouped so that (say) $i\in\{1,2,3\}$ is group $A$, $i\in\{4,5,6,7\}$ is group $B$, etc. I would like to test the null hypothesis of no ...
1
vote
1answer
38 views

How to calculate $\sum(X_i-\bar{X})^2$ in R

I'm trying to figure out how to calculate $\sum(X_i-\bar{X})^2$ in R, specifically identifying it in either the aov function or $\operatorname{lm}(y\sim x)$ function. I am trying to use it to ...
1
vote
0answers
23 views

Polynomial regression - differences between algorithms

I know that I can find a polynomial regression's coefficients doing $(X'X)^{-1}X'y$ (where $X'$ is the transpose). This is a way of finding them; now, there is (as far as I know) at least one other ...
1
vote
0answers
28 views

Linear Regression with multiple equations

I am trying to implement a linear regression algorithm to fit a set of "true" points with their "observed" location. The points are specified using spherical coordinates on a unit sphere. I have a ...
1
vote
0answers
90 views

Formulize / eureqa any replacements?

Greets Now that Formulize / Eureqa now charge $30.00 a month for use and have crippled the trial version does anyone know of any replacements that can do similar things like find an equation given ...
1
vote
0answers
42 views

Linear Regression with limited information

You have grades ($Y $) for men ($D = 0$) and women ($D = 1$). The mean grades (out of total possible score of 100) are 65 for men and 72 for women. Regression of $Y$ on $D$ yields: $Y_i = b_0 + ...
1
vote
1answer
42 views

Best fit line using geometric distance (not vertical distance)

There must be a theory of finding the best fit line to a bunch of points in the plane, where "best fit" is defined by the geometric distance, not vertical distance. In other words, we are trying to ...
1
vote
1answer
25 views

Non-close-form Regression Research

As I try to process some physic experiment data that I don't have the closed form formula with unknown parameters, I have to use some regression models like polynomials or normal distributions . The ...
1
vote
0answers
46 views

Multiple regression and hypothesis test $H_0$:$\beta_2=0$

Multiple regression model $H_0$:$\beta_2=0$, $H_1$:$\beta_2 \neq 0$ where $\beta_2$ is the vector of elements ($\beta_2, \beta_3, \dots, \beta_k$) and $\beta$ is slope of regression line. Why it is ...
1
vote
1answer
40 views

Implicit Curve Fitting

I have 100 points scattered in the 3D space along the $z$ coordinate axis. The points appear to lie on a curve. Is it possible to find an (implicit) curve that fit these points and option to insert ...
1
vote
0answers
33 views

Sampling data prior to nonlinear regression

As my question shows it, I am not a statistician. My problem is that I have too many data points to be used in a nonlinear fit (I have millions of them, automatically acquired). Is there a methodology ...
1
vote
1answer
93 views

Mathematical model building with dependent and independent variables

I have been working with data and building models on data. I have developed models in regression using cubic and power series. It works fine for variables with one dependent and one independent ...
1
vote
0answers
84 views

Multivariate regression model

Supposing I have a graph involving time (x axis time in seconds) and log of nubers of bacteria: I would like to adjust a model given by a formula involving 5 coefficients like: $Number = Log\left ...
1
vote
1answer
24 views

Iterative Power Regression

If I have a set of data points that would fit inside a power equation of the form y = a*x^b, what is the best ITERATIVE method to find the values of 'a' and 'b'. I thought I could compute the error ...
1
vote
0answers
360 views

Least Squares “analytic expression” for fitting a 2D quadratic function to measurements

I have n scattered elevation measurements: $ \{x_i,y_i,z_i\}_{i=1..n} $ that I want to fit a quadratic function to: $ z = ax^2 + by^2 + cxy + dx + ey + f$. The problem can be written as a vector ...
1
vote
1answer
113 views

Relationship between lagrange multiplier and constraint

I know there is one to one relationship between $\lambda$ and $t$ in the following two equivalent optimization formulation. But what is exact relationship? A) $$ \sum_i(y_i - \sum_k \beta_k ...
1
vote
1answer
73 views

How to apply logistic regression to analyse and predict kind of problem

Say I want to predict whether a new question posted in stack overflow will get an answer within 24 hrs. I was given details about previous questions and all data. Now I have thousands of observations ...
1
vote
0answers
35 views

Determining t-values in multiple regression without STDERR(parameter)

If one of the t-values (and its SE Coef) was erased from the output below, how could you still determine its value, from other output shown? Output: The regression equation is Cons = 29.6 + ...
1
vote
1answer
34 views

Question regarding Bayesian VAR

Reference: http://support.sas.com/rnd/app/da/new/801ce/ets/chap4/sect30.htm. So there is a VAR equation that is to be treated in Bayesian way: $\mathbb{y} = (X \otimes I_k)\beta + e$ where $\beta$ ...
1
vote
0answers
273 views

What is the Moore-Penrose pseudoinverse for scaled linear regression?

The matrix equation for linear regression is: $$ \vec{y} = X\vec{\beta}+\vec{\epsilon} $$ The Least Square Error solution of this forms the normal equations: $$ ({\bf{X}}^T \bf{X}) \vec{\beta}= ...
1
vote
0answers
50 views

Conditioning on $X$ equal to premultiplying by $X'$?

I am coming across similar thing in many problems in econometrics and I have not been able to figure out whether it is some general notion or only a "coincidence". To take two examples: Deriving ...
1
vote
1answer
73 views

Nonlinear regression analysis of a vector

I'm trying to get a nonlinear fit of a vector in Matlab with no success. Let's assume that I have a vector called data: data = [1,30,250,55,22,76] which can be ...
1
vote
0answers
97 views

Broken Line Regression

$X = $Lot & $Y = $Cost Give a broken line linear model with a breakpoint at $250$: $$Y = B_0 + B_1X_1 + B_2X_2 + B_3X_3 + e$$ where $X_2 = 0$ or $1$ depending on whether the lot size is $\geq ...
1
vote
0answers
61 views

Residual Plot Analysis

I'm working on building a regression model for a large set of data (n>54000). Clearly a ton of assumptions are being violated that I have to try and adjust for. I'm all for transformations of data and ...
1
vote
0answers
62 views

Proving $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$

I have asked in another question how $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$. Note that $\hat{y}_h$ = $b_0 + b_1X_h$ which is a regression line ...
1
vote
0answers
227 views

How to calculate probability with sigmoid output in feedforward neural network?

first of all I'm sorry for my not very skilled English, but I will do my best to explain my problem. I'm trying to create a feedforward neural network with one hidden layer (with probably arctan ...
1
vote
0answers
45 views

regression coefficient

Consider observations on three variables X1;X2 and X3: Suppose that X1 is regressed on X2: When the residual of the above regression is regressed on X3; the regression coefficient of X3 is b3: When X1 ...
1
vote
1answer
21 views

Initializing Variables using Shrinkage

I have a user-user model which which users can rate their friendships(r) with others and also can have activities with them(a). I am using Matrix Factorization and Gradient Descent for updating the ...
1
vote
0answers
48 views

About the weights assigned in the linear regression

I have this confusion related to linear regression. Lets say I have two predictors $x_1$ and $x_2$ and the target is $y$. I learn a linear regression with $y \sim x_1,x_1 \cdot x_2,x_2$ with $x_1 ...
1
vote
0answers
37 views

Sequential problem for n=1, non linear regression

I am trying to understand an example in my stats course notes, the example relates to calculating the best value for the next experiment. The function of the line is very simple: $$ln(Y_i) = ...