Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
9 views

Can the dependent variable in a multivariate linear regression be binary when the independent variables are continuous?

Can the dependent variable in a multivariate linear regression be binary when the independent variables are continuous?
0
votes
1answer
26 views

How to fit a set of 3D points to a helical curve?

suppose I have a set of points in $\mathbb{R}^3$, and I want to find an arbitrary helix which best approximates these points. An arbitrary helix in $\mathbb{R}^3$ can be parametrized as ...
6
votes
4answers
5k views

Correlation Coefficient and Determination Coefficient

I'm really new to linear regression and am trying to teach myself. In my textbook there's a problem that asks why $R^{2}$ in the regression of $Y$ on $X =$ the sample correlation between X and Y the ...
1
vote
1answer
50 views

How to Fit a Curve to a Given Model with Constraints?

The input are triples $\left\{ x,y,v\right\}$ where $x,y,v \in \mathbb{R} ^{+}$ I need to find function $f(x,y) = v$ by finding parameters of the following model $f(x,y) = a + bx^c + dy^e $ ...
0
votes
0answers
11 views

Realtionship between Statical test and CI, and restriction of the regression model. [on hold]

$1$.What is the relationship between a statistical test, such as the 𝑡-test, and a confidence interval? $2$.The linear regression model is excessively restrictive since it only allows for a ...
-1
votes
0answers
44 views

Overfitting: Mean and variance

I have read in a book the next porperty: Lets consider the following true data generating process: $$y=x_1 \beta_1+...+x_p \beta_p + \epsilon = x'true \beta + \epsilon$$ where $E(\epsilon)=0 \ \ \ ...
1
vote
1answer
962 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
0
votes
0answers
4 views

Calculating person product moment correlation coefficient on a 3 X 3 table

Usually we are given problems that only involve 2 rows (x and y), but recently saw a problem asking how to compute the correlation coefficient on a table of data that has 3 rows and am not sure how to ...
0
votes
0answers
25 views

Linear or nonlinear modell

Given those three modells and the assignment to decide whether or not those modells can be transformed into linear modells: (a) $Y_i = \beta_0 + ...
2
votes
1answer
38 views

Kalman filter using regressed model

I'm currently polishing flight control system for KSP, and I'm fightinng high-frequency noise in state vector measurements right now. I want to try to apply Kalman filter to provide more smooth values ...
1
vote
0answers
20 views

Local quadratic approximation

I wanted to implement some penalized regression parameter estimation algorithm by Fan&Li (http://sites.stat.psu.edu/~rli/research/penlike.pdf, section 3.3), but cannot catch the idea of some ...
1
vote
0answers
20 views

What would be the standard errors of this transformed regression model, given that I know the standard errors of the original model

Say I have the following regression model: $\ln\left(\dfrac{y_i}{x_{2i}}\right)=\alpha_1+\alpha_2\ln(x_{2i}) + \alpha_3\ln(x_{3i}) +e_i$ where I know the values of the regression coefficients and ...
1
vote
0answers
17 views

Does increasing sample size have any effect on omitted variable bias?

Say I have a multiple linear regression model, where two of the variables are positively correlated, and I omit one of these variables from the model. First question - if I increase the sample size, ...
0
votes
0answers
18 views

Coefficient Correlation r of Exponential Functions Regression

I'm writing an exponent regression calculator $Ae^{Bx}$ Sample Data Set (X,Y) is (9, 1) (7, 10) (6,11) (20, 10) (15, 1) A = 5.287 and B = -0.0232. So $F(x) = ...
0
votes
0answers
11 views

What are the causes of overfitting in regression/classification for statistical data?

Say I have some n-dimensional data, and I want to come up with some hypothesis function which generalizes that data for future predictions in the model. "Overfitness" of my hypothesis function is a ...
0
votes
0answers
19 views

$n$th order Polynomial for $(n+1)$ points

I was reading about Polynomial Fitting and found this sentence: How can one reach this conclusion and prove it?
0
votes
0answers
24 views

Consequences of fitting a regression model with an intercept term when it should be through the origin

Suppose a true model is $Y_i=\beta X_i +e_i$, where $e$ is the random error. Suppose instead we fit the model (using least squares) as $Y_i=\alpha_0+\alpha_1 X_i +v_i$, where $v$ is the random error. ...
0
votes
1answer
494 views

Convert odds ratio based on unit change to several unit changes

Imagine to have two groups of people, the first one more strongly exposed to a pollutant than the second one, and the first one developing a certain disease more often. Having measurements of the ...
0
votes
0answers
15 views

What is the equivalent of $R^2$ ( coefficient of determination in linear regression) for non linear regression?

I have a dataset with two correlating variables. The relation cannot be described as: $y= a+bx$. Therefore I told my math programm to calculate a nonlinear regression line. But unlike in linear ...
0
votes
0answers
15 views

Using optim and fitdistr in R to find parameters

I am using R to fit distributions. I have been given the data and have been asked to find the optimised parameters(for lognormal, weibull, exponential and gamma functions) using: 1) fitdistr and 2) ...
0
votes
0answers
9 views

What does fixed regressor say about our linearity condition?

The linearity condition states that $y_i=(\vec{x}_i)^{T}\vec{\beta}$ for all $i$. Now, if we have fixed regressors, $\{\vec{x}_1,\vec{x}_2,\cdots\}$, our linearity condition only says for those ...
0
votes
0answers
14 views

An example where pearson is wildy different to spearman? [duplicate]

Im looking to spearman and pearson, and from what i understand spearman is better at looking at curves. Can i see an example of a small set of data (10 or less) where this difference is large.
0
votes
2answers
34 views

Linear Regression - Predction

With this question I can input data and I can find the linear regression line - but I am totally failing to get the last part - predicting how many hats will be sold in $2017$. How do you do it. ...
1
vote
1answer
44 views

Fit a Quadratic Curve to Data

I have some data and I want to fit a quadratic curve for my data But I don't know that how to it do? My data : $x,y = 100,45;$ $x_1,y_1= 101, 50$; $x_2,y_3=99,35$; $\ldots$ For instance this ...
0
votes
0answers
14 views

An example of when pearson or regression analysis is drastically different to spearman?

Im looking into spearmans rank. I know pearson and regression struggles with curves, but does anyone have any example of when pearson or regression differs with spearman and what this means? Ideally ...
-3
votes
0answers
22 views

How to Fitting Function to Data

Hi I have some data And I want to fit a polynom these data Firstly I think it should be : $ax^2 +bx+c$ and then $x+x1=-\frac{b}{a}$ ;$x*x1=\frac{c}{a}$ but I can understand that This method ...
0
votes
1answer
26 views

What is the difference in how $\mathrm{R}^2$ and $\mathrm{R}$ values are interpreted?

In statistics, there is the $\mathrm{R}$ value for the product moment correlation coefficient and the $\mathrm{R}^2$ value for the coefficient of determination. In both cases they are described as a ...
0
votes
0answers
7 views

Without homoscedasticity, is OLS still the best estimator (aka BestLinearUnbiasedEstimator…BLUE)?

Consider the Gauss Markov assumptions. Suppose we have a random sample $\lbrace x_n,y_n \rbrace_{n=1}^{N}$. Assume for a simple linear regression model $y_n = \beta_0 + \beta_1 x_n + \varepsilon_n$ we ...
-1
votes
0answers
13 views

How is the $R^2$ value for exponential regression calculated if not by product moment correlation coefficient?

I am analysing some $x$ and $y$ values using Excel by plotting them on a graph and adding a line of best fit then using the equation for the lines of best fit. The exponential line of best fit has a ...
0
votes
0answers
36 views

Automatic curve fitting to find order of an algorithm?

I'm a newbies in mathematics. I'm looking for an automatic best curve fitting function to find the order of an algorithm. I would like to know if it does exists a math library function that would ...
1
vote
1answer
173 views

Can an Artificial Neural Network with Only One Hidden Layer Suffice for all Applications?

I have heard that only a single layer is needed for an artificial neural network to fit any possible function (input to output). 1. Is this true and where is this shown? 2. If this is true, then ...
1
vote
2answers
67 views

How to Change Summation Expression $\sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{\Omega}^{-1}\mathbf{X}_i$ into Matrix Expression

Let $\mathbf{X}_i$ be a $G \times K$ matrix, and suppose are $i=1,...,N$ of these matrices. Note that \begin{align} \sum_{i=1}^N \mathbf{X}_i^{\top}\mathbf{X}_i &= \begin{bmatrix} ...
3
votes
1answer
433 views

An intuitive explanation for neural networks as function approximators?

I know we use normal linear regression for modeling functions on datasets, but can someone explain how neural networks help in approximating more complex functions, especially when they are nonlinear? ...
3
votes
2answers
601 views

Confidence interval of a random variable for an ordinary linear regression

I have a small problem. With my limited stats background I am not sure I am getting this one right. After fitting an ordinary linear regression model I get ...
0
votes
0answers
42 views

Kalman Filter and OLS Results Are Different

I read that Kalman Filters can be used for continuous / online linear regression and at the end of the regression its results and ordinary linear regression (OLS) results would be the same. I tried it ...
4
votes
1answer
481 views

Determine whether ARMA(p,q) is stationary and/or invertible?

Determine whether an ARMA(p,q) process is stationary and invertible such that $y_t = \sum_{i=1}^{p} \phi_i y_{t-i} + \sum_{i=1}^{p} \theta_{i} \epsilon_{t-i}$ with the restriction that ...
0
votes
1answer
414 views

Fast way of finding RSS of Multiple Linear Regression

Is there any smarter way to compute Residual Sum of Squares(RSS) in Multiple Linear Regression other then fitting the model -> find coefficients -> find fitted values -> find residuals -> find norm of ...
1
vote
1answer
668 views

bayesian networks for regression

Would it be possible to use bayesian network for regression and/or prediction? I understand that it is a tool one can use to compute probabilities, but I haven't found much material about possible ...
0
votes
1answer
29 views

Correlation and Linear Regression

I'm tasked with this question but unable to proceed on. Q: Calculate the linear product moment correlation coefficient between x and m for these samples: $$ \Sigma x=205,\\ \Sigma m=1240, \\ \Sigma ...
0
votes
1answer
45 views

Linear Regression without X? :

(Have been working in matrix algebra) Given model: $ y_i = a + e_i$ ( $y_i= α+ϵ_i$ ) That is $y$ subset $i$ and error term subset $i$ Where the expected value of each error term for each entry ...
2
votes
1answer
19 views

Deriving the identity: $\hat{\beta}_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}$

For some reason I am having an extremely hard time finding out how the following expression is derived $$ \hat{\beta}_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2} $$ Is ...
1
vote
3answers
29 views

What is the difference between linear regression on y with x and x with y

I'm plotting the regression line of (GDP$\%$ Change, Poverty Rate$\%$)$\to (x,y)$ in Mathematica What would it mean if I were to switch the axis? (Poverty Rate $\%$, GDP change $%$) ...
15
votes
3answers
6k views

Finding the intersection point of many lines in 3D (point closest to all lines)

I have many lines (let's say 4) which are supposed to be intersected. (Please consider lines are represented as a pair of points). I want to find the point in space which minimizes the sum of the ...
0
votes
0answers
32 views

Curve fitting on non-linear ODE data

Background The graph below was generated by a set non-linear ODEs. For those of you who might want to know: It shows the maximum distance achieved by a cylinder when fired at a specified ...
0
votes
0answers
14 views

How to use leave-one-out cross-validation scheme to compute the accuracy of a linear model fit

Using the least squares estimation I calculated the model fit for a dataset where: $$ p = \beta_{0} + \beta_{1} * t $$ How could I use leave one out cross-validation(CV) scheme to compute accuracy ...
0
votes
2answers
36 views

The best fit for variables in a number of equations?

Let's say I have 2 variables $x$ and $y$ and 4 equations. The parameters in capital are known parameters. $$I_1=xA_1+yB_1$$ $$I_2=xA_2+yB_2$$ $$I_3=xA_3+yB_3$$ $$I_4=xA_4+yB_4$$ What's the strategy ...
0
votes
0answers
16 views

Estimating elasticity of y with respect to x in a log-log specification

The question My rudimentary workings so far is that; log(y_i/x_i) = log(y_i)-log(x_i) Factorise, so, log(y_i/x_i) = log(y_i) + upsilon_i - log(gamma_i + 1) Thus, elasticity of y to x is always >1 ...
1
vote
3answers
46 views

Question regarding Sum Notation in the least squares formula [closed]

I'm attempting to figure out the difference between Σx^2 and (Σx)^2 in this least squares regression formula http://i.imgur.com/HwxnM28.jpg. Any ideas? I figure there must be a difference.
0
votes
0answers
10 views

Logistic Regression Varimp Always Different From Other Models; Text Analytics R

I've been running logistic regression, neural networks, naive bayes, and SVM models on my tweets dataset. I'm doing a sentiment analysis, where R is predicting whether a text is positive, neutral, or ...
5
votes
1answer
93 views

Complexity of Gaussian Process algorithms is $\mathcal{O}(n^3)$

It is often quoted that the complexity of Gaussian Process algorithms is $\mathcal{O}(n^3)$ due to the need to invert an $n \times n$ matrix, where $n$ is the number of data points. But as far as I ...