Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

3
votes
2answers
598 views

Confidence interval of a random variable for an ordinary linear regression

I have a small problem. With my limited stats background I am not sure I am getting this one right. After fitting an ordinary linear regression model I get ...
3
votes
1answer
425 views

An intuitive explanation for neural networks as function approximators?

I know we use normal linear regression for modeling functions on datasets, but can someone explain how neural networks help in approximating more complex functions, especially when they are nonlinear? ...
0
votes
0answers
41 views

Kalman Filter and OLS Results Are Different

I read that Kalman Filters can be used for continuous / online linear regression and at the end of the regression its results and ordinary linear regression (OLS) results would be the same. I tried it ...
2
votes
0answers
30 views

Kalman filter using regressed model

I'm currently polishing flight control system for KSP, and I'm fightinng high-frequency noise in state vector measurements right now. I want to try to apply Kalman filter to provide more smooth values ...
0
votes
0answers
11 views

Econometrics regression model BLUE [on hold]

Consider the regression model $Y_i=α+ϵi$ where $E(ϵi)=0$ and $Var(ϵi)=σ^2({x_i}^2)$ now how can I construct the GLS and derive the BLUE of α and compute its variance?
4
votes
1answer
462 views

Determine whether ARMA(p,q) is stationary and/or invertible?

Determine whether an ARMA(p,q) process is stationary and invertible such that $y_t = \sum_{i=1}^{p} \phi_i y_{t-i} + \sum_{i=1}^{p} \theta_{i} \epsilon_{t-i}$ with the restriction that ...
0
votes
1answer
399 views

Fast way of finding RSS of Multiple Linear Regression

Is there any smarter way to compute Residual Sum of Squares(RSS) in Multiple Linear Regression other then fitting the model -> find coefficients -> find fitted values -> find residuals -> find norm of ...
1
vote
1answer
654 views

bayesian networks for regression

Would it be possible to use bayesian network for regression and/or prediction? I understand that it is a tool one can use to compute probabilities, but I haven't found much material about possible ...
0
votes
1answer
28 views

Correlation and Linear Regression

I'm tasked with this question but unable to proceed on. Q: Calculate the linear product moment correlation coefficient between x and m for these samples: $$ \Sigma x=205,\\ \Sigma m=1240, \\ \Sigma ...
0
votes
1answer
42 views

Linear Regression without X? :

(Have been working in matrix algebra) Given model: $ y_i = a + e_i$ ( $y_i= α+ϵ_i$ ) That is $y$ subset $i$ and error term subset $i$ Where the expected value of each error term for each entry ...
2
votes
1answer
19 views

Deriving the identity: $\hat{\beta}_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2}$

For some reason I am having an extremely hard time finding out how the following expression is derived $$ \hat{\beta}_1 = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sum (x_i - \bar{x})^2} $$ Is ...
1
vote
3answers
28 views

What is the difference between linear regression on y with x and x with y

I'm plotting the regression line of (GDP$\%$ Change, Poverty Rate$\%$)$\to (x,y)$ in Mathematica What would it mean if I were to switch the axis? (Poverty Rate $\%$, GDP change $%$) ...
0
votes
0answers
12 views

Cox proportional hazards nested regression models - dummy variables vs. numerical variables [on hold]

Background: Suppose there is a Cox proportional hazards model for estimating hazard for cancer patient at different stages of the disease: Stage 1, 2, 3, 4 The first model takes the stages as a ...
15
votes
3answers
6k views

Finding the intersection point of many lines in 3D (point closest to all lines)

I have many lines (let's say 4) which are supposed to be intersected. (Please consider lines are represented as a pair of points). I want to find the point in space which minimizes the sum of the ...
0
votes
0answers
28 views

Curve fitting on non-linear ODE data

Background The graph below was generated by a set non-linear ODEs. For those of you who might want to know: It shows the maximum distance achieved by a cylinder when fired at a specified ...
0
votes
0answers
14 views

How to use leave-one-out cross-validation scheme to compute the accuracy of a linear model fit

Using the least squares estimation I calculated the model fit for a dataset where: $$ p = \beta_{0} + \beta_{1} * t $$ How could I use leave one out cross-validation(CV) scheme to compute accuracy ...
0
votes
2answers
35 views

The best fit for variables in a number of equations?

Let's say I have 2 variables $x$ and $y$ and 4 equations. The parameters in capital are known parameters. $$I_1=xA_1+yB_1$$ $$I_2=xA_2+yB_2$$ $$I_3=xA_3+yB_3$$ $$I_4=xA_4+yB_4$$ What's the strategy ...
0
votes
0answers
13 views

Estimating elasticity of y with respect to x in a log-log specification

The question My rudimentary workings so far is that; log(y_i/x_i) = log(y_i)-log(x_i) Factorise, so, log(y_i/x_i) = log(y_i) + upsilon_i - log(gamma_i + 1) Thus, elasticity of y to x is always >1 ...
1
vote
3answers
46 views

Question regarding Sum Notation in the least squares formula [closed]

I'm attempting to figure out the difference between Σx^2 and (Σx)^2 in this least squares regression formula http://i.imgur.com/HwxnM28.jpg. Any ideas? I figure there must be a difference.
0
votes
0answers
14 views

Fourier transform used for time series prediction?

For a given time series data set $(t=0,...T)$, we can use Fourier transform to data fitness $$ X(n) = \mu + \sum_{k}\left( A_k cos \frac{2\pi k n}{N} + B_k cos \frac{2\pi k n}{N} \right) +\varepsilon ...
0
votes
0answers
9 views

Logistic Regression Varimp Always Different From Other Models; Text Analytics R

I've been running logistic regression, neural networks, naive bayes, and SVM models on my tweets dataset. I'm doing a sentiment analysis, where R is predicting whether a text is positive, neutral, or ...
5
votes
1answer
93 views

Complexity of Gaussian Process algorithms is $\mathcal{O}(n^3)$

It is often quoted that the complexity of Gaussian Process algorithms is $\mathcal{O}(n^3)$ due to the need to invert an $n \times n$ matrix, where $n$ is the number of data points. But as far as I ...
2
votes
1answer
23 views

Can a prediction interval be interpreted as a probability?

Suppose I find a 90% prediction interval for some data distribution. This implies that if I sample large enough data from this distribution, then 90% of such data will lie inside the prediction ...
0
votes
0answers
9 views

Hat matrix and leverages in classical multiple regression

What is Hat matrix and leverages in classical multiple regression? What are their roles? And Why do use them? Please explain them or give satisfactory book/ article references to understand them. ...
1
vote
0answers
27 views

Assigning levels in factorial design.

I am sorry if the question is too basic. Actually while doing some experiment on 2-level factorial design, I assigned +1 to a low level and -1 to high level. I just need the sign of the regression ...
0
votes
1answer
22 views

If X and Z are uncorrelated and Z is normal with mean zero and constant variance, why can I assume Z is zero?

I have a data set that I have used to calculate the coefficients for a linear regression. The data set is of the form $\lbrace x_i,y_i\rbrace_{i=1}^{n} $ Let $$Y = \alpha + \beta X + Z$$ where ...
1
vote
1answer
547 views

Linear Regression Model, linearity in parameters/ variables

I am confusing with the wording here. I was reading a book on linear regression. "The primary concern for linear models is that they display linearity in the parameters. Therefore, when we refer to a ...
1
vote
1answer
22 views

Help with Matrix Regression

The question i have is: Consider two independent random variables $ξ_1$ and $ξ_2$, such that $ξ_1 ∼ N(0,1) $ and $ ξ_2 ∼ N(0,2)$. Let $η_1 =(ξ_1+ξ_2, ξ_2)^{T} ,η_2 =(ξ_1, ξ_1−ξ_2)^{T}$. Find the ...
0
votes
0answers
8 views

Help with Regression question for Revision.

I Have my exam coming up in a few weeks, and am not sure how to go about answering a few questions. One being: For a fixed i = 1,...,n, derive $Cov(\hat{β},Y_{i})$ and $Cov(Y − (\hat{α} − α) − ...
0
votes
0answers
6 views

Multiple Calculations of Dummy Variable effects?

If I am using dummy variables to fit a regression model, I know that I am comparing each variable to whatever the baseline that I decide is. So let's say that I have a dummy variable with 5 levels in ...
0
votes
1answer
18 views

Can regressors be considered as random variables?

In the linear regression model $$y = \beta_1 X_1 + \cdots + \beta_p X_p + \varepsilon \, ,$$ can the regressors $\{X_i\}_{i \in \{1, \ldots, p\}}$ be considered as random variables? I know that what ...
24
votes
3answers
8k views

derivative of cost function for Logistic Regression

I am going over the lectures on Machine Learning at Coursera. I am struggling with the following. How can the partial derivative of ...
0
votes
0answers
14 views

Codification of matrix $X$ in $Y=XB+\epsilon$

The variables for the data below is age, group (treatment 1,2,3), Y response variable. ...
0
votes
1answer
24 views

Formula for finding variables by regression

I'm trying to fit data to the following formula: $$y = a + b x + c/(Sqrt[x]+d)$$ $y=a + b x$ can be fitted easily with linear regression, but I'm lost when it comes to anything more complicated. ...
0
votes
0answers
24 views

what's the difference between the following two main functions

what's the difference between the following two main functions? Let's say if I have a response Y and predictor X and Z and Z is a factor, what's the difference between these two functions 1). Y ~ ...
0
votes
1answer
12 views

Covariance Matrices Help

Consider two independent random variables $ξ_1$ and $ξ_2$, such that $ξ_1 ∼ N(0,1)$ and $ξ_2 ∼ N(0,2)$. Let $η_1 =(ξ_1+ξ_2, ξ_2)^{T}$, $η_2 =(ξ_1, ξ_1−ξ_2)^{T}$. Find the covariance matrix between ...
0
votes
1answer
16 views

how to find AIC values for both models using R software?

I'm studying survival analysis. I estimated both Cox regression model and Buckley&James regression model. In order to determine which model is better for my dataset, I used Akaike Information ...
1
vote
0answers
14 views

Matrix Regression help for exam revision

My regression exam is a month away and i am trying to learn Matrix regression however and struggling with the questions as a whole they are: (a) Consider two independent random variables ξ1 and ξ2, ...
0
votes
0answers
12 views

Regression and Variance Help

If a value for $σ^{2}$ is known, explain briefly how you could use the information from the n pairs of observations to decide whether for every unit change of x there is a significant change in the ...
0
votes
1answer
68 views

Non-Linear Regression for Parameter Estimation

I have a second order system, it's response to a step change can be expressed in the s-space as: $$Y(s)=\frac{K_{2}e^{-\theta s}}{s(\tau_{1} s + 1)(\tau_{2} s + 1)}$$ Which can be inverse ...
1
vote
1answer
929 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
0
votes
0answers
22 views

What would be the best method to model population growth? [migrated]

I am trying to find the best method to model the population growth in a school. In my possession I have the student count by semester in the last 16 years. Moreover the first information I consist of ...
1
vote
1answer
58 views

Geometrical Properties of a curve in 3D

I have $n$ curves in the 3d space, which I represented with a certain amount of points. (That is, for every curve $i$, there is a vector $v_i$ with $m$ points which belong to the curve) My goal is to ...
1
vote
1answer
17 views

calculate the internally studentized residual

This is from my textbook: it says that ...an ordinary residual divided by an estimate of its standard deviation $s(e_{i})$ As we can see from the example that mean for four residuals is 0, so ...
-1
votes
0answers
4 views

Hierarchical model without fixed intercept

I understood it is common sense to almost always use the intercept in a model. Let's assume I have a model withouth intercept, that measure heights. The predictor are also physical measurements ...
0
votes
2answers
12 views

Can you break up a regression slope coefficient into a product of slopes? $\beta_{A,C}=\beta_{A,B}\beta_{B,C}$

Any regression slope coefficient $\beta$ is defined as: $\beta_{X,Y}=Cov(X,Y)/Var(Y)$ It seems intuitive that you can break up a regression slope coefficient like this: ...
0
votes
0answers
35 views

How do I find the line of best fit with only the points?

How do I calculate the line of best fit with the smallest chi-square value for multiple degrees? In other words, given some points, how do I calculate $ax^5+bx^4...fx^0 = y$ WITHOUT USING EXCEL OR A ...
0
votes
1answer
450 views

Convert odds ratio based on unit change to several unit changes

Imagine to have two groups of people, the first one more strongly exposed to a pollutant than the second one, and the first one developing a certain disease more often. Having measurements of the ...
-1
votes
1answer
51 views

linear regression-slope

Your friend in the U.S. gives you a simple regression fit for predicting house prices from square feet. The estimated intercept is -44850 and the estimated slope is 280.76. You believe that your ...
2
votes
1answer
895 views

Understanding Regularization parameters in Machine Learning/Statistics

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...