Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
21 views

Quantile as solution to minimization problem

I'm studying basics of quantile regression now and I have trouble prooving that $\tau-$th quantile of real-valued random variable $Y$ is a solution to the following minimization problem (in the ...
0
votes
0answers
20 views

How to calculate the coordinate of a point which depends on other points on a plane with specific distances

I have $8$ points on a plane $(x_1,y_1)....(x_8,y_8)$ among these $8$ points I know the coordinates for $7$ points and I have to find the $8^{th}$ point. Each points has the difference between all ...
0
votes
0answers
9 views

Ordered logistic regression with likert scales

I'm currently have a bit of difficulty determining how to analyze this data via logistic regression analysis. Q18 = DV (satisfaction score ranging from 1-10) Q10_1 = IV (Customer Service likert ...
0
votes
1answer
17 views

Regression when the variance of the residuals depends on the independent variable

When the residuals follow a normal distribution, the most likely function that fits the data is found using least squares. In that case: $y = f(x_i) + r_i, \quad r\sim\mathcal{N}(0, \sigma^2)$ ...
0
votes
1answer
11 views

Contribution of each variable in multiple linear regression

What will be the best measure of the contribution of a variable in multiple linear regression? I was thinking of using the coefficient ratio as a marker of a variable's contribution. For example: ...
4
votes
2answers
60 views

Rewriting the matrix equation $AX = YB$ as $Y = CX$?

Is it possible in general, if $A,B,C,X,Y$ are square and of the same dimensions? If so, does it generalize to non-square matrices (using a pseudoinverse)? I'm doing some curve fitting in which I have ...
1
vote
0answers
7 views

Finding posterior of normal distributions and logistic regression.

$P(w_0 | x) = \frac{1}{1 + e^{-log\frac{P(x|w_0)}{P(x|w_1)}-log\frac{P(w_0)}{P(w_1)}}}$ Note: x = $[x_1, \dots, x_d]^T$; a $d$ dimensional vector. $w$ can take on one of two values: $w_0$ or $w_1$. ...
0
votes
0answers
12 views

Spatially model

Can someone explain to me what a 'spatially lagged autoregressive model' is ? I came across this 'model' by searching new techniques for modeling geographical data.
0
votes
0answers
18 views

Estimation of real estate

I'm working on a project on estimating the price value of real estate. First I have collected a lot of data (500 000 instances) with details such as postal code, number of bedrooms, build year, ...
0
votes
1answer
22 views

Find optimal least square solution to the normal equation

What is the optimal solution for $\beta_1$ and $\beta_2$ in the following normal equation: $$\beta _{ 1 }\sum _{ i=1 }^{ n }{ { x }_{ i } } +\beta _{ 0 }=\sum _{ i=1 }^{ n }{ { y }_{ i } } $$ EDIT ...
1
vote
1answer
30 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
3
votes
0answers
27 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
1
vote
0answers
17 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
0
votes
0answers
9 views

Derivative and linear fitting model

Let $V=v_1,v_2,\ldots,v_n$ be the measured velocities and $A=a_1,a_2,\ldots,a_n$ be the measured accelerations of a vehicle at times $T=t_1,t_2,\ldots,t_n$. Let $Y=c_1+c_2t+c_3t^2$ be the best ...
1
vote
1answer
19 views

Model selection in regression: Estimated parameters seem to be “non-significant”

I have conducted an experiment which manipulated three factors (Factor 1: 3 levels, Factor 2: 2 levels, Factor 3: 2 levels). The response variable is binomially distributed (1 = correct or 0 = not ...
1
vote
1answer
29 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
1
vote
1answer
13 views

Dummy recoding for more than two categorical variables

Say I am doing a study with 3 different types of fruit and I want to make a regression depending on the type that tries to predict the amount sold. I know that I could make 2 dummy variables: orange ...
1
vote
1answer
30 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
0
votes
1answer
31 views

Linear regression as $\dim(\beta) \rightarrow \infty$

Consider the linear regression, $$ Y_i = X_i\beta + U_i \qquad E[X_i'U_i]=0 $$ where $X_i=(1,W_{i},W_{i}^2,..\ldots,W_i^K)$ and $\beta \in \mathbb{R}^{K+1}$. The joint distribution of $(X_i,Y_i)$ is ...
0
votes
1answer
22 views

How to create a model with high multicollinearity

I am going to create a model strictly for predictive purposes. Some of my independent variables are highly correlated. When I try to create the regression with all of the variables together, then, I ...
0
votes
1answer
15 views

Can I make one of my predictors categorical and continuous?

I would like to add age as a predictor for my regression, but I would also like to make it a binary categorical predictor with a cutoff of 18 years of age. I would like to do this because I suspect ...
0
votes
0answers
25 views

Metric for movement in 2D space

I have a set of points that represent the coordinates of an object moving in the 2D space at different points in time. Using this points I want to get a "measurement" that will describe the general ...
0
votes
0answers
22 views

Trying to prove Slope of linear regression without Calculus

Hi so what I'm trying to do is prove that the slope of a simple linear regression line is equal to the $cor(X,Y) * sd(Y)/sd(X)$ without using calculus(I know it can be done easily with calculus but ...
2
votes
0answers
26 views

Recursive Least Square (RLS) algorithm for regression. [closed]

What is meant by convergence of recursive least square (RLS) in online regression ?
0
votes
1answer
36 views

Derivative of an exponentially weighted moving average

It has been a while since my university math courses, so let me apologize right off the bat... I'm using GSL to perform non-linear regression analysis and am mostly happy with the outcome, however, ...
0
votes
0answers
7 views

Convert odds ratio based on unit change to several unit changes

Imagine to have two groups of people, the first one more strongly exposed to a pollutant than the second one, and the first one developing a certain disease more often. Having measurements of the ...
2
votes
1answer
24 views

Gaussian prior favors values closest to zero?

I am reading an article on Bayesian Logistic Regression, where they're using Logistic Regression, imposing a Gaussian prior (with mean = 0) on its parameters. They state that a Gaussian prior favors ...
0
votes
0answers
15 views

evaluation of polynomial regression

I have a data set $(x_i$ $y_i)$ if=1...20. I have to fit the data using polynomial feature. How can I evaluate what the complexity of model should be chosen? There is a hint in the task using RMSE ...
4
votes
1answer
25 views

Using regression results to predict?

I run some Poisson regressions with the following results: (with number of associations an individual belongs to as the dependent variable) ...
-1
votes
0answers
8 views

Bayes Classifier for items with correlation

I am familiar with Bayes classifiers and regression in general, but I am not a mathematician and so I was hoping someone here could answer this. I have a set of objects, for this purpose lets say ...
0
votes
1answer
14 views

Derivation of SSE gradient for Linear Regression

In my text book it gives the derviation of the gradient for the likelihood of a linear regression model (to minimize the negative log likelihood by minimizing the Sum Squared Error). The first line ...
5
votes
2answers
56 views

Update a regression on the fly?

Say I have 100 people each with a height, weight, and age. I make a regression that predicts age based on height and weight. Now, I would like to update that model when I meet someone new. I don't ...
0
votes
0answers
20 views

R squared (Proportion of variance explained) in terms of conditional variance?

My question concerns a comparison between 2 models in terms of proportion of variance explained. Let $y_{t+1}$ denotes the variable I want to explain or predict and $\mathcal{F}_t$ the information ...
-1
votes
0answers
22 views

Which subset of variables is best in multiple regression

I had a question regarding multiple regression and choosing the best subset. In my textbook it says you can find the best fit by testing the regression of all the variables in subsets (e.g if there ...
0
votes
0answers
24 views

Unconditional sampling distribution of regression coefficients

I am trying to find the (unconditional) sampling distribution of a regression coefficient in a simple linear regression. The linear regression $Y = \beta_0 + X\beta_1 + \epsilon$ is conducted for $N$ ...
1
vote
1answer
15 views

How to perform a monotonic function fitting of data points?

I'm seeking suggestions for general purpose function fitting of a set of data points, where, based on physical intuition, the relationship is expected to be "monotonic", i.e. the function should be ...
0
votes
2answers
35 views

Testing if $X_1$ has an influence of $Y$

Consider you have the suspicion that $Y$ is influenced by two attributes $X_1$ and $X_2$: $$ Y=\theta_0+\theta_1X_1+\theta_2X_2+\theta_3 X_1X_2+U $$ The following data are given. Test ...
0
votes
0answers
12 views

Multivariate Multiple Regression with Repeated Measures

I have a dataset with p predictors for i items (so multiple regression). For each of s subjects, I have r repeated observations of v dependent variables (so it's a multivariate problem). I wish to ...
0
votes
0answers
12 views

Do I have classical measurement error?

this is an econometrics question, so I hope I'm in the right place. Consider the following OLS regression: Y = alpha + beta X + M + epsilon I'm interested in the effect of X on Y. For some ...
0
votes
1answer
23 views

When residual standard error is equal to standard deviation of dependent variable in linear regression?

I wonder when residual standard error is equal to standard deviation of dependent variable in linear regression? Could someone provide some information on this topic and explanation?
0
votes
1answer
40 views

Guess a function that fits empirical data

This is my empirical data: Which function it looks like? I tried to guess (1) a dumped (exponential decaying) sinusoidal, but it does not oscillate after overshoot; (2) a sigmoid, but it oscillate ...
0
votes
1answer
30 views

Is the sum of predicted y values equal to the sum of actual y values?

Say I have a set of points Y and I want to accuratly predict the values of Y by using three variables X1,X2,X3. Hence my equation is Y=intercept + C1*X1 + C2*X2 + C3*X3 After performing linear ...
0
votes
0answers
14 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
1
vote
2answers
37 views

Autocorrelation and var-cov matrix

$$Y_t=\beta_1+\beta_2 X_{t2}+\dots +\beta_k X_{tk}+\epsilon_t \qquad (t=1,\dots,T)$$ $$\epsilon_t=\rho \epsilon_{t-1}+v_t, \qquad v_t \sim \mathrm{i.i.d.}(0,\sigma^2_v)$$ GLS estimation under AR(1) ...
1
vote
0answers
25 views

Properties of best linear predictor?

Conside two scalar random variables, $Y,X$. The best linear predictor of $Y\mid X$ under square loss function is $\theta_0=\operatorname{argmin}_{\theta} ...
2
votes
1answer
46 views

Trigonometric regression

What methods are performed for regression with trigonometric functions? E.g. : Sequence: $$-1, 0, 1, -1, 0, 1, \text{.....}$$ Regression: ...
3
votes
2answers
63 views

Why Linear Regression

First i will like to say that i am not a statistician nor am i good in the field. I have been collecting data for over a period of e.g 100 days and each day has a varying amount of data that i can ...
0
votes
2answers
24 views

Strong vs weak relationship in this correlation

I produced this plot and regression line in R and I thought my results were quite odd. Is the relationship of the correlation determined by how steep the regression line is? So in this case it isn't ...
0
votes
1answer
28 views

Logistic Regression derivation

From the Wikipedia article http://en.wikipedia.org/wiki/Multinomial_logistic_regression: $ln \frac{\Pr(Y_i=1)}{\Pr(Y_i=K)} = \beta_1 \cdot \mathbf{X}_i $ $ln \frac{\Pr(Y_i=2)}{\Pr(Y_i=K)} = ...
2
votes
0answers
92 views

Is it compulsory to make transformation to the econometric model in order to have only diagonal elements on variance-covariance matrix of errors?

I need some sharped and advanced advices for the following issue ... Model and its assumptions I'm working on the methodology of a two-way error component model. Here is the model: $y_{jis} = ...