Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
22 views

Deriving estimators for the parameters a and b that minimize the random error - setting up linear regression variables?

I'm reviewing old notes, and I know I solved this way back when, but can't remember how to know: Consider the simple linear regression model: $$Y_i = a + bX_i + \epsilon_i$$ where $Y_i$ is the ...
1
vote
0answers
44 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
0
votes
2answers
41 views

Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
1
vote
1answer
25 views

Finding best predictors of a classification function

I have a large dataset where each element has a number of "input" categories that are either present or not (or if you like, true or false, 1 or 0 etc). Each one also has an output category, again a ...
0
votes
0answers
47 views

Quantile as solution to minimization problem

I'm studying basics of quantile regression now and I have trouble prooving that $\tau-$th quantile of real-valued random variable $Y$ is a solution to the following minimization problem (in the ...
0
votes
0answers
28 views

How to calculate the coordinate of a point which depends on other points on a plane with specific distances

I have $8$ points on a plane $(x_1,y_1)....(x_8,y_8)$ among these $8$ points I know the coordinates for $7$ points and I have to find the $8^{th}$ point. Each points has the difference between all ...
0
votes
0answers
10 views

Ordered logistic regression with likert scales

I'm currently have a bit of difficulty determining how to analyze this data via logistic regression analysis. Q18 = DV (satisfaction score ranging from 1-10) Q10_1 = IV (Customer Service likert ...
0
votes
1answer
20 views

Regression when the variance of the residuals depends on the independent variable

When the residuals follow a normal distribution, the most likely function that fits the data is found using least squares. In that case: $y = f(x_i) + r_i, \quad r\sim\mathcal{N}(0, \sigma^2)$ ...
0
votes
1answer
19 views

Contribution of each variable in multiple linear regression

What will be the best measure of the contribution of a variable in multiple linear regression? I was thinking of using the coefficient ratio as a marker of a variable's contribution. For example: ...
4
votes
2answers
85 views

Rewriting the matrix equation $AX = YB$ as $Y = CX$?

Is it possible in general, if $A,B,C,X,Y$ are square and of the same dimensions? If so, does it generalize to non-square matrices (using a pseudoinverse)? I'm doing some curve fitting in which I have ...
1
vote
0answers
13 views

Finding posterior of normal distributions and logistic regression.

$P(w_0 | x) = \frac{1}{1 + e^{-log\frac{P(x|w_0)}{P(x|w_1)}-log\frac{P(w_0)}{P(w_1)}}}$ Note: x = $[x_1, \dots, x_d]^T$; a $d$ dimensional vector. $w$ can take on one of two values: $w_0$ or $w_1$. ...
0
votes
0answers
15 views

Spatially model

Can someone explain to me what a 'spatially lagged autoregressive model' is ? I came across this 'model' by searching new techniques for modeling geographical data.
0
votes
1answer
32 views

Find optimal least square solution to the normal equation

What is the optimal solution for $\beta_1$ and $\beta_2$ in the following normal equation: $$\beta _{ 1 }\sum _{ i=1 }^{ n }{ { x }_{ i } } +\beta _{ 0 }=\sum _{ i=1 }^{ n }{ { y }_{ i } } $$ EDIT ...
1
vote
1answer
114 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
3
votes
0answers
35 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
1
vote
0answers
35 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
0
votes
0answers
15 views

Derivative and linear fitting model

Let $V=v_1,v_2,\ldots,v_n$ be the measured velocities and $A=a_1,a_2,\ldots,a_n$ be the measured accelerations of a vehicle at times $T=t_1,t_2,\ldots,t_n$. Let $Y=c_1+c_2t+c_3t^2$ be the best ...
1
vote
1answer
32 views

Model selection in regression: Estimated parameters seem to be “non-significant”

I have conducted an experiment which manipulated three factors (Factor 1: 3 levels, Factor 2: 2 levels, Factor 3: 2 levels). The response variable is binomially distributed (1 = correct or 0 = not ...
1
vote
1answer
51 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
1
vote
1answer
19 views

Dummy recoding for more than two categorical variables

Say I am doing a study with 3 different types of fruit and I want to make a regression depending on the type that tries to predict the amount sold. I know that I could make 2 dummy variables: orange ...
1
vote
1answer
62 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
0
votes
1answer
35 views

Linear regression as $\dim(\beta) \rightarrow \infty$

Consider the linear regression, $$ Y_i = X_i\beta + U_i \qquad E[X_i'U_i]=0 $$ where $X_i=(1,W_{i},W_{i}^2,..\ldots,W_i^K)$ and $\beta \in \mathbb{R}^{K+1}$. The joint distribution of $(X_i,Y_i)$ is ...
0
votes
1answer
24 views

How to create a model with high multicollinearity

I am going to create a model strictly for predictive purposes. Some of my independent variables are highly correlated. When I try to create the regression with all of the variables together, then, I ...
0
votes
1answer
16 views

Can I make one of my predictors categorical and continuous?

I would like to add age as a predictor for my regression, but I would also like to make it a binary categorical predictor with a cutoff of 18 years of age. I would like to do this because I suspect ...
0
votes
0answers
27 views

Metric for movement in 2D space

I have a set of points that represent the coordinates of an object moving in the 2D space at different points in time. Using this points I want to get a "measurement" that will describe the general ...
0
votes
1answer
63 views

Derivative of an exponentially weighted moving average

It has been a while since my university math courses, so let me apologize right off the bat... I'm using GSL to perform non-linear regression analysis and am mostly happy with the outcome, however, ...
0
votes
1answer
39 views

Convert odds ratio based on unit change to several unit changes

Imagine to have two groups of people, the first one more strongly exposed to a pollutant than the second one, and the first one developing a certain disease more often. Having measurements of the ...
2
votes
1answer
30 views

Gaussian prior favors values closest to zero?

I am reading an article on Bayesian Logistic Regression, where they're using Logistic Regression, imposing a Gaussian prior (with mean = 0) on its parameters. They state that a Gaussian prior favors ...
0
votes
0answers
22 views

evaluation of polynomial regression

I have a data set $(x_i$ $y_i)$ if=1...20. I have to fit the data using polynomial feature. How can I evaluate what the complexity of model should be chosen? There is a hint in the task using RMSE ...
4
votes
1answer
35 views

Using regression results to predict?

I run some Poisson regressions with the following results: (with number of associations an individual belongs to as the dependent variable) ...
0
votes
1answer
21 views

Derivation of SSE gradient for Linear Regression

In my text book it gives the derviation of the gradient for the likelihood of a linear regression model (to minimize the negative log likelihood by minimizing the Sum Squared Error). The first line ...
5
votes
2answers
70 views

Update a regression on the fly?

Say I have 100 people each with a height, weight, and age. I make a regression that predicts age based on height and weight. Now, I would like to update that model when I meet someone new. I don't ...
0
votes
0answers
46 views

R squared (Proportion of variance explained) in terms of conditional variance?

My question concerns a comparison between 2 models in terms of proportion of variance explained. Let $y_{t+1}$ denotes the variable I want to explain or predict and $\mathcal{F}_t$ the information ...
0
votes
0answers
37 views

Unconditional sampling distribution of regression coefficients

I am trying to find the (unconditional) sampling distribution of a regression coefficient in a simple linear regression. The linear regression $Y = \beta_0 + X\beta_1 + \epsilon$ is conducted for $N$ ...
2
votes
1answer
45 views

How to perform a monotonic function fitting of data points?

I'm seeking suggestions for general purpose function fitting of a set of data points, where, based on physical intuition, the relationship is expected to be "monotonic", i.e. the function should be ...
0
votes
2answers
36 views

Testing if $X_1$ has an influence of $Y$

Consider you have the suspicion that $Y$ is influenced by two attributes $X_1$ and $X_2$: $$ Y=\theta_0+\theta_1X_1+\theta_2X_2+\theta_3 X_1X_2+U $$ The following data are given. Test ...
0
votes
0answers
24 views

Multivariate Multiple Regression with Repeated Measures

I have a dataset with p predictors for i items (so multiple regression). For each of s subjects, I have r repeated observations of v dependent variables (so it's a multivariate problem). I wish to ...
0
votes
0answers
15 views

Do I have classical measurement error?

this is an econometrics question, so I hope I'm in the right place. Consider the following OLS regression: Y = alpha + beta X + M + epsilon I'm interested in the effect of X on Y. For some ...
0
votes
1answer
265 views

When residual standard error is equal to standard deviation of dependent variable in linear regression?

I wonder when residual standard error is equal to standard deviation of dependent variable in linear regression? Could someone provide some information on this topic and explanation?
0
votes
1answer
50 views

Guess a function that fits empirical data

This is my empirical data: Which function it looks like? I tried to guess (1) a dumped (exponential decaying) sinusoidal, but it does not oscillate after overshoot; (2) a sigmoid, but it oscillate ...
0
votes
1answer
138 views

Is the sum of predicted y values equal to the sum of actual y values?

Say I have a set of points Y and I want to accuratly predict the values of Y by using three variables X1,X2,X3. Hence my equation is Y=intercept + C1*X1 + C2*X2 + C3*X3 After performing linear ...
1
vote
2answers
49 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
1
vote
2answers
48 views

Autocorrelation and var-cov matrix

$$Y_t=\beta_1+\beta_2 X_{t2}+\dots +\beta_k X_{tk}+\epsilon_t \qquad (t=1,\dots,T)$$ $$\epsilon_t=\rho \epsilon_{t-1}+v_t, \qquad v_t \sim \mathrm{i.i.d.}(0,\sigma^2_v)$$ GLS estimation under AR(1) ...
1
vote
0answers
27 views

Properties of best linear predictor?

Conside two scalar random variables, $Y,X$. The best linear predictor of $Y\mid X$ under square loss function is $\theta_0=\operatorname{argmin}_{\theta} ...
2
votes
1answer
50 views

Trigonometric regression

What methods are performed for regression with trigonometric functions? E.g. : Sequence: $$-1, 0, 1, -1, 0, 1, \text{.....}$$ Regression: ...
3
votes
2answers
68 views

Why Linear Regression

First i will like to say that i am not a statistician nor am i good in the field. I have been collecting data for over a period of e.g 100 days and each day has a varying amount of data that i can ...
0
votes
2answers
50 views

Strong vs weak relationship in this correlation

I produced this plot and regression line in R and I thought my results were quite odd. Is the relationship of the correlation determined by how steep the regression line is? So in this case it isn't ...
0
votes
1answer
42 views

Logistic Regression derivation

From the Wikipedia article http://en.wikipedia.org/wiki/Multinomial_logistic_regression: $ln \frac{\Pr(Y_i=1)}{\Pr(Y_i=K)} = \beta_1 \cdot \mathbf{X}_i $ $ln \frac{\Pr(Y_i=2)}{\Pr(Y_i=K)} = ...
2
votes
0answers
98 views

Is it compulsory to make transformation to the econometric model in order to have only diagonal elements on variance-covariance matrix of errors?

I need some sharped and advanced advices for the following issue ... Model and its assumptions I'm working on the methodology of a two-way error component model. Here is the model: $y_{jis} = ...
1
vote
0answers
92 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...