Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
20 views

Find optimal least square solution to the normal equation

What is the optimal solution for $\beta_1$ and $\beta_2$ in the following normal equation: $$\beta _{ 1 }\sum _{ i=1 }^{ n }{ { x }_{ i } } +\beta _{ 0 }=\sum _{ i=1 }^{ n }{ { y }_{ i } } $$ EDIT ...
1
vote
1answer
29 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
3
votes
0answers
19 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
1
vote
0answers
16 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
0
votes
0answers
9 views

Derivative and linear fitting model

Let $V=v_1,v_2,\ldots,v_n$ be the measured velocities and $A=a_1,a_2,\ldots,a_n$ be the measured accelerations of a vehicle at times $T=t_1,t_2,\ldots,t_n$. Let $Y=c_1+c_2t+c_3t^2$ be the best ...
1
vote
1answer
17 views

Model selection in regression: Estimated parameters seem to be “non-significant”

I have conducted an experiment which manipulated three factors (Factor 1: 3 levels, Factor 2: 2 levels, Factor 3: 2 levels). The response variable is binomially distributed (1 = correct or 0 = not ...
0
votes
0answers
17 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
1
vote
1answer
13 views

Dummy recoding for more than two categorical variables

Say I am doing a study with 3 different types of fruit and I want to make a regression depending on the type that tries to predict the amount sold. I know that I could make 2 dummy variables: orange ...
0
votes
1answer
26 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
0
votes
1answer
29 views

Linear regression as $\dim(\beta) \rightarrow \infty$

Consider the linear regression, $$ Y_i = X_i\beta + U_i \qquad E[X_i'U_i]=0 $$ where $X_i=(1,W_{i},W_{i}^2,..\ldots,W_i^K)$ and $\beta \in \mathbb{R}^{K+1}$. The joint distribution of $(X_i,Y_i)$ is ...
-6
votes
0answers
41 views

Best fit line for a set of data. [closed]

I have a whole bunch of data values I would like to calculate the volume (integral or summation) but I am drawing a blank as to how get the equation of my best-fit line first. If you would help me ...
0
votes
1answer
21 views

How to create a model with high multicollinearity

I am going to create a model strictly for predictive purposes. Some of my independent variables are highly correlated. When I try to create the regression with all of the variables together, then, I ...
0
votes
1answer
15 views

Can I make one of my predictors categorical and continuous?

I would like to add age as a predictor for my regression, but I would also like to make it a binary categorical predictor with a cutoff of 18 years of age. I would like to do this because I suspect ...
0
votes
0answers
25 views

Metric for movement in 2D space

I have a set of points that represent the coordinates of an object moving in the 2D space at different points in time. Using this points I want to get a "measurement" that will describe the general ...
0
votes
0answers
22 views

Trying to prove Slope of linear regression without Calculus

Hi so what I'm trying to do is prove that the slope of a simple linear regression line is equal to the $cor(X,Y) * sd(Y)/sd(X)$ without using calculus(I know it can be done easily with calculus but ...
2
votes
0answers
23 views

Recursive Least Square (RLS) algorithm for regression. [closed]

What is meant by convergence of recursive least square (RLS) in online regression ?
0
votes
1answer
35 views

Derivative of an exponentially weighted moving average

It has been a while since my university math courses, so let me apologize right off the bat... I'm using GSL to perform non-linear regression analysis and am mostly happy with the outcome, however, ...
0
votes
0answers
7 views

Convert odds ratio based on unit change to several unit changes

Imagine to have two groups of people, the first one more strongly exposed to a pollutant than the second one, and the first one developing a certain disease more often. Having measurements of the ...
2
votes
1answer
24 views

Gaussian prior favors values closest to zero?

I am reading an article on Bayesian Logistic Regression, where they're using Logistic Regression, imposing a Gaussian prior (with mean = 0) on its parameters. They state that a Gaussian prior favors ...
0
votes
0answers
15 views

evaluation of polynomial regression

I have a data set $(x_i$ $y_i)$ if=1...20. I have to fit the data using polynomial feature. How can I evaluate what the complexity of model should be chosen? There is a hint in the task using RMSE ...
4
votes
1answer
25 views

Using regression results to predict?

I run some Poisson regressions with the following results: (with number of associations an individual belongs to as the dependent variable) ...
-1
votes
0answers
8 views

Bayes Classifier for items with correlation

I am familiar with Bayes classifiers and regression in general, but I am not a mathematician and so I was hoping someone here could answer this. I have a set of objects, for this purpose lets say ...
0
votes
1answer
14 views

Derivation of SSE gradient for Linear Regression

In my text book it gives the derviation of the gradient for the likelihood of a linear regression model (to minimize the negative log likelihood by minimizing the Sum Squared Error). The first line ...
5
votes
2answers
55 views

Update a regression on the fly?

Say I have 100 people each with a height, weight, and age. I make a regression that predicts age based on height and weight. Now, I would like to update that model when I meet someone new. I don't ...
-1
votes
0answers
17 views

What machine learning tool is best suited for taking time series and descriptive and making a binomial classification. [migrated]

I have an interesting task of utilizing log data from computer servers in a server farm and predicting if a particular server is likely to fail in the next 24 hours. My data set will be comprised of ...
0
votes
0answers
19 views

R squared (Proportion of variance explained) in terms of conditional variance?

My question concerns a comparison between 2 models in terms of proportion of variance explained. Let $y_{t+1}$ denotes the variable I want to explain or predict and $\mathcal{F}_t$ the information ...
0
votes
0answers
21 views

Which subset of variables is best in multiple regression

I had a question regarding multiple regression and choosing the best subset. In my textbook it says you can find the best fit by testing the regression of all the variables in subsets (e.g if there ...
0
votes
0answers
23 views

Unconditional sampling distribution of regression coefficients

I am trying to find the (unconditional) sampling distribution of a regression coefficient in a simple linear regression. The linear regression $Y = \beta_0 + X\beta_1 + \epsilon$ is conducted for $N$ ...
1
vote
1answer
15 views

How to perform a monotonic function fitting of data points?

I'm seeking suggestions for general purpose function fitting of a set of data points, where, based on physical intuition, the relationship is expected to be "monotonic", i.e. the function should be ...
0
votes
2answers
35 views

Testing if $X_1$ has an influence of $Y$

Consider you have the suspicion that $Y$ is influenced by two attributes $X_1$ and $X_2$: $$ Y=\theta_0+\theta_1X_1+\theta_2X_2+\theta_3 X_1X_2+U $$ The following data are given. Test ...
0
votes
0answers
11 views

Multivariate Multiple Regression with Repeated Measures

I have a dataset with p predictors for i items (so multiple regression). For each of s subjects, I have r repeated observations of v dependent variables (so it's a multivariate problem). I wish to ...
0
votes
0answers
11 views

Do I have classical measurement error?

this is an econometrics question, so I hope I'm in the right place. Consider the following OLS regression: Y = alpha + beta X + M + epsilon I'm interested in the effect of X on Y. For some ...
0
votes
1answer
20 views

When residual standard error is equal to standard deviation of dependent variable in linear regression?

I wonder when residual standard error is equal to standard deviation of dependent variable in linear regression? Could someone provide some information on this topic and explanation?
0
votes
1answer
40 views

Guess a function that fits empirical data

This is my empirical data: Which function it looks like? I tried to guess (1) a dumped (exponential decaying) sinusoidal, but it does not oscillate after overshoot; (2) a sigmoid, but it oscillate ...
0
votes
1answer
29 views

Is the sum of predicted y values equal to the sum of actual y values?

Say I have a set of points Y and I want to accuratly predict the values of Y by using three variables X1,X2,X3. Hence my equation is Y=intercept + C1*X1 + C2*X2 + C3*X3 After performing linear ...
0
votes
0answers
14 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
1
vote
2answers
36 views

Autocorrelation and var-cov matrix

$$Y_t=\beta_1+\beta_2 X_{t2}+\dots +\beta_k X_{tk}+\epsilon_t \qquad (t=1,\dots,T)$$ $$\epsilon_t=\rho \epsilon_{t-1}+v_t, \qquad v_t \sim \mathrm{i.i.d.}(0,\sigma^2_v)$$ GLS estimation under AR(1) ...
1
vote
0answers
25 views

Properties of best linear predictor?

Conside two scalar random variables, $Y,X$. The best linear predictor of $Y\mid X$ under square loss function is $\theta_0=\operatorname{argmin}_{\theta} ...
2
votes
1answer
45 views

Trigonometric regression

What methods are performed for regression with trigonometric functions? E.g. : Sequence: $$-1, 0, 1, -1, 0, 1, \text{.....}$$ Regression: ...
3
votes
2answers
63 views

Why Linear Regression

First i will like to say that i am not a statistician nor am i good in the field. I have been collecting data for over a period of e.g 100 days and each day has a varying amount of data that i can ...
0
votes
2answers
24 views

Strong vs weak relationship in this correlation

I produced this plot and regression line in R and I thought my results were quite odd. Is the relationship of the correlation determined by how steep the regression line is? So in this case it isn't ...
0
votes
1answer
26 views

Logistic Regression derivation

From the Wikipedia article http://en.wikipedia.org/wiki/Multinomial_logistic_regression: $ln \frac{\Pr(Y_i=1)}{\Pr(Y_i=K)} = \beta_1 \cdot \mathbf{X}_i $ $ln \frac{\Pr(Y_i=2)}{\Pr(Y_i=K)} = ...
2
votes
0answers
92 views

Is it compulsory to make transformation to the econometric model in order to have only diagonal elements on variance-covariance matrix of errors?

I need some sharped and advanced advices for the following issue ... Model and its assumptions I'm working on the methodology of a two-way error component model. Here is the model: $y_{jis} = ...
0
votes
0answers
20 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...
0
votes
1answer
21 views

Expected error of best possible linear fit?

I asked the following question on stat SE, but I could not get a mathematically rigorous answer, and I have decided to ask here again. In my textbook, there is a statement mentioned on the topic of ...
0
votes
1answer
22 views

Is it a wrong expression for the local log-likelihood of logistic regression?

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
1
vote
0answers
22 views

How does Linear Regression classification work?

I am currently trying to understand the following: Logistic regression is a probabilistic, linear classifier. It is parametrized by a weight matrix $W$ and a bias vector $b$. Classification is ...
1
vote
1answer
25 views

Is the Inverse of the Vectorised Solid Angle Equation for $n$ Circular Discs Continuous?

I have a continuous function$^{*1}$ that takes in 3 arguments, and returns 24 outputs. I want to know if the inverse of this function is continuous. The 3 input arguments are the x, y, and z position ...
1
vote
1answer
61 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
0
votes
0answers
11 views

Testing linear restriction in regression by rewriting model

For the model \begin{equation*} y=\beta_0 + \beta_1 x_1+\beta_2 x_2+\beta_3x_3+\beta_4x_4 +u \end{equation*} How can I test $H_0:\beta_1+\beta_2+\beta_3=1$ against the alternative $H_1:$ not $H_0$ by ...