0
votes
0answers
25 views

weights go to infinity in logistic regression with linearly separable data

I have the loss function of logistic regression $L(W)$ = - $\sum_{i=1}^n {y_i}.log[\sigma(w^Tx)] + {(1-y_i)}.log[1- \sigma(w^Tx)]$ I have derived the Hessian and proven it's positive semi-definite ...
0
votes
0answers
16 views

Likelihood Functions of Nonparametric Simple Regression

I'm trying to find the likelihood function of a nonparametric simple regression model. Nonparametric statistics is new to me however, so I'm having some trouble wrapping my head around some of the ...
0
votes
1answer
30 views

More variables = better fit?

When fitting (let's say) a linear regression model, it is always true, that the more variables we include in our model, the better fit is (in R^2 sense)? I don't want to discuss here overfitting, ...
1
vote
1answer
25 views

Prove that the the variance estimator $\widehat{\sigma}^2=MSE/(n-2)$ is biased is the simple linear regression model

This is in scope of the simple linear model. Im trying to prove that $\mathbb{E}\left(\widehat{\sigma}^2\right) = \sigma^2$ for $$\widehat{\sigma}^2 = \frac{1}{n-2}\sum^n_{i=1} ...
0
votes
1answer
42 views

Examining the effect of a quantitative factor on response.

To examine the effect of a quantitative factor temperature on yield,the researcher has a plan to use the following model for the analysis: $$y_{ix}=\beta_0+\beta_1 x+\epsilon_{ix}$$ where $y_{ix}$ ...
0
votes
0answers
21 views

What is distinction between Functional Linear Regression and Functional Linear Models?

Thanks in advance for the help. I want to make sure that I understand two concepts correctly. A functional linear model is a particular type of linear model while functional linear regression is the ...
1
vote
2answers
31 views

Errors and Residual

Why are errors independent but residuals dependent? As far i know the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also ...
0
votes
0answers
15 views

factor models and using cross sectional regression?

I have been doing some reading on factor models. In the literature it mentions that when creating a portfolio that maximises particular attributes it may lead to unwanted bias to other factors. I ...
1
vote
1answer
28 views

time-series regression with missing data

I have a regression as follows for time-series data (e.g. stock prices versus other variables): $$ Y = b \cdot X + b_1 \cdot X_1 + e$$ where $X_1$ will be missing based on pre-determined dates ...
2
votes
0answers
11 views

Showing Hat matrix equal specific values

Consider a one way layout model $y_{ij}$ = $\mu_i + e_{ij}$ (1 $\leq$ i $\leq$ a, 1 $\leq$ j $\leq$ $n_i$) where a = 3 and $n_1$ = 2, $n_2$ = 3, $n_3$ = 4. Show that the hat matrix for this design ...
0
votes
1answer
62 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
0
votes
1answer
37 views

Regression with Mean, Standard Deviation, Range and Correlation

A research team collected data on students in a statistics course. Their dependent variable was the student’s score on the final examination, which ranged from 200 to 800 points. The observed average ...
1
vote
0answers
23 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
1
vote
1answer
24 views

Finding best predictors of a classification function

I have a large dataset where each element has a number of "input" categories that are either present or not (or if you like, true or false, 1 or 0 etc). Each one also has an output category, again a ...
0
votes
0answers
12 views

Spatially model

Can someone explain to me what a 'spatially lagged autoregressive model' is ? I came across this 'model' by searching new techniques for modeling geographical data.
0
votes
0answers
21 views

Estimation of real estate

I'm working on a project on estimating the price value of real estate. First I have collected a lot of data (500 000 instances) with details such as postal code, number of bedrooms, build year, ...
3
votes
0answers
33 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
1
vote
0answers
24 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
1
vote
1answer
22 views

Model selection in regression: Estimated parameters seem to be “non-significant”

I have conducted an experiment which manipulated three factors (Factor 1: 3 levels, Factor 2: 2 levels, Factor 3: 2 levels). The response variable is binomially distributed (1 = correct or 0 = not ...
1
vote
1answer
42 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
1
vote
1answer
15 views

Dummy recoding for more than two categorical variables

Say I am doing a study with 3 different types of fruit and I want to make a regression depending on the type that tries to predict the amount sold. I know that I could make 2 dummy variables: orange ...
1
vote
1answer
39 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
0
votes
1answer
32 views

Linear regression as $\dim(\beta) \rightarrow \infty$

Consider the linear regression, $$ Y_i = X_i\beta + U_i \qquad E[X_i'U_i]=0 $$ where $X_i=(1,W_{i},W_{i}^2,..\ldots,W_i^K)$ and $\beta \in \mathbb{R}^{K+1}$. The joint distribution of $(X_i,Y_i)$ is ...
0
votes
1answer
43 views

Derivative of an exponentially weighted moving average

It has been a while since my university math courses, so let me apologize right off the bat... I'm using GSL to perform non-linear regression analysis and am mostly happy with the outcome, however, ...
5
votes
2answers
66 views

Update a regression on the fly?

Say I have 100 people each with a height, weight, and age. I make a regression that predicts age based on height and weight. Now, I would like to update that model when I meet someone new. I don't ...
0
votes
0answers
27 views

R squared (Proportion of variance explained) in terms of conditional variance?

My question concerns a comparison between 2 models in terms of proportion of variance explained. Let $y_{t+1}$ denotes the variable I want to explain or predict and $\mathcal{F}_t$ the information ...
0
votes
0answers
29 views

Unconditional sampling distribution of regression coefficients

I am trying to find the (unconditional) sampling distribution of a regression coefficient in a simple linear regression. The linear regression $Y = \beta_0 + X\beta_1 + \epsilon$ is conducted for $N$ ...
0
votes
2answers
36 views

Testing if $X_1$ has an influence of $Y$

Consider you have the suspicion that $Y$ is influenced by two attributes $X_1$ and $X_2$: $$ Y=\theta_0+\theta_1X_1+\theta_2X_2+\theta_3 X_1X_2+U $$ The following data are given. Test ...
0
votes
1answer
42 views

When residual standard error is equal to standard deviation of dependent variable in linear regression?

I wonder when residual standard error is equal to standard deviation of dependent variable in linear regression? Could someone provide some information on this topic and explanation?
0
votes
1answer
69 views

Is the sum of predicted y values equal to the sum of actual y values?

Say I have a set of points Y and I want to accuratly predict the values of Y by using three variables X1,X2,X3. Hence my equation is Y=intercept + C1*X1 + C2*X2 + C3*X3 After performing linear ...
1
vote
2answers
37 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
2
votes
1answer
47 views

Trigonometric regression

What methods are performed for regression with trigonometric functions? E.g. : Sequence: $$-1, 0, 1, -1, 0, 1, \text{.....}$$ Regression: ...
3
votes
2answers
63 views

Why Linear Regression

First i will like to say that i am not a statistician nor am i good in the field. I have been collecting data for over a period of e.g 100 days and each day has a varying amount of data that i can ...
0
votes
2answers
31 views

Strong vs weak relationship in this correlation

I produced this plot and regression line in R and I thought my results were quite odd. Is the relationship of the correlation determined by how steep the regression line is? So in this case it isn't ...
0
votes
1answer
29 views

Logistic Regression derivation

From the Wikipedia article http://en.wikipedia.org/wiki/Multinomial_logistic_regression: $ln \frac{\Pr(Y_i=1)}{\Pr(Y_i=K)} = \beta_1 \cdot \mathbf{X}_i $ $ln \frac{\Pr(Y_i=2)}{\Pr(Y_i=K)} = ...
2
votes
0answers
96 views

Is it compulsory to make transformation to the econometric model in order to have only diagonal elements on variance-covariance matrix of errors?

I need some sharped and advanced advices for the following issue ... Model and its assumptions I'm working on the methodology of a two-way error component model. Here is the model: $y_{jis} = ...
0
votes
0answers
29 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...
0
votes
1answer
22 views

Expected error of best possible linear fit?

I asked the following question on stat SE, but I could not get a mathematically rigorous answer, and I have decided to ask here again. In my textbook, there is a statement mentioned on the topic of ...
0
votes
1answer
22 views

Is it a wrong expression for the local log-likelihood of logistic regression?

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
0
votes
0answers
9 views

Find sample size underlying these regression results.

I have been trying to work out the answer to this question and have been having no luck, so hopefully you can help. The questions asks you to find sample size from two regression results as below: ...
1
vote
1answer
38 views

Does more data give you a better forecast?

Say I have a large set of data. Each data point corresponds to a particular day in the year, so for 1 year I will have 365 points. Say I have collected this sort of data for 5 years. Now, I want to ...
0
votes
0answers
17 views

stats project - good model, what to do with it?

I've recently been working on a stats project for school. I have been comparing a country's 'quality of life index' with 'moral' opinions survey to see if there are relations. Here's some example ...
0
votes
0answers
47 views

Help larry water his tomato plants with math

I have a bit of a real world problem that I believe Math can help me solve. I think it might be easiest to phrase in a manor similar to that of high school textbook. Larry has a device that can ...
0
votes
1answer
16 views

Variance of Estimated Coefficients in Logistic Regression

I have a logistic regression model with a binary variable as the response and a categorical variable with 3 categories as a predictor. The fitted model is: logit(P(Y=1)) = intercept -0.19*C2 + ...
1
vote
1answer
32 views

Calculating R-squared with duplicate data

I have the following question regarding the proper usage of R-squared value. Say I have an equation, that predicts energy consumption for the month of a building. One of the input variables accounts ...
3
votes
0answers
59 views

Determine whether ARMA(p,q) is stationary and/or invertible?

Determine whether an ARMA(p,q) process is stationary and invertible such that $y_t = \sum_{i=1}^{p} \phi_i y_{t-i} + \sum_{i=1}^{p} \theta_{i} \epsilon_{t-i}$ with the restriction that ...
0
votes
0answers
17 views

multicollinearity with intervals

You have multicollinearity when you have 2 variables (X1,X2) that have a relationship, X1=a+X2 where a is constant. My question is: is there still a multicollinearity issue if a is not constant, ...
1
vote
1answer
112 views

Intuition and the math behind normalization

What exactly is the purpose of normalization. From what I read, it is to adjust two different sets of values so you can compare them, but I don't understand why, nor the math behind it. Could anyone ...
1
vote
0answers
14 views

Proofs on regression analysis

How can I prove: 1) estimating population variance $\hat\sigma^2={1 \over n-2}[S_{YY}-{S^2_{XY} \over S_{XX}}]$. 2)expected value of error mean square=$E(EMS)=\sigma^2$ To prove (2): I showed that ...
2
votes
0answers
41 views

What is ${\rm cov}(e_i, \hat y_i)$ in simple linear regression?

The model is $y_i = \beta_0 + \beta_1x_i + \epsilon_i$ What is ${\rm cov}(e_i, \hat y_i)$? What is ${\rm cov}(\epsilon_i, \hat \beta_1)$? What is ${\rm cov}(e_i, \epsilon_i)$? For 1, I am writing ...