0
votes
1answer
29 views

Linear regression as $\dim(\beta) \rightarrow \infty$

Consider the linear regression, $$ Y_i = X_i\beta + U_i \qquad E[X_i'U_i]=0 $$ where $X_i=(1,W_{i},W_{i}^2,..\ldots,W_i^K)$ and $\beta \in \mathbb{R}^{K+1}$. The joint distribution of $(X_i,Y_i)$ is ...
1
vote
0answers
25 views

Properties of best linear predictor?

Conside two scalar random variables, $Y,X$. The best linear predictor of $Y\mid X$ under square loss function is $\theta_0=\operatorname{argmin}_{\theta} ...
0
votes
1answer
26 views

Logistic Regression derivation

From the Wikipedia article http://en.wikipedia.org/wiki/Multinomial_logistic_regression: $ln \frac{\Pr(Y_i=1)}{\Pr(Y_i=K)} = \beta_1 \cdot \mathbf{X}_i $ $ln \frac{\Pr(Y_i=2)}{\Pr(Y_i=K)} = ...
0
votes
0answers
20 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...
0
votes
1answer
21 views

Expected error of best possible linear fit?

I asked the following question on stat SE, but I could not get a mathematically rigorous answer, and I have decided to ask here again. In my textbook, there is a statement mentioned on the topic of ...
0
votes
0answers
60 views

Can't find gradient for MLE for mult-class logistic regression

$$P(k | x_i;w)= \frac{exp(w_k^tx_i)}{\sum_{j=1}^K exp(w_k^tx_i)}$$ $y_i^k$ is a vector that uses 1-of-k encoding. Thus, if $y_i=k$, then the vector $y_i$ has a 1 in the kth spot and a 0 everywhere ...
1
vote
0answers
24 views

Multinomial Logistic Regression

(1) $$P(y^{(i)} =1\mid X,W) = \frac{\exp(W^{(i)^T}X)}{\sum_{j=1}^m \exp(W^{(j)^T}X)}$$ $W$ and $y$ are vectors where the superscript is an index. And there are $m$ classes (that is, there are $m$ ...
1
vote
0answers
27 views

Adjusting regression for small sample bias

I have a set of data points $\{x_i\}$. These data points are grouped so that (say) $i\in\{1,2,3\}$ is group $A$, $i\in\{4,5,6,7\}$ is group $B$, etc. I would like to test the null hypothesis of no ...
0
votes
1answer
38 views

Probability, Linear Models, Expectation

I'm trying to find a way of predicting various models from one "perfect model", using EXCEL. i.e. If I assume that all models should behave like the original one, for which I have all the ...
0
votes
0answers
75 views

What is a good topic to start learning functional linear regression?

I am trying to get a deeper understanding of functional linear regression. I already know the general concepts from here and here and PCA-like estimators for linear regression problems with a ...
1
vote
2answers
1k views

Examples of logistic regression and multinomial logistic regression

I've been studying to understand the concept of logistic regression and I think I understand the idea more or less, but there are still some gaps to fill. What I'm looking for is an example of ...
2
votes
1answer
28 views

Determine which parameter has correlation with result and which is not

sorry for probably silly question, it's the first time when I need to do such work. I have large data set with regarding clicks on some element on web page. It contains some characteristics of such ...
1
vote
0answers
51 views

Conditioning on $X$ equal to premultiplying by $X'$?

I am coming across similar thing in many problems in econometrics and I have not been able to figure out whether it is some general notion or only a "coincidence". To take two examples: Deriving ...
1
vote
0answers
267 views

How to calculate probability with sigmoid output in feedforward neural network?

first of all I'm sorry for my not very skilled English, but I will do my best to explain my problem. I'm trying to create a feedforward neural network with one hidden layer (with probably arctan ...
3
votes
1answer
73 views

Why is $\sum x^2 _t \times \text{Var}(\beta)=\frac{\sum x^2 _t \times \sigma^2}{ \sum x^2 _t} = \sigma^2$?

I do not get this connection. Is is reliable to divide this equation by $\sum x^2 _t$ to get just $\sigma^2$ ? $$\sum x^2 _t \times E(\hat \beta - \beta)^2=\sum x^2 _t \times ...
1
vote
0answers
48 views

Show that $Y$ has constant regression with respect to $X$ and/but that $X$ and $Y$ are not independent.

The joint density of $X$ and $Y$ is given by $$ f(x,y) = \left\{ \begin{array}{l l} 1/2 & \quad \text{$|x|+|y|\leq 1$}\\ 0 & \quad \text{otherwise} \end{array} \right.$$ Show ...
2
votes
1answer
87 views

Linear relationship of a company's profit

Assume a linear relationship for a company that has several shops is not known. Let $Y_i$ be the profit the shop number $i$ makes in the coming year. Let $x_i$ be the size of the shop number ...
0
votes
1answer
2k views

Can someone explain what plim is?

In my Introductory Econometrics class we discussed a concept of "plim" or "probability limit. I'm not sure what this means though and my professor doesn't explain it well at all. Can someone tell me ...
2
votes
1answer
63 views

Probabilistic regression on outliers

I have a given data set $D = \{ x_i, y_i \}_{i=1}^n$ for a regression problem. When I plot the data, it looks like there is an underlying parabola (2nd order linear model) and some outliers. I want ...
3
votes
3answers
501 views

Correlation between variables

I asked this question on stats SE but did not find a suitable answer so far. Maybe someone can help. Given n random variables x1,...,xn (one-dimensional). The following is known (corr() = Pearson ...
3
votes
1answer
134 views

Bound of linear regression's object function

Randomly uniformly select $n$ numbers from a set $\{1,2,...,U\}$ with/without replacement, $y_i$ is the $i$th number selected, and $x_i$ is the rank of $y_i$ in the $n$ numbers. The rank is the order ...