Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
21 views

Expected in-sample error of linear regression with respect to a dataset D

In my textbook, there is a statement mentioned on the topic of linear regression/machine learning, and a question, which is simply quoted as, Consider a noisy target, $ y = (w^{*})^T \textbf{x} + ...
0
votes
1answer
21 views

Expected error of best possible linear fit?

I asked the following question on stat SE, but I could not get a mathematically rigorous answer, and I have decided to ask here again. In my textbook, there is a statement mentioned on the topic of ...
0
votes
1answer
22 views

Is it a wrong expression for the local log-likelihood of logistic regression?

In page 206 of the book 'Elements of statistical learning', the author wrote: The local log-likelihood for this $J$ class model can be written $\sum_{i=1}^NK_\lambda (x_0, ...
1
vote
0answers
24 views

How does Linear Regression classification work?

I am currently trying to understand the following: Logistic regression is a probabilistic, linear classifier. It is parametrized by a weight matrix $W$ and a bias vector $b$. Classification is ...
1
vote
1answer
26 views

Is the Inverse of the Vectorised Solid Angle Equation for $n$ Circular Discs Continuous?

I have a continuous function$^{*1}$ that takes in 3 arguments, and returns 24 outputs. I want to know if the inverse of this function is continuous. The 3 input arguments are the x, y, and z position ...
1
vote
1answer
61 views

Extremely poor polyfit, what am I doing wrong?

I have a dataset with me: http://pastebin.com/YZArky1j, which I am trying to polyfit. This is what I used to perform the polyfit: ...
0
votes
0answers
11 views

Testing linear restriction in regression by rewriting model

For the model \begin{equation*} y=\beta_0 + \beta_1 x_1+\beta_2 x_2+\beta_3x_3+\beta_4x_4 +u \end{equation*} How can I test $H_0:\beta_1+\beta_2+\beta_3=1$ against the alternative $H_1:$ not $H_0$ by ...
0
votes
0answers
9 views

Find sample size underlying these regression results.

I have been trying to work out the answer to this question and have been having no luck, so hopefully you can help. The questions asks you to find sample size from two regression results as below: ...
1
vote
1answer
35 views

Does more data give you a better forecast?

Say I have a large set of data. Each data point corresponds to a particular day in the year, so for 1 year I will have 365 points. Say I have collected this sort of data for 5 years. Now, I want to ...
1
vote
1answer
14 views

Linear regression with normalized variables

Suppose I have two variables X and Y such that mean(X) = 0 = mean(Y) and sd(X) = 1 = sd(Y). The slope of the linear regression line for Y vs X is cov(X,Y)/var(X) = corr(X,Y) since X and Y are ...
1
vote
2answers
61 views

Plotting an Ellipse after an Ellipse Fit

I wonder if someone can assist my understanding as I'm a bit stumped with this... I have taken the following (x,y) data which lies roughly on an ellipse: $$ \begin{pmatrix} 0.000234491 & 6855810 ...
0
votes
0answers
37 views

How to take partial derivative of summation?

This is about L$^1$ iteratively reweighted least squares. Where did the (-a$_i$$_k$) come from?
0
votes
1answer
44 views

Distribution of log-log linear regression

Edit: Sorry yeah not too clear, probably posted this too late at night... Essentially I have data which appears to be in exponential form - a log-log graph put it close to a straight line. Using R, I ...
0
votes
0answers
17 views

stats project - good model, what to do with it?

I've recently been working on a stats project for school. I have been comparing a country's 'quality of life index' with 'moral' opinions survey to see if there are relations. Here's some example ...
0
votes
0answers
8 views

Measure for the quality of the slope of a linear regression?

I calculated a linear regression for a dataset using R. Now I'd like to have a measure of quality for the slope. Which ones are usually used and what's their meaning?
0
votes
0answers
42 views

Help larry water his tomato plants with math

I have a bit of a real world problem that I believe Math can help me solve. I think it might be easiest to phrase in a manor similar to that of high school textbook. Larry has a device that can ...
0
votes
0answers
8 views

Reducing regression equations into one

Suppose we are interested in English scores ($E_{ij}$) and Math scores ($M_{ij}$) in students in various classes. The scores are in different scales from each other. We perform two linear regressions. ...
0
votes
1answer
14 views

Variance of Estimated Coefficients in Logistic Regression

I have a logistic regression model with a binary variable as the response and a categorical variable with 3 categories as a predictor. The fitted model is: logit(P(Y=1)) = intercept -0.19*C2 + ...
1
vote
1answer
32 views

Calculating R-squared with duplicate data

I have the following question regarding the proper usage of R-squared value. Say I have an equation, that predicts energy consumption for the month of a building. One of the input variables accounts ...
3
votes
0answers
44 views

Determine whether ARMA(p,q) is stationary and/or invertible?

Determine whether an ARMA(p,q) process is stationary and invertible such that $y_t = \sum_{i=1}^{p} \phi_i y_{t-i} + \sum_{i=1}^{p} \theta_{i} \epsilon_{t-i}$ with the restriction that ...
0
votes
0answers
11 views

Measure for functional distance?

Let's say there are input points $X$ and two functions $f$ and $f'$ which have results $Y$ and $Y'$ when applied to $X$. There is no noise in this case. Are there any distance measures that exist to ...
2
votes
1answer
40 views

How can I plot this?

Given a bunch of data $x_i$ , $y_i$, how do I plot $$f(\theta_2,\theta_2)= \frac{1}{2M} \sum_{i=1}^{M} (\theta_1\cdot x_i -\theta_2 y_i)^2$$ in matlab? I know it should be parabolic, but my code ...
1
vote
1answer
11 views

Meaning of $\mathcal{I}_t$ in assumption $E[u_t\mid\mathcal{I}_t]$ of distributed-lag model

When considering \begin{equation*} y_t = \beta_0 + \beta_1 x_t + \ldots + \beta_r x_{t-r} + u_t \end{equation*} an assumption made is \begin{equation*} E[u_t\mid\mathcal{I}_t] = 0 \end{equation*} ...
1
vote
1answer
36 views

Wolfram Exponential Fit not match as formulas

I am using WolframAlpha Exponential-Fit formulas to find equation of Exponential Regression http://mathworld.wolfram.com/LeastSquaresFittingExponential.html but after implementation, I tested with a ...
0
votes
0answers
17 views

multicollinearity with intervals

You have multicollinearity when you have 2 variables (X1,X2) that have a relationship, X1=a+X2 where a is constant. My question is: is there still a multicollinearity issue if a is not constant, ...
0
votes
0answers
30 views

Multiple Linear Regression sample problem:

I am currently studying linear regression, but I am not sure I understand everything correctly. I was trying to solve some of the exercises at the end of my book, and I picked a random one below. I am ...
1
vote
1answer
38 views
0
votes
0answers
13 views

Simple Linear Regression variance question

I was wondering if someone could help me clarify the following statement from my lecture notes: The within-sample variance of $Y$ is \begin{equation*} Var(Y) = Var(\hat{\alpha} +\hat{\beta} X + ...
1
vote
1answer
66 views

Intuition and the math behind normalization

What exactly is the purpose of normalization. From what I read, it is to adjust two different sets of values so you can compare them, but I don't understand why, nor the math behind it. Could anyone ...
1
vote
1answer
24 views

Simple linear regression prove variables are uncorrelated:

I am working on the following problem: In a problem of simple linear regression, $$Y = \hat\beta_0 + \hat\beta_1 x(bar),$$ show that the random variables $\hat\beta_1$ and $Y$ are un-correlated (All ...
0
votes
1answer
62 views

Convexity of LASSO

I would like to know if some variables in design matrix are correlated then LASSO is convex or not. If you give me a proof for convexity of LASSO and ADAPTIVE lasso, I will be thankful. LASSO is ...
0
votes
0answers
25 views

Multiple regression - interpretation of coefficients

Assume that one has two input variables (X1, X2) and one output variable (Y). One can approach regression in two ways: One can first run a univariate regression between X1 and X2, have a residual ...
0
votes
0answers
18 views

Variance of regression coefficient

The formula per below gives the regression coefficient under OLS. the textbook that i am using (Elements of statistical learning) subsequently states that the variance of B is as per below (with ...
1
vote
0answers
11 views

Proofs on regression analysis

How can I prove: 1) estimating population variance $\hat\sigma^2={1 \over n-2}[S_{YY}-{S^2_{XY} \over S_{XX}}]$. 2)expected value of error mean square=$E(EMS)=\sigma^2$ To prove (2): I showed that ...
0
votes
2answers
53 views

Is the reference point (x, y) above or below the non-linear equation?

BACKGROUND In short, I have a series of 3 to 10 data points that will be used to represent a curve. For example: $X=0, Y=10$ $X=4, Y=7$ $X=9, Y=12$ $X=16, Y=10$ What I am trying to do is ...
2
votes
0answers
27 views

What is ${\rm cov}(e_i, \hat y_i)$ in simple linear regression?

The model is $y_i = \beta_0 + \beta_1x_i + \epsilon_i$ What is ${\rm cov}(e_i, \hat y_i)$? What is ${\rm cov}(\epsilon_i, \hat \beta_1)$? What is ${\rm cov}(e_i, \epsilon_i)$? For 1, I am writing ...
0
votes
0answers
21 views

Estimating variance in least squares in regression

How to show $\hat\sigma^2=$$1\over n-2$$[S_{yy}-$$S^2_{xy}\over S_{xx}$]. I don't know the matrix form in regression.I don't even understand how to begin this
1
vote
1answer
26 views

properties of least square estimators in regression

$Y_i=\beta_0+\beta_1 X_i+\epsilon_i$ where $\epsilon_i$ is normally distributed with mean $0$ and variance $\sigma^2$ . The least square estimators of this model are $\hat\beta_0$ and $\hat\beta_1$. ...
1
vote
0answers
19 views

Problem on Linear Regression

Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean 0 and variance 1; $y_i = α + β(x_i − \bar x) + e_i , i = ...
1
vote
1answer
8 views

Linear regression or ANOVA with unordered independent variable

I have a set of data, let's say describing a group of people. Let's say we know their income and color of hair: ...
3
votes
1answer
32 views

Logarithmic Functions algebra question

This is my first post and I honestly just want a second opinion on my answer to a question I got incorrect on an exam before I go arguing over it with my professor. Basically, is this mathematically ...
1
vote
1answer
28 views

clustering of singular values

let us consider following graph of singular values i want to make some kind of clustering of these data,namely to seperate main components from non main components,let say signal components ...
0
votes
0answers
38 views

Curve fitting a sinusoidal function

I need to fit the function $y = a\sin(\pi x)$ to these points: $[0,0], [0.5,a], [1,0]$ and another given point $p_0$. Please guide me through tackling this problem, because I don't know what to do. ...
0
votes
3answers
69 views

Reliability of linear regression to predict future

When we have a set of data, where X is the cause, and Y is the effect, we can use linear regression to predict values for Y, based on values of X. I have learned that you may only safely apply this ...
0
votes
1answer
46 views

exponential regression fit error problem

I have the following data and im trying to get an exponential fit. Ive tried a variety of different tools for this, which all seem to give quite a large error margin at the top of the curve. Plotting ...
0
votes
1answer
31 views

reverse a logarithm

I have some data which produces the following logarithmic curve. As you can see, the curve produces the exact opposite of what Im trying to achieve (my data is the line with dots, the logarithm is the ...
0
votes
0answers
61 views

Can't find gradient for MLE for mult-class logistic regression

$$P(k | x_i;w)= \frac{exp(w_k^tx_i)}{\sum_{j=1}^K exp(w_k^tx_i)}$$ $y_i^k$ is a vector that uses 1-of-k encoding. Thus, if $y_i=k$, then the vector $y_i$ has a 1 in the kth spot and a 0 everywhere ...
0
votes
0answers
12 views

Regression Problem Interpretation help

A response Y is a function of three independent variables $x_1,x_2,$ and $x_3$ that are related as follows $Y=\beta_0+\beta_1 x_1+\beta_2 x_2 +\beta_3 x_3 +\epsilon$. a)Fit the model to the $n=7$ ...
0
votes
3answers
26 views

Scatter plot : Are the two observed data related

I have a regression question that ask to draw the scattered plot graph and then conclude if the two data lists are related. The two data lists are years of people and their cholesterol level. I went ...
1
vote
0answers
26 views

Multinomial Logistic Regression

(1) $$P(y^{(i)} =1\mid X,W) = \frac{\exp(W^{(i)^T}X)}{\sum_{j=1}^m \exp(W^{(j)^T}X)}$$ $W$ and $y$ are vectors where the superscript is an index. And there are $m$ classes (that is, there are $m$ ...