Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
3answers
36 views

Reliability of linear regression to predict future

When we have a set of data, where X is the cause, and Y is the effect, we can use linear regression to predict values for Y, based on values of X. I have learned that you may only safely apply this ...
0
votes
0answers
15 views

exponential regression fit error problem

I have the following data and im trying to get an exponential fit. Ive tried a variety of different tools for this, which all seem to give quite a large error margin at the top of the curve. Plotting ...
0
votes
1answer
25 views

reverse a logarithm

I have some data which produces the following logarithmic curve. As you can see, the curve produces the exact opposite of what Im trying to achieve (my data is the line with dots, the logarithm is the ...
-2
votes
2answers
39 views

what do you think is the best equation that fits this data points? [on hold]

the file is here. kindly check it out. https://www.dropbox.com/s/ljso0kqyyjf9ku9/HELP.xls
0
votes
1answer
12 views

(Linear regression and data points) Does anyone have links to good tutorials, or advice? [on hold]

I have gathered several sets of 1500 data points. I am trying to import the data and run a simple linear regression model on the points. I have found very vague tutorials to teach me how to do this. I ...
0
votes
0answers
23 views
+50

Can't find gradient for MLE for mult-class logistic regression

$$P(k | x_i;w)= \frac{exp(w_k^tx_i)}{\sum_{j=1}^K exp(w_k^tx_i)}$$ $y_i^k$ is a vector that uses 1-of-k encoding. Thus, if $y_i=k$, then the vector $y_i$ has a 1 in the kth spot and a 0 everywhere ...
0
votes
0answers
4 views

Muliple Logistic regression

How to find the effect of democrat instead of Independent on the estimated odds of support for legalized abortion from logistic regression equation logit[Phat(y = 1)] = 0.11 + 0.16s - 0.57r1- 0.66r2 + ...
0
votes
1answer
10 views

Regression Problem Interpretation help

A response Y is a function of three independent variables $x_1,x_2,$ and $x_3$ that are related as follows $Y=\beta_0+\beta_1 x_1+\beta_2 x_2 +\beta_3 x_3 +\epsilon$. a)Fit the model to the $n=7$ ...
1
vote
0answers
26 views

What statistical tests for headache journal? [migrated]

I track my pain levels in an online spreadsheet along with daily habits and trigger events. I want to test whether changes in my pain over time follow a trend (not concerned whether it is linear or ...
0
votes
3answers
20 views

Scatter plot : Are the two observed data related

I have a regression question that ask to draw the scattered plot graph and then conclude if the two data lists are related. The two data lists are years of people and their cholesterol level. I went ...
1
vote
0answers
11 views

Multinomial Logistic Regression

(1) $$P(y^{(i)} =1\mid X,W) = \frac{\exp(W^{(i)^T}X)}{\sum_{j=1}^m \exp(W^{(j)^T}X)}$$ $W$ and $y$ are vectors where the superscript is an index. And there are $m$ classes (that is, there are $m$ ...
0
votes
1answer
21 views

Testing for a random walk with drift

I was wondering if someone could help me clarify something from my lecture notes. It concerns the last step. I was wondering why we test if $\frac{\hat{\beta}}{\textrm{SE}(\hat{\beta}}<0$ and ...
0
votes
0answers
21 views

Identification of linear regression function under $\ell_1$-norm criterion.

Consider a linear system \begin{align*} y = \theta^Tx + e, \end{align*} where $x\in\mathbb{R}^n$ and $e\in\mathbb{R}$ are independent Gaussian random variables with distribution $\mathcal{N}(0,I_n)$ ...
0
votes
0answers
13 views

how to calculate the covariance between residuals?

suppose y_i_hat = beta_hat * x_i + e_i what is the cov(e_i, e_j)? I derived beta_hat to be sum(x_i * y_i) / sum((x_i)^2), y_j) I substitute e_i = y_i - y_i_hat and used the above formula for ...
0
votes
0answers
19 views

Coefficient of Determination (Alternative Solution)

Consider the following problem - where I have purposefully omitted the numbers in question since it is of no interest for my question. From $40$ observations on \begin{align} ...
0
votes
1answer
22 views

Linear system of equations and multiple linear regression: Numerical solving

I am currently implementing a test procedure for data, namely a linear form of the Kramers-Kronig relations (paper here: http://jes.ecsdl.org/content/142/6/1885.abstract). This includes solving a ...
1
vote
1answer
20 views

Equations for Cubic Regression

So, I'm making a simple program for drawing graphs, and I'm looking at making some simple best-fit curves using some basic regression analysis. I've happily got linear and quadratic regression working ...
-1
votes
0answers
15 views

How to Build a Foresight System? [migrated]

For a research project, I'm asked to find ways to build an economic foresight system. For example, for the production of cheese. We will have data about the market indicators, like price, demand etc. ...
0
votes
1answer
24 views

Regression function - conditional mean

I am trying to understand the statistical fundamentals behind linear regression, and i have never been able to intuitively understand the following; really would appreciate if someone could give an ...
1
vote
0answers
12 views

General 2D taylor surfaces from axial behaviour and discrete points

I have a problem as follows: I have a nonlinear function, f(x,y), for which I (numerically) know the axial behaviours, f(x,y0) and f(x0,y), where x0 and y0 are constants. I can calculate discrete ...
1
vote
2answers
44 views

Is it possible to fit any regression line to a set of data points?

If you have a set of data points (x1,y1), (x2,y2),...,(xn,yn) And you know it fits a trend y=f(x) where ...
0
votes
0answers
10 views

The simple regression model?

Suppose we have a simple regression model: $y=b_0+b_1x+\hat u$, in this case the sum of $\hat u=0$, and for regression through origin: $y=b_1x+\hat u$, proof that the sum of $\hat u$ not equal to ...
0
votes
0answers
11 views

Directional Derivative of a function containing an Indicator function.

I'm trying to understand a passage in Koenker's Quantile regression book (p.33). It says: (note that y,x, are vectors and w is the direction vector) With the first part of the outcome no problem: ...
1
vote
0answers
17 views

Predicting y from a log-linear regression

I was wondering if someone could explain to me the very last step on the right hand slide. Why is do we have a sum rather than a product. Thank you very much.
0
votes
0answers
17 views

The simple and multiple regression model?

Any help on this would be immensely appreciate ! 1. Suppose we have a simple regression model: y=b0+b1x+uhat, in this case the sum of uhat=0, and for regression through origin: y=b1x+uhat, proof that ...
0
votes
0answers
19 views

Deming estimator

Can someone help me prove that Deming estimator $\hat{\beta_{1}}$ is monotone in $\delta$: (See https://en.wikipedia.org/wiki/Deming_regression). When deriving the estimator w.r.t $\delta$, I find ...
0
votes
0answers
12 views

Residuals normally distributed + no heteroscedasticity = data normally distributed?

If my residuals are normally distributed and no heteroscedasticity was found in the data, can it be assumed that data is also normally distributed when using ordinary least squares regression?
0
votes
2answers
11 views

Naive question re. normal equation for linear regression

The typical normal equation for linear regression is $\theta=(X^TX)^{−1} X^T Y$ such that the gradient of $J(\theta)$ is zero. Why does $X^{-1} Y$ not work? What are the numerical reasons for this?
1
vote
0answers
16 views

Logistic regression eye treacting data (need model)

I have two sets of time course data, they are for an eye-tracking study. The data is 20 100ms chunks, one category being percent fixations for canonical sentences, and the other being percent looks ...
1
vote
2answers
53 views

derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
1
vote
0answers
12 views

Understanding the regularization parameter in polynomial regression

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...
2
votes
0answers
29 views

Smallest set of Liner equations, which exactly fit a set of points

I have a set of 2-d points,(it can be of any arbitrary dimension n). I want to find the minimum set of straight lines(linear equations) which exactly passes through the given 2-d points (unlike ...
0
votes
0answers
12 views

Help with finding a time series model

Use everything in your power to tentatively identify a reasonable model for the generated data. I am looking for four (4) things in your report: The model in words, e.g., "and MA(2)", The ...
0
votes
1answer
18 views

Fitting 2nd Order multivariate quadratic with matrices

Hopefully you at least entertain this question as it took forever to construct the below matrix using TeX. Any ways, so I have a list of data points ($X_1$,$X_2$,Y), with the X's being independent ...
0
votes
1answer
29 views

How to interpret coefficients in polynomial regression?

I am working on my thesis (study) about poverty incidence rate and its socio-economic factors using second-order polynomial regression without interaction. The final model in my study is ...
0
votes
0answers
14 views

Asymptotic Mean Square Error for kernel regression estimator

I want to derive the optimal rate of convergence for the kernel-based estimator for $E [Y|X = x]$ based on observations $(X_{1},Y_{1})$,...,$(X_{n},Y_{n})$ (where the $X_{i}$'s are $\mathbb{R}^{d}$ ...
0
votes
0answers
10 views

Some kind of regression on complex numbers

I need to compute (or at least tell something about) an expression like this: $$\sum_{k=1}^n ||\rho e^{i(t_k\omega-\theta)}-\sum_{j=1}^p \rho_j e^{i(t_k\omega_j-\theta_j)}||^2$$ The idea is to find ...
0
votes
0answers
28 views

How do I go from $(X\beta)'Y$ to $Y'(X\beta)$ when solving OLS in matrix form?

When I first learned how to derive the OLS $\beta$ in matrix form, I learned to take the derivative of the summation first, then convert into matrix form. In other words Using calculus, we derive ...
0
votes
2answers
24 views

Prove $\sum_i \frac{\bar{x}(x_i-\bar{x})}{nS_{xx}} =0$

Prove $$\sum_i \frac{\bar{x}(x_i-\bar{x})}{nS_{xx}} =0$$ That is one detail in a proof of the variance of the intercept $\alpha$ in the simple linear regression $Y_i=\alpha+\beta x_i$.
1
vote
2answers
92 views

Prove $SST=SSE+SSR$

Prove $$SST=SSE+SSR$$ I start with $$SST= \Sigma (y_i-\bar{y})^2=...=SSE+SSR+ \Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )$$ and I don't know how to prove that $\Sigma 2( y_i-y_i^*)(y_i^*-\bar{y} )=0$ a ...
1
vote
1answer
41 views

regression on circular data

How would one design a regression where the dependent variable is measured in degrees on a circle? The dependent variable is on the range [0, 360), and the independent variables are demographic ...
0
votes
1answer
16 views

Regression concepts clarified.

I want to understand regression in a clearer way. Suppose I assume the relationship between $Y$ and $X$ is linear. The regression equation I posit is: $Y_{i} = b_{0} +b_{1}X + e_{i}$. This means that ...
1
vote
1answer
33 views

What is the difference between Curve Fitting and Machine Learning?

I know that Machine Learning regression algorithms try to find the function of the data. That is, if we have 1000 data points (x,y), to find a general continuous function that follows the trends of ...
0
votes
0answers
21 views

Parabolic regression with restricted shape

How can I calculate the parabolic regression with vertex at minimum. Is it possible? I have a set of points from which I estimate the parabola using the (I believe) standard equation (from ...
1
vote
0answers
13 views

Regression factors and covariance matrix

I am trying to follow some notes. They have two matrices. One is called comfact (company factors). This is a 580 x 5 matrix. The 580 rows represent 580 different companies. The 5 columns represent 5 ...
0
votes
0answers
22 views

Multivariate Linear Regression for a System of Linear Equations

I have a system of linear equations in the form of $A\vec{x}=\vec{y}$, where $A$ is an $n\times n$ matrix and $\vec{x}$ and $\vec{y}$ are $n\times 1$ matrices. Suppose $\vec{x}$ and $\vec{y}$ ...
1
vote
1answer
31 views

What's the “ridge” in Ridge Regression?

In normal least squares, we try to find $\hat\beta$ which minimizes $$\|y-X\beta\|^2$$ Ridge regression expands this to "penalize" certain values of $\beta$ via a matrix $\Gamma$: ...
0
votes
0answers
18 views

linear regression optimization

If I have this equation to be minimized (it is from linear regression) and the example is: or in its complete form: so it says that one needs to minimize the function J(theta). The parts of ...
0
votes
0answers
10 views

Multinomial Logistic Regression with an Unknown Number of Categories

The problem I'm facing involves determining the number of categories in a data set in a principled way. I feel that logistic regression may be the right tool for this project, but is it possible to do ...
2
votes
0answers
33 views

A* vs D* vs Dijkstra [closed]

I understand the basis of A* as being a derivative of Dijkstra, however, I recently found out about D*. From wikipedia, I can understand the algorithm. What I do not understand is why I would use D* ...