Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
11 views

Calculating decreased cost with increasing quantity

I have a hand made table I've been using to give customers price per unit on my items, which gives a better price for the more items that they buy. My sample table right now I need to keep the ...
0
votes
1answer
68 views
+50

Program to find closest function to fit arbitrary data

I've wanted this for years, but have never come across anything; a program for Windows to find the closest function to fit arbitrary data. The data I feed it is simple: A table with two columns ...
1
vote
1answer
23 views

Solution of overdetermined polynomial system

Some of you will find this question pretty straightforward to answer, but I desperately need some help in solving a problem involving several equations and 2 unknowns, for an engineering application. ...
1
vote
1answer
15 views

Correlated explanatory variables in linear regression

Is it any reason to assume that if two strongly correlated explanatory variables have impact on response that regression coefficients for these variables have the same signs ? Could such assumption be ...
0
votes
0answers
11 views

Standard deviation errors in log scale

I have a not so common issue with error bars in the log log scale. To be more precise, I have measurements of a quantity Y with an associated standard error Yer that has normal distribution and these ...
1
vote
1answer
18 views

Effects of feature scaling on weight vectors for linear regression

Given that linear regression or polynomial regression can be represented as: $\textbf{w} = (X^{T}X)^{-1}X^{T}Y$ It is standard practice in machine learning to scale each column in their training ...
0
votes
3answers
43 views

Will someone explain this polynomial regression equation?

I am in high school and I need to write a program that does polynomial regression to any degree on a set of data for a personal project. I think that this Wikipedia Article has the equation that I ...
1
vote
0answers
54 views

How to reach Moore-Penrose pseudoinverse solution to minimize error function

Edit I'm trying to figure the derivation of the Moore-Penrose pseudoinverse for linear regression. The starting expression is the standard error function. I'm not quite sure how to expand on this ...
0
votes
1answer
22 views

Pros and cons of including controls in a regression?

Assume we have conducted a random experiment for the benefits of a drug. Let $Y_i$ be the outcome of interest , $X_i$ be some control variables (e.g. age, sex etc.) and $$D_i= ...
0
votes
0answers
14 views

How to find the closest integer linear equation to given real linear equation

I am given a set of points in an n-dimensional plane. I want to find the closest (lowest co-variance) integer linear equation that characterizes the points. I find the real linear equation using r^2 ...
0
votes
0answers
30 views

Nearest points / residuals on a total least squares parabola

Consider fitting a parabola $y = a + bx + cx^2$ to 2d data $X_i, Y_i$ with noise in both X and Y, using the the singular value decomposition as in Total_least_squares (TLS): $\qquad X = [ 1\ \ Xdata\ ...
1
vote
0answers
29 views

Notating the components of the $\hat{\beta}$ matrix when $\hat{Y}$ is multidimensional

This is a question on The Elements of Statistical Learning. We have from the linear model $$\hat{Y} = X^{T}\hat{\beta}$$ where $\hat{Y}^{T} = \begin{bmatrix} \hat{Y}_1 & \hat{Y}_2 & \cdots ...
3
votes
1answer
461 views

Fitting a sine function to data

I have a sequence of $n$ points $(x_i,y_i)$, for $i=1,\dots,n$. I would like to find the function, of the form $y=V\sin(x+\phi)$, which best fits the points. Which numerical method could I use? I have ...
0
votes
1answer
13 views

Invertibility of $X^TX$ when sever multicollinearity in regression

I am studying about multicollinearity in regression and in the book it says, "if there is severe (but not perfect) multicollinearity, two or more predictor variables are highly correlated, so $X^TX$ ...
0
votes
1answer
16 views

How to derive this solution to this minimization problem in vector form?

We want to minimize the mean squared error $$ \sum_{t=1}^n (y_t - \theta^T x_t - \theta_0)^2. $$ Letting $X = [x_t, 1]$, we can rewrite the above problem in vector form as $$ \sum_{t=1}^n (y_t - ...
0
votes
0answers
14 views

Least squares: Calculus to find residual minimizers?

Reading a section on simple regression in "An Introduction to Statistical Learning with Applications in R" I got a question on residual sum of squares minimization. Quoting from the book: [quote] ... ...
2
votes
1answer
28 views

Machine Learning: Linear Regression models

I'm currently in a course learning about neural networks and machine learning, and I came across these two formulas in this textbook page on linear regression: 1) $y(x) = a + bx$ and 2) $y(x) = ...
0
votes
1answer
50 views

Example of a real-world situation where multivariate analysis is applicable.

I have searched a lot of site to understand the situation where multivariate analysis is applicable. But not got any easily understandable example. Would you please give me a real-world example where ...
0
votes
0answers
21 views

Mutiple Regression, calculating R-squared

If I have two regressors in multiple regression equation y=b0 + b1*X1 + b2*X2, how can I find R-squared for the model?I need to know the written formula(not in excel) for two independent variables as ...
0
votes
0answers
45 views

Fitting an ellipse such that the ratio of its radii is in a range

I need to fit an ellipse to a group of points. However, I have an issue and I appreciate if anyone can help me. The issue is that I need to have the fitted ellipse such that the ratio of its radii is ...
2
votes
1answer
300 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: $y$=value, $x_1$=material, $x_2$=weight, $x_3$=color $x_1$ and $x_2$ are interacting variables but $x_3$ ...
0
votes
1answer
32 views

Linear regression with constrained weights

I have a set of $n$ linear combinations, each with $m$ parameters and desired value $b$. I want to find the set of weights $w$ which minimizes the total equations distances (e.g. the sum of distances ...
0
votes
0answers
20 views

Application of Multivariate Analysis

The following situation is proven valuable where multivariate analysis can be applied. This example is taken from the book Applied Multivariate Statistical Analysis ...
5
votes
3answers
3k views

Correlation Coefficient and Determination Coefficient

I'm really new to linear regression and am trying to teach myself. In my textbook there's a problem that asks why $R^{2}$ in the regression of $Y$ on $X =$ the sample correlation between X and Y the ...
0
votes
0answers
10 views

Chi sqaured table for degrees of freedom 616?

In order to check heteroskedasticity, we use the White's test. I tried to follow this method below, however, could not find a table with df=2016 and 95,5% confidence. I don't understand how we get ...
0
votes
1answer
22 views

estimate coefficients of $y = \alpha x + \beta y + \gamma z + \epsilon$

I know how to find $m$ and $b$ for $y= mx +b$, which is : $m= \frac{\bar{x}\bar{y}- \bar{xy}}{(\bar{x})^2 - \bar{x^2}}$ and $b= \bar{y} - m\bar{x}$ How can we estimate $\alpha, \beta, \gamma$ and ...
0
votes
0answers
27 views

how can I find outliers in a 2 vector data set

I have a two data set $(X,Y)$ where $X$ represents the angles and $Y$ represent the signals. $X$ is always correct because I increment it by coding $x=x+1$. However, $Y$ could be sometimes wrong ...
15
votes
6answers
16k views

Regression vs Classification

This is more machine learning questions, but perhaps someone will be able to help. I would like to know what is the difference between regression and classification when we try to generate output for ...
2
votes
2answers
30 views

Likelyhood function analysis

I've done some calculations on a large number of data, and created the following graph in excel representing the data: How do I go about analysing this regression in order to find the formula that ...
1
vote
1answer
35 views

How to apply non-linear regression to Logistic (sigmoid) curve

I've been looking at a useful way to represent Doppler shift from a satellite passing over a ground station. I've calculated the Doppler shift frequency values at 1-second interval for the duration of ...
1
vote
2answers
62 views

Maple: How do I type “solve” with an arrow under?

I am trying to learn using Maple 18 (Mac). I have defined a function with a list of X and Y values. f := x->LinReg(X, Y, x) Now I would like to output the unknown "x" value that correlates with ...
3
votes
1answer
50 views

Rates of convergence of an OLS estimator

I have a linear regression model $$ y_t=x_t\beta+e_t,\quad t=1,\ldots,N. $$ Here $x_t$ is non-random and given by $(1,\delta_t t)$ where $\delta_t$ is 1 for odd $t$ and $0$ otherwise. Moreover, ...
2
votes
1answer
34 views
0
votes
1answer
16 views

Develop relation between dependent and independent Using Tobit model

Depenent variable (Y): Range (0 to 10) (Not less than 0 and not more than 10) (range which i collected from field survey) Independent Variables: X1 - Time (in sec) X2 - Distance (in meter) X3 - ...
1
vote
2answers
59 views

Using inverse of matrix A as approximate inverse of matrix that is very close to A

Say we have two matrices, $A$ and $A'$ so that $A\approx A'$, and we have the inverse of $A$, $B$, where $AB=I$, and the inverse of $A'$ where $A'B'=I$. If we have some guarantee about how big any ...
2
votes
1answer
32 views

Analytic solution for matrix factorization using alternating least squares

The standard form for ridge regression aims to minimize the following cost function. $$ \min\ \ \sum_i(y_i-x_i^T\beta)^2 + \lambda\sum_j\beta^2_j $$ As described here, it's possible to differentiate ...
0
votes
0answers
49 views

standard deviation and adjusted R-squared for simultaneous regressions

I am conducting a study that requires two steps of statistical estimation. First, I run a regular OLS regression, from which I gather three outputs that I need: coefficient values standard ...
1
vote
1answer
69 views

MATLAB curve fitting - least squares method - wrong “fit” using high degrees

Anyone here that could help me with the following problem? The following code calculates the best polynomial fit to a given data-set, that is; a polynomial of a specified degree. Unfortunately, ...
0
votes
0answers
13 views

Multivariate regression with nonindependent variables

I'm trying to run a multivariate regression in which not all variables are independent, and an not sure if this is possible. The reason is as follows: Let's say we have a large number of contracts, ...
0
votes
1answer
40 views

adjusted R squared with multiple dependent varialbles

A question about regression in statistics. What is the formula for adjusted R squared if there are multiple dependent variables
0
votes
1answer
32 views

Recalculate R^2 deleting 1 point

Is there a way to recalculate $R^2$ of a regression that I delete a point (for example an outlier point)? The idea is to get the $R^2$ without a point but without recalculating all the regression. ...
0
votes
1answer
28 views

How to proof that least square estimator $\hat{B}$ doesnt exist when $x$ is linearly dependent?

For the linear regression model $Y=xB+e$, prove that if the columns of $X$ are linearly dependent, the least square estimator $\hat{B}$ does not exist I know that since $\hat{B}$ is an unbiased ...
0
votes
0answers
19 views

questions about distribution of multivariate normal

I'm looking at this past exam question, For A) Cbhat~N(CU,C(summation)C') B)I have very faint idea of what to do, I tried finding some theroems about distribution but couldn't find any that ...
1
vote
1answer
31 views

Question on regression

So I've been given this formula For regression $R^2=1 - \sum \frac{{(y_i - \hat{y}_i)}^2}{(y_1-\bar{y})^2}$ Now an obvious question that has come to me is why $R^2$ stays the same in certain ...
0
votes
1answer
31 views

Linear regression involving angles in a triangle.

In a survey experiment, three independent measurements $29.5^{\circ}$, $30.5^{\circ}$, $120.5^{\circ}$ are obtained from the three angles $\alpha,\beta,\gamma$ of a triangle. Formulate the ...
0
votes
0answers
18 views

Trying to find similarity between collection of points

This is a kind of weird problem, and I'm not sure what the best Stack Exchange to post this on is, but I assume Mathematics could help the most. I have many sets of points in 3D space (xyz ...
0
votes
1answer
17 views

how binary quantile regression divides the dependent variable into quantiles

I am not very clear with binary quantile regression. As if it was ordinary quantile regression, it would divide the dependent variable's value by its ascending value into quantiles. But I cannot ...
0
votes
2answers
55 views

Parameter optimization using a regression model.

I am working on an optimization problem. I build a regression model to understand the behavior of a system which depends on two variables which are functions of another two variables. My regression ...
0
votes
1answer
29 views

Proof of Correlation Coefficients

Good evening, I have a problem with an exercise: Let $X$ and $Y$ be two real square integrable random variables with var$X>0$, var$Y>0$. The correlation Corr$(X,Y)$ quantifices how far $X$ and ...
9
votes
1answer
5k views

Computational complexity of least square regression operation

In a least square regression algorithm, I have to do the following operations to compute regression coefficients: Matrix multiplication, complexity: $O(C^2N)$ Matrix inversion, complexity: ...