Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

2
votes
1answer
129 views

Find data to perform regression analysis

I'm trying to find some data (two continuous variables that I believe are correlated) online for which I can perform a regression anaylsis, my assignment sheet says: The data may be found anywhere ...
2
votes
1answer
4k views

Equations For Quadratic Regression

Does anyone know the specific equations for the three parameters in a least-squares quadratic regression? I'm looking for something like $\beta_1=,\beta_2=,\beta_3=$ for each of ...
2
votes
1answer
103 views

Linear relationship of a company's profit

Assume a linear relationship for a company that has several shops is not known. Let $Y_i$ be the profit the shop number $i$ makes in the coming year. Let $x_i$ be the size of the shop number ...
2
votes
4answers
89 views

Seeking a function based on its level set

I'm trying to create a function for a research project, but I fear my math knowledge is insufficient to derive it from the attached diagram I've created showing its desired behavior. I'm hoping ...
2
votes
3answers
463 views

How to deal with Linear Regression model with some data aggregated

Lets say I am trying to find a linear regression between Weight and Height of a person. $W=b_0+b_1 H+e$ The data I have gathered from 8 people is like this: ...
2
votes
1answer
4k views

How to calculate hyperbola from data points?

I have 4 data points, from which I want to calculate a hyperbola. It seems that the Excel trendline feature can't do it for me, so how do I find the relationship? The points are: (x,y) (3, 0.008) ...
2
votes
1answer
55 views

Outlier detection with robust multiple regression model

I have a set of features (eg, location, income, budget, education) that I use to predict a continuous variable (say, amount spent per day on the internet). I am interested in detecting outliers. I ...
2
votes
1answer
28 views

Prove a result in multiple linear regression

This arises in multiple linear regression. Given $m, n \in \mathbb{N}$ and matrices $X \in \mathbb{R}^{m \times (n+1)} (m > n + 1), H = X(X'X)^{-1}X' \in \mathbb{R}^{m\times m}, I = I_m$ and $J ...
2
votes
1answer
46 views

Is it possible to have two lines of best fit?

Could you rig a data set to have two lines of equally good (and best) fit? Or is it impossible?
2
votes
1answer
20 views

Three-Perpendicular Theorem for linear regressions

For a random vector $X=(X_1,\ldots,X_p)'$, we define $$ \mathcal{L}(X)=\{b_0+b_1X_1+\cdots+b_pX_p,b_0,\ldots,b_p\in\mathbb{R}\}. $$ The linear regression of the $q$-dimensional random vector ...
2
votes
1answer
385 views

Analytic solution for matrix factorization using alternating least squares

The standard form for ridge regression aims to minimize the following cost function. $$ \min\ \ \sum_i(y_i-x_i^T\beta)^2 + \lambda\sum_j\beta^2_j $$ As described here, it's possible to differentiate ...
2
votes
1answer
44 views

Standard Error in OLS Regression

Assuming I have the following linear regression set-up: $y_i = \alpha + x_i * \beta + \epsilon_i$ for $i = 1,2,..., n$. When I run the regession, I get a $\beta$ and $\alpha$ estimates, along ...
2
votes
1answer
83 views

Prove that $E(\mathbf{u}|\mathbf{X})=\mathbf{0}$ implies $Cov(\mathbf{x},\mathbf{u})=\mathbf{0}$

Let \begin{equation} \mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{u} \end{equation} where $\mathbf{y}=\begin{bmatrix}y_1 \\ \vdots \\ y_n\end{bmatrix}$, $\mathbf{X}=\begin{bmatrix}X_{11} & ...
2
votes
1answer
161 views

Least Squares Regression Matrix for Rational Functions

So first off no, this isn't a homework problem. Second, I'm trying to understand how this works, NOT find a program that will do it for me. Okay so I've known for a while how to use Gaussian-Jordan ...
2
votes
1answer
34 views

Gaussian prior favors values closest to zero?

I am reading an article on Bayesian Logistic Regression, where they're using Logistic Regression, imposing a Gaussian prior (with mean = 0) on its parameters. They state that a Gaussian prior favors ...
2
votes
1answer
237 views

Multicollinearity and SVD

I compute the Singular Value Decomposition of a $n \times n$ matrix. If the matrix is not full rank, and I have 2 collinear columns, I end up with one singular value equal to 0. Is it possible to find ...
2
votes
2answers
114 views

derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
2
votes
2answers
64 views

Find parameters for curve fitting (simple linear regression involved?)

I would like to fit data in g~t scatterplot, where ...
2
votes
2answers
44 views

Merging Linear Regression

If I have built two linear regression models over sets $A$ and $B$, and now want a linear regression over set $A\cup{}B$. Is there a way to reuse what I already have?
2
votes
2answers
38 views

Regression model when under-estimations costs us more than over-estimations

We have a factory and we are planning how many items produce in 2014. During the learning process we minimize the mean squared error. But, under-estimations costs us more than over-estimations. Let's ...
2
votes
2answers
496 views

How to fit a sinusoidal function through 2 points with known slopes?

I can define my sinusoidal function as $y(x) = A\sin(B x+c) + D$ or as $y(x) = A \sin(B x) + C \cos(B x) + D$ Now, I have two points with known slopes that I must fit this sine wave to, thus my ...
2
votes
1answer
58 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
2
votes
1answer
959 views

Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t $$ Regression with ...
2
votes
2answers
1k views

Linear Regression: Expectation Proof

I found the following proof in my notes: $E(Y_i) = E[\beta_0 + \beta X_i + \varepsilon_i] =\cdots= \beta_0 + \beta X_i$. This does not seem right to me, however. Why would $E(\beta_1 X_i) = \beta_1 ...
2
votes
1answer
190 views

Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets

I have five sets of observations of measured y as some function of measured $x_1, x_2, x_3,\ldots$ and I want to fit five functions to these observations. They have the form $$ y = f(x_1, x_2, ...
2
votes
1answer
384 views

Finding uncertainty in the slope/intercept for a non-linear least squares fit

I have the following function: $$M = a(\log_{10}W-2.5)+b$$ I also have a set of data with actual measured values of $W$ and $M$ (each have individual $\pm$ errors). Here's a small sampling of the ...
2
votes
1answer
128 views

Least Squares Regression To Half of a Parabola

I have a set of points in two dimensional space, and I know a priori that they approximate half of a parabola. I want to find the coefficients for a quadratic function where all of the points fall on ...
2
votes
2answers
256 views

maximize log determinant subject to a linear constraint

Does anyone know any efficient method to solve the following problem? $ (\alpha,\beta) = \text{argmax} \log \det (\alpha K_1 + \beta K_2)$ s.t. $c_1 \alpha + c_2 \beta = c_3, \alpha\geq0, \beta\geq ...
2
votes
1answer
63 views

Possibility of Unboundedness in Least Squares Minimization

Suppose we have the quadratic minimization problem \begin{equation} \min_x \frac{1}{2} x^TPx + q^Tx +r \end{equation} We know that when $P$ is symmetric positive semi-definite, but the optimality ...
2
votes
1answer
100 views

What am I reinventing? RE: Linear regression modeling for frequency of discrete events

I'm looking to model the frequency of events to quantify how much that frequency is increasing or decreasing. For the sake of concreteness think of the events as web page hits for several low traffic ...
2
votes
1answer
67 views

Biased linear regression

I have a set $S$ of coordinates $(x,y)$, and am estimating $f(x) = ax + b$ where $a > 0$. I also happen to know that $\forall x,y((x,y) \in S \implies y < f(x))$. The question is how I can ...
2
votes
1answer
203 views

How to find a parametric equation?

I want to find an equation for a race track, so I could get the position of a point with respect to time. Let's say I have this track and here are a few points on it: Could it be possible to model ...
2
votes
2answers
605 views

Binary Logistic Regression Model Processing

Thanks for showing interest and wanting to help out. My aim is to develop a model that - as accurately as possible - predicts how entities in a population will either cooperate or defect, as a % of ...
2
votes
1answer
98 views

Nonlinear regression with correlated errors

it's my first post here and I'm a newbie in statistics, so please forgive me if I'm doing something wrong or explaining myself badly. Anyway, I have a problem similar to this: How to perform ...
2
votes
1answer
431 views

variance of multiple regression coefficients

If I consider universal kriging (or multiple spatial regression) in matrix form as: ${\bf{V = XA + R }}$ where $\bf{R}$ is the residual and $\bf{A}$ are the trend coefficients, then the estimate of ...
2
votes
1answer
66 views

Probabilistic regression on outliers

I have a given data set $D = \{ x_i, y_i \}_{i=1}^n$ for a regression problem. When I plot the data, it looks like there is an underlying parabola (2nd order linear model) and some outliers. I want ...
2
votes
1answer
328 views

An intuitive explanation for neural networks as function approximators ?

We use normal linear regression for modelling functions on datasets . But Can someone explain how neural networks help in approximating more complex ,especially non-linear functions ? intuitively , ...
2
votes
1answer
432 views

Curve fitting with upper and lower bounds for derivatives

I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
2
votes
2answers
212 views

How to fit an equation to a curve with disturbances

For example, I have the following data: Y = 366 measured values X = 366 measured values t = [ 1 : 366 ], representing the days of the year (index) So at each t (day), we have value of Y and ...
2
votes
2answers
141 views

Choosing set of best estimators for linear least squares

I have a measured experimental dataset which is well approximated by the sum of several basis functions in linear combinations. Linear least squares of course gives me the optimal weight for each ...
2
votes
2answers
764 views

Fitting data to a portion of an ellipse or conic section

Is there a straightforward algorithm for fitting data to an ellipse or other conic section? The data generally only approximately fits a portion of the ellipse. I am looking for something that doesn't ...
2
votes
0answers
19 views

Can I adjust linear growth of a a subpopulation to a linear decay of the general population?

I need to estimate the amount of CF patients in Poland in the next four years. I have: estimations of the Polish population for the future years a CF patients' register for the last couple of years ...
2
votes
0answers
27 views

Show $\hat{\beta}$ and $s^2$ are independent?

I have the model: $y=X{\beta}+{\epsilon}$ I know $\hat{\beta}=(X'X)^{-1}X'y$ and that it is an unbiased estimator of ${\beta}$ and that $s^2=\hat{\epsilon}'\hat{\epsilon}/(n-k)$ and is an unbiased ...
2
votes
1answer
13 views

Problem with verifying variance of residual

I am supposed to show the following: $$Var(e_{ij}) = \sigma^{2}\left(1-\frac{1}{n_i}\right)$$ Where the follwing is known: $$y_{ij} = \mu + \alpha_{i} + \varepsilon_{ij}$$ $$e_{ij} = y_{ij} - ...
2
votes
0answers
21 views

Proof – OLS estimator regression [closed]

I am having trouble figuring out how I need to form and present an answer to a question. I completely understand the concepts of the math and analysis, I just don't understand how to give an answer ...
2
votes
0answers
85 views

How to reach Moore-Penrose pseudoinverse solution to minimize error function

Edit I'm trying to figure the derivation of the Moore-Penrose pseudoinverse for linear regression. The starting expression is the standard error function. I'm not quite sure how to expand on this ...
2
votes
1answer
65 views

Machine Learning: Linear Regression models

I'm currently in a course learning about neural networks and machine learning, and I came across these two formulas in this textbook page on linear regression: 1) $y(x) = a + bx$ and 2) $y(x) = ...
2
votes
2answers
39 views

Likelyhood function analysis

I've done some calculations on a large number of data, and created the following graph in excel representing the data: How do I go about analysing this regression in order to find the formula that ...
2
votes
1answer
38 views

Predicting the increase/decrease of number

I have these entries in my database that looks like this: ...
2
votes
0answers
58 views

Predicting profit with price variation

I am currently working on a high school project that aims to predict profit from X amount of items to Y amount of profit based off a deviated sale price. For instance: I sale 10 cookies for 10 ...