# Tagged Questions

Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

62 views

86 views

### Outlier detection with robust multiple regression model

I have a set of features (eg, location, income, budget, education) that I use to predict a continuous variable (say, amount spent per day on the internet). I am interested in detecting outliers. I ...
41 views

626 views

### Least Squares Regression Matrix for Rational Functions

So first off no, this isn't a homework problem. Second, I'm trying to understand how this works, NOT find a program that will do it for me. Okay so I've known for a while how to use Gaussian-Jordan ...
140 views

### orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
40 views

### Gaussian prior favors values closest to zero?

I am reading an article on Bayesian Logistic Regression, where they're using Logistic Regression, imposing a Gaussian prior (with mean = 0) on its parameters. They state that a Gaussian prior favors ...
45 views

### Is the Inverse of the Vectorised Solid Angle Equation for $n$ Circular Discs Continuous?

I have a continuous function$^{*1}$ that takes in 3 arguments, and returns 24 outputs. I want to know if the inverse of this function is continuous. The 3 input arguments are the x, y, and z position ...
143 views

### derivation of simple linear regression parameters

I know there are some proof in the internet, but I attempted to proove the formulas for the intercept and the slope in simple linear regression using Least squares, some algebra, and partial ...
236 views

### Fitting a simple linear regression

A professor in the School of Business in a university polled a dozen colleagues about the number of professional meetings they attended in the past five years $x$ and the number of papers they ...
79 views

### Find parameters for curve fitting (simple linear regression involved?)

I would like to fit data in g~t scatterplot, where ...
58 views

### Merging Linear Regression

If I have built two linear regression models over sets $A$ and $B$, and now want a linear regression over set $A\cup{}B$. Is there a way to reuse what I already have?
40 views

### Regression model when under-estimations costs us more than over-estimations

We have a factory and we are planning how many items produce in 2014. During the learning process we minimize the mean squared error. But, under-estimations costs us more than over-estimations. Let's ...
745 views

### How to fit a sinusoidal function through 2 points with known slopes?

I can define my sinusoidal function as $y(x) = A\sin(B x+c) + D$ or as $y(x) = A \sin(B x) + C \cos(B x) + D$ Now, I have two points with known slopes that I must fit this sine wave to, thus my ...
114 views

### Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
2k views

### Lasso - constraint form equivalent to penalty form

We know that there are two definitions to describe lasso. Regression with constraint definition: $$\min\limits_{\beta} \|y-X\beta\|^2, \sum\limits_{p}|\beta_p|\leq t, \exists t$$ Regression with ...
I found the following proof in my notes: $E(Y_i) = E[\beta_0 + \beta X_i + \varepsilon_i] =\cdots= \beta_0 + \beta X_i$. This does not seem right to me, however. Why would $E(\beta_1 X_i) = \beta_1 ... 1answer 274 views ### Techniques to find regression parameters for multiple datasets where a subset of parameters should be the same for all datasets I have five sets of observations of measured y as some function of measured$x_1, x_2, x_3,\ldots$and I want to fit five functions to these observations. They have the form $$y = f(x_1, x_2, x_3,\... 1answer 516 views ### Finding uncertainty in the slope/intercept for a non-linear least squares fit I have the following function:$$M = a(\log_{10}W-2.5)+b$$I also have a set of data with actual measured values of W and M (each have individual \pm errors). Here's a small sampling of the ... 2answers 331 views ### maximize log determinant subject to a linear constraint Does anyone know any efficient method to solve the following problem? (\alpha,\beta) = \text{argmax} \log \det (\alpha K_1 + \beta K_2) s.t. c_1 \alpha + c_2 \beta = c_3, \alpha\geq0, \beta\geq ... 1answer 65 views ### Possibility of Unboundedness in Least Squares Minimization Suppose we have the quadratic minimization problem $$\min_x \frac{1}{2} x^TPx + q^Tx +r$$ We know that when P is symmetric positive semi-definite, but the optimality ... 1answer 103 views ### What am I reinventing? RE: Linear regression modeling for frequency of discrete events I'm looking to model the frequency of events to quantify how much that frequency is increasing or decreasing. For the sake of concreteness think of the events as web page hits for several low traffic ... 1answer 71 views ### Biased linear regression I have a set S of coordinates (x,y), and am estimating f(x) = ax + b where a > 0. I also happen to know that \forall x,y((x,y) \in S \implies y < f(x)). The question is how I can ... 1answer 277 views ### How to find a parametric equation? I want to find an equation for a race track, so I could get the position of a point with respect to time. Let's say I have this track and here are a few points on it: Could it be possible to model ... 1answer 106 views ### Nonlinear regression with correlated errors it's my first post here and I'm a newbie in statistics, so please forgive me if I'm doing something wrong or explaining myself badly. Anyway, I have a problem similar to this: How to perform ... 1answer 585 views ### variance of multiple regression coefficients If I consider universal kriging (or multiple spatial regression) in matrix form as: {\bf{V = XA + R }} where \bf{R} is the residual and \bf{A} are the trend coefficients, then the estimate of ... 1answer 69 views ### Probabilistic regression on outliers I have a given data set D = \{ x_i, y_i \}_{i=1}^n for a regression problem. When I plot the data, it looks like there is an underlying parabola (2nd order linear model) and some outliers. I want ... 3answers 2k views ### Log-likelihood gradient and Hessian Considering a binary classification problem with data D = \{(x_i,y_i)\}_{i=1}^n, x_i \in \mathbb{R}^d and y_i \in \{0,1\}. Given the following definitions: f(x) = x^T \beta p(x) = \sigma(f(... 1answer 535 views ### Curve fitting with upper and lower bounds for derivatives I compute (at a great cost) upper and lower bounds f_u(x) and f_l(x) of an unknown function f(x) at points x in [0,1]. Now I am interested in an estimation of the derivative f'(x). I ... 2answers 228 views ### How to fit an equation to a curve with disturbances For example, I have the following data: Y = 366 measured values X = 366 measured values t = [ 1 : 366 ], representing the days of the year (index) So at each t (day), we have value of Y ... 2answers 155 views ### Choosing set of best estimators for linear least squares I have a measured experimental dataset which is well approximated by the sum of several basis functions in linear combinations. Linear least squares of course gives me the optimal weight for each ... 2answers 1k views ### Fitting data to a portion of an ellipse or conic section Is there a straightforward algorithm for fitting data to an ellipse or other conic section? The data generally only approximately fits a portion of the ellipse. I am looking for something that doesn't ... 0answers 24 views ### What is the relationship between the function \mathbb{E}(Y \mid X = x) and linear regression? Consider the function$$ r(x) = \mathbb{E}(Y \mid X = x) $$This has been called the regression function in a textbook I'm using. I'm trying to figure out the relationship between this function ... 0answers 13 views ### Derive the Hat Matrix to map actual response to estimated resposne In order to measure the quality of a regression we can calculate the Hat Matrix. Using it we can estimate the response variable as if we used the predictor variables to regress them. For linear ... 1answer 29 views ### Local quadratic approximation I wanted to implement some penalized regression parameter estimation algorithm by Fan&Li (http://sites.stat.psu.edu/~rli/research/penlike.pdf, section 3.3, [1]), but cannot catch the idea of some ... 0answers 33 views ### What does it mean to regress out current features? First of all, I'd like to say that this is the intro to a homework problem. Please do not post any answers, I am only looking for clarification on some terminology in the setup. I am trying to ... 1answer 50 views ### How do I calculate regression line using a data set with repeated values indicated as frequencies? I have a data set that comprises of Independent Variable (X) and Dependent Variable (Y) values with a certain frequency (F). I know that I have to find x^2 and xy but how do I factor in the ... 0answers 27 views ### Ridge Regression Centering Proof This is a ridge regression problem. The following two problems are equivalent: (w_t, b_\lambda ) = argmin_{w,b}\{\sum_{i=1}^m (y_i-b-w^Tx_i)^2+\lambda w^Tw\} (w_t, b_\lambda )= argmin_{w,b}\{\... 1answer 42 views ### Add weights to inputs of x-value function to optimize regression [closed] Say I have n functions (not the regression function) each with n inputs. These functions compute the x-values. The function is a simple summation function where the input is multiplied by a ... 0answers 46 views ### Does linear regression form a subspace? The author writes Given a vector of inputs X^T = (X_1, \dots ,X_p), we can predict an output Y via$$ \hat{Y} = \beta_0 + \sum_{j = 1}^p X_j \beta_j$$He goes on to note that if we include a 1 in ... 0answers 23 views ### standard deviation for regression The first slide is the denifition of simple linear regression model, the second slides is an example I have two questions: 1.Did I get the right calculation of the standard deviation? 2.I still ... 0answers 27 views ### Non linear regression calculator Are there any really good non linear regression calculators around the web? Or is something like matlab the best solution? I tried using excel and its solver tool, but it's complete garbage lol. ... 1answer 34 views ### Regression maximum likelihood Given this regression model:$y_{i}=\beta_{0}+\beta_{1}x_{i}+E_{i}$. All the assumptions are valid except that now:$E_{i}\sim N(0,x_{i}\sigma^{2})$Find Maximum likelihood parameters for$\beta_0$,$...
Hi I as the title says I'm looking at the proof that $r^2$ = $R^2$ in the case of simple linear regression, but I don't understand one part. There are different versions of the proof, but in most of ...