Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
27 views

Unexplainable determination coëfficiënt

I have a series of data (more specific, they are coördinates of a package attached to the end of a mini level-luffing crane. The "flightpath" is linear and horizontal.) Now when I plot the data: ...
1
vote
1answer
588 views

Convert nonlinear regression equation to a linear regression equation

The question is: "Show how the nonlinear regression equation y=aX^B can be converted to a linear regression equation solvable by the method of least ...
1
vote
1answer
66 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
117 views

Simple linear regression - understanding given

The question is to fill out the missing numbers (A-L) of a simple linear regression model. I am having problems with converting and interpreting the given table in terms of variables. Would it be ...
1
vote
1answer
67 views

Logistic regression with interactions

I'm supposed to do a model using logistic regression. So I have a series of $N$ observed data points each of which consists of $m$ explanatory variables $x = (x_{1,i}, ... , x_{m,i})$ and associated ...
1
vote
1answer
432 views

Mean response in linear regression

What does mean response in linear regession mean? I don't understand the definition given in wikipedia. This is the definition: Mean response is an estimate of the mean of the $y$ population ...
1
vote
1answer
136 views

Machine Learning, why not use matrix multiplication instead of gradient descent?

If we want to minimize our Cost function for a given set of data, why do we use gradient descent and continually guess values until we find a min value for theta when when can just use matrix ...
1
vote
1answer
149 views

How do I solve the weighted normal equations?

I am trying to solve the normal equations for a 3D LSE of a general quadric: $$ z = ax^2 + bx + cxy + dy^2 + ey + f$$ Write as a vector equation: $$ \vec{z}= \bf{X}\vec{\beta}$$ where the 'ith row ...
1
vote
1answer
181 views

Linear fit with horizontal and vertical error bars

I'm searching an equation to calculate the parameters for a linear fit. With parameters a and b, the $\chi ^{2}$ is used: $\chi ^{2} = \sum_{i=0}^N (y_{i}-a.x_{i}-b)^{2}$ And with errors: $\chi ...
1
vote
1answer
113 views

Simple Linear Regression Question

Let $Y_{i} = \beta_{0} + \beta_{1}X_{i} + \epsilon_{i}$ be a simple linear regression model with independent errors and iid normal distribution. If $X_{i}$ are fixed what is the distribution of ...
1
vote
1answer
694 views

Prediction Model for forecasting using Linear regression

I am very new to inferral statistics. I am trying to build a prediction model for forecasting the revenue for physicians based on some historical data. I was planning to use Multiple Linear Regression ...
1
vote
1answer
93 views

How to gather useful information from a residue plot

You can usually see how good your linear regression line is by looking at the residue plot. If you see the points randomly distributed, you're good. But if you see a pattern, it means there is ...
1
vote
1answer
48 views

What is the way to determin how good a sequence will interpolate?

Say I have to sequences of numbers: $$[5, 10, 14, 21, 27, 31]$$ $$[1, 20, 21, 22, 30, 31]$$ Even though they both get to $31$ by the $6$th element, logic tells me that only the first one is a good ...
1
vote
1answer
44 views

Cointegration for Price levels Time Series

I don't understand why is the difference between price levels is a stationary process while the time series of price levels themselves is a non-stationary process. For example: ...
1
vote
2answers
265 views

Question about logistic regression

A logistic regression is meant for a binary/categorical variable. Sort of like age vs baldness. 1) So, does the "S-curve" regression equation output give the odds of having that condition for a ...
1
vote
1answer
221 views

Linear regression: b1 has the minimum variance among all unbiased linear estimators of beta1

There is a proof provided in Applied Linear Regression Models (1983) by Kutner et al. (Page 64), which is quite clear and easy to understand, except one point, namely, it assumes that $\sum k_i d_i = ...
1
vote
1answer
242 views

QR factorization for ridge regression

I am solving an overdetermined system of equations: $$Ax= b$$ Using QR factorization, we can solve this system easily by posing it as: $$Rx= Q'b$$ I would like to regularize my estimate of $x$. I ...
1
vote
1answer
329 views

A couple of questions on the NURBS basis functions

I read a little about NURBS curves (specifically from http://www.cs.mtu.edu/~shene/COURSES/cs3621/NOTES/), and I have a couple of questions about the motivation behind the choices made in designing ...
1
vote
2answers
130 views

Least Squares Derivation

I was reading this to review the derivation of the ordinary least squares estimator but I'm having trouble differentiating (4). Can someone please help explain why $ \dfrac{\partial ...
1
vote
2answers
488 views

Exponential Regression Model

I need to model my data ($(x,y)$ pairs) using the following exponential function: $$f(x) = \exp((x + a)/b) - c$$ So, I need to find $a, b, c$ coefficients that are the best fit for my data. What is ...
1
vote
1answer
217 views

Approximation using Legendre polynomials

my aim is to fit data points by the use of Legendre Polynomials. Has anybody experience with this task? My final aim is to do this automatically with mathematica. Thanks, rainer
1
vote
2answers
384 views

How to do a regression with only integer values and a fixed intercept?

I need to write some code for an application that takes in a series of 2D points whose values are integers, and determines a polynomial regression that passes through the origin. I know how to do this ...
1
vote
1answer
94 views

Orthogonal fitted values

I have two regression models $$Y=X\beta+\varepsilon,\quad \beta\in\mathbb{R}^k$$ $$Y=Z\alpha+u\quad \alpha\in\mathbb{R}^m$$ it is known that using OLS estimates $\hat{\beta},\hat{\alpha}$ fitted ...
1
vote
2answers
370 views

vertical vs. horizontal regression

A horizontal regression is defined as the following: $$m=\frac{\sum_{i=1}^n (x_i-\operatorname{average(x)})(y_i-\operatorname{average(y))}}{\sum_{i=1}^n (x_i-\operatorname{average(x)})^2}$$ whereas ...
1
vote
2answers
721 views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = ...
1
vote
1answer
112 views

Econometrics OLS estimates

I cant seem to use the formula to calculate B1 without knowing xi and yi. Is it possible to calculate using just the variances and covariance? Please help! The classical linear regression model ...
1
vote
1answer
39 views

Getting the formula of a live counter

I'm looking to replicate this greenhouse gases counter in my website. Poking around i found the initial data for the formula. The counter use the following information: Beginnig date: 2012/03/01 ...
1
vote
2answers
359 views

Simple Least Squares Regression?

I have a vector X of 50 real numbers and a vector Y of 50 real numbers. I want to model them as y = ax + b How do I determine a and b such that it minimizes the ...
1
vote
3answers
605 views

Find parameters for exponential function fitting to datapoints

I have a set of datapoints, in this case the temperature of an object adjusting to the environment temperature over time. Because I know these kind of processes take the form of $$f(x)=Ae^{x/B}+C$$ I ...
1
vote
4answers
205 views

How to fit a curve to my data

I have a datasheet. It looks like an hyperbola. How can I fit a curve to it? And how can I plot a curve of the first derivative? ...
1
vote
1answer
344 views

Curve fitting with upper and lower bounds for derivatives

I compute (at a great cost) upper and lower bounds $f_u(x)$ and $f_l(x)$ of an unknown function $f(x)$ at points $x$ in $[0,1]$. Now I am interested in an estimation of the derivative $f'(x)$. I ...
1
vote
1answer
82 views

Measuring how monotonically “staircase-like” a set of values is

A bit of a bizarre question here -- I'm looking for assistance in generating a robust metric to measure how monotonically "step-wise" a series of values is. The set must not start or end at a specific ...
1
vote
1answer
1k views

help with using the “simple regression (least squares) method” of forecasting

This problem is from an engineering management textbook (Morse & Babcock, 5th ed) : 2005 $48k 2006 $64k 2007 $67k 2008 $83k "What is the ...
1
vote
1answer
301 views

bayesian networks for regression

Would it be possible to use bayesian network for regression and/or prediction? I understand that it is a tool one can use to compute probabilities, but I haven't found much material about possible ...
1
vote
1answer
324 views

Recursive coefficient of determination (R2)

Is there a way to compute the coefficient of determination $R^2$ in a recursive way? $R^2$ is defined as following: $$R^2 \equiv 1 - \frac{SS_{\rm err} }{ SS_{\rm tot}} = 1 - \frac{\sum_i (y_i - ...
1
vote
1answer
130 views

can an artificial neural network with only one hidden layer fit all purposes/applications/functions?

I have heard that only a single layer is needed for an ANN to fit any possible function (input to output). Is this true and where is this investigated/state/found? Then what is the advantage of having ...
1
vote
2answers
25 views

Errors and Residual

Why are errors independent but residuals dependent? As far i know the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also ...
1
vote
2answers
22 views

Simple linear regression seems off

I have the following datapoints: $$p1(52,730)$$ $$p2(53,409)$$ $$p3(52,250)$$ $$p4(52,90)$$ Now I want to find the best fitting line between these points. When I use simple linear regression I get $$y ...
1
vote
2answers
30 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
1
vote
0answers
12 views

Non-Linear Regression involving the maximum function

How do you calculate the regression of this model? I know Minitab and MATLAB, so if you guide me with these software I would totally appreciate it. $$Y=c+\max(X^{-n}, 0.34) $$ Here c and n are ...
1
vote
0answers
14 views

Good MSE doesnt imply good prediction in logistic regression?

I am writing some code for regularized logistic regression. I observe this interesting phenomena and wonder if it is a normal thing or just my code is wrong. For loss function, I am using the ...
1
vote
0answers
20 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
1
vote
0answers
8 views

Finding posterior of normal distributions and logistic regression.

$P(w_0 | x) = \frac{1}{1 + e^{-log\frac{P(x|w_0)}{P(x|w_1)}-log\frac{P(w_0)}{P(w_1)}}}$ Note: x = $[x_1, \dots, x_d]^T$; a $d$ dimensional vector. $w$ can take on one of two values: $w_0$ or $w_1$. ...
1
vote
1answer
35 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
1
vote
0answers
20 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
1
vote
1answer
22 views

Model selection in regression: Estimated parameters seem to be “non-significant”

I have conducted an experiment which manipulated three factors (Factor 1: 3 levels, Factor 2: 2 levels, Factor 3: 2 levels). The response variable is binomially distributed (1 = correct or 0 = not ...
1
vote
1answer
19 views

How to perform a monotonic function fitting of data points?

I'm seeking suggestions for general purpose function fitting of a set of data points, where, based on physical intuition, the relationship is expected to be "monotonic", i.e. the function should be ...
1
vote
2answers
24 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
1
vote
2answers
37 views

Autocorrelation and var-cov matrix

$$Y_t=\beta_1+\beta_2 X_{t2}+\dots +\beta_k X_{tk}+\epsilon_t \qquad (t=1,\dots,T)$$ $$\epsilon_t=\rho \epsilon_{t-1}+v_t, \qquad v_t \sim \mathrm{i.i.d.}(0,\sigma^2_v)$$ GLS estimation under AR(1) ...
1
vote
0answers
27 views

Properties of best linear predictor?

Conside two scalar random variables, $Y,X$. The best linear predictor of $Y\mid X$ under square loss function is $\theta_0=\operatorname{argmin}_{\theta} ...