Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
41 views

regression on principal component analysis

I have done a PCA to get my principal components and now do a principal component regression. In the PCR the 1., 2. and 4. component are significant and the 3. component is insignificant. Can anyone ...
1
vote
1answer
35 views

Unexplainable determination coëfficiënt

I have a series of data (more specific, they are coördinates of a package attached to the end of a mini level-luffing crane. The "flightpath" is linear and horizontal.) Now when I plot the data: ...
1
vote
3answers
153 views

How to fit logarithmic curve to data, in the least squares sense?

How to fit logarithmic curve to data, in the least squares sense? I have simple data of the type $(x,y)$, that is 2D. I need to fit curve of the type: $y = c_1 + c_2\ln(x)$. So I have the $x$'s and ...
2
votes
1answer
108 views

Creating a lift chart for a classification tree

This is likely a simple question but I'm new to data mining techniques and am trying to compare two different predictive models. I've created a logistic regression and a classification tree and would ...
0
votes
1answer
179 views

Regression with equally spaced set

I'm working on an algorithm (written in Python/Cython, but it reads like pseudo-code) that estimates the gradient of each point in noisy data, using a variable window size. It's working very well, but ...
0
votes
1answer
140 views

Jacobian in Levenberg-Marquardt for 4-Parameter equation

I am trying to fully understand how I can use Levenberg-Marquardt to minimise a 4 parameter equation. There are lots of fancy programs to do this but the documentation about the mathematics is ...
0
votes
1answer
67 views

How to check linear independence

How can I check the linear independence of my variables? I have this system $Ax=b$ where $A$ is a $N \times 4$ matrix. I want to check the linear independence between the 4 variables in matrix $A$.
0
votes
1answer
56 views

Large Regression dataset

For a project I need a large regression (least squares) dataset: If $n$ is the number of samples and $p$ the number of features, then I need $p < n$ and $p,n$ both very large. For example ...
1
vote
2answers
77 views

Multiple linear regression

For a homework we have to determine the effect of a predictor variable on an outcome variable using simple linear regression. We have lots of data (about 300 variables) and we may include some other ...
1
vote
0answers
53 views

Linear Regression with limited information

You have grades ($Y $) for men ($D = 0$) and women ($D = 1$). The mean grades (out of total possible score of 100) are 65 for men and 72 for women. Regression of $Y$ on $D$ yields: $Y_i = b_0 + ...
2
votes
1answer
2k views

Convert nonlinear regression equation to a linear regression equation

The question is: "Show how the nonlinear regression equation y=aX^B can be converted to a linear regression equation solvable by the method of least ...
2
votes
1answer
107 views

Best fit line using geometric distance (not vertical distance)

There must be a theory of finding the best fit line to a bunch of points in the plane, where "best fit" is defined by the geometric distance, not vertical distance. In other words, we are trying to ...
1
vote
1answer
126 views

Weird formula for linear regression

I'll try to make the matter as clear as possible given the circumstances. My boss asked me to look at an old report a former employee wrote around a couple of months ago. Apparently the report ...
1
vote
1answer
32 views

Non-close-form Regression Research

As I try to process some physic experiment data that I don't have the closed form formula with unknown parameters, I have to use some regression models like polynomials or normal distributions . The ...
0
votes
1answer
102 views

Formulating regression model in matrix notation

The observations $y_1, y_2, y_3$ were taken on the random variables $Y_1, Y_2, Y_3$ where $Y_1=\theta+e_1$ $Y_2=2\theta - \phi+e_2$ $Y_3=\theta +2 \phi+e_3$ and $E(e_i)=0, var(e_i)=\sigma^2 ...
3
votes
1answer
95 views

Solution to linear system of equations

Notation. Let $y$, $a$, and $b$ be $n\times 1$, $p\times 1$, and $q\times1$ real vectors. Let also $X$ and $Z$ be $n\times p$ and $n \times q$ real matrices. Suppose that there is no solution, $a$, ...
2
votes
1answer
64 views

Choosing the right regression

I'm trying to analyze my sleep using regression analysis. Each night is rated (dependent variable). I'm trying to explain this rating with, for example, my sleep duration and each night's bed time's ...
0
votes
1answer
47 views

Derive a formula of a specific curve

I have this curve And I know that the first point is $$A(0,5)$$ and the last point is $$C(1650,9.5)$$ The point almost at the center where the curve changes (if you look close, you can see a ...
1
vote
1answer
491 views

Simple linear regression - understanding given

The question is to fill out the missing numbers (A-L) of a simple linear regression model. I am having problems with converting and interpreting the given table in terms of variables. Would it be ...
1
vote
1answer
57 views

Multiple regression and hypothesis test $H_0$:$\beta_2=0$

Multiple regression model $H_0$:$\beta_2=0$, $H_1$:$\beta_2 \neq 0$ where $\beta_2$ is the vector of elements ($\beta_2, \beta_3, \dots, \beta_k$) and $\beta$ is slope of regression line. Why it is ...
0
votes
1answer
203 views

Is a traditional Multi Layer Perceptron Network capable of non-linear regression? Which activation function should be used for that purpose?

I need to use a Multi Layer Perceptron Network in order to perform some non-linear regression. Any ideas if it's possible to perform a task like that and how? Which activation function should be used ...
0
votes
3answers
69 views

Finding a function given a set of points, without knowing what type of function

If I have a set of points, but am not sure what type of function they could fit, is there a way to find that function? Here are a few of the points... ...
0
votes
0answers
172 views

Which machine learning algorithm to use?!

I have a training set which is set of essays written by students for a question. These essays are all scored by human evaluators with labels such as 1, 2 , 3 which is actually marks allotted for those ...
1
vote
1answer
70 views

Implicit Curve Fitting

I have 100 points scattered in the 3D space along the $z$ coordinate axis. The points appear to lie on a curve. Is it possible to find an (implicit) curve that fit these points and option to insert ...
2
votes
0answers
40 views

Sampling data prior to nonlinear regression

As my question shows it, I am not a statistician. My problem is that I have too many data points to be used in a nonlinear fit (I have millions of them, automatically acquired). Is there a methodology ...
1
vote
1answer
99 views

Logistic regression with interactions

I'm supposed to do a model using logistic regression. So I have a series of $N$ observed data points each of which consists of $m$ explanatory variables $x = (x_{1,i}, ... , x_{m,i})$ and associated ...
1
vote
0answers
2k views

Prove that the expectation of residual sum of squares (RSS) is equal to $\sigma^2(n-2)$

The assumed regression model is $E(Y_i|x)=\gamma+\beta(x-\bar{x})$ and $Var(Y|x)=\sigma^2$. So I have: $E(RSS)= E(\sum\limits_{i=1}^n (y_i-\hat{y_i})^2)= E(\sum\limits_{i=1}^n ...
0
votes
2answers
91 views

Autocorrelation problem, regression analysis

Bit stuck on my econometrics course (old exam q), not big on mathematical statistics, anyway this is the problem: Given some model $y_{it}=\beta_0+\beta_1x_{it}+u_{it}$ and suppose that the ...
0
votes
1answer
62 views

Statistical inference and t-stats?

I have this linear regression model with an intercept(b0) and 3 variables(b1,b2,b3). Then they drop b2 and b3 and they give a new regression line with a new b0 and b1 and consequently new standard ...
0
votes
1answer
511 views

Calculate the covariance matrix of $\hat{\beta}$

$y=X\beta+u$ where $u \sim N(0,\Sigma)$ and $\Sigma$ is symmetric & idempotent. $X: n*k$, $y:n*1$, $\beta=k*1$, $u:n*1$ vector. Suppose you apply LS(least square) to the model. Calculate the ...
0
votes
1answer
489 views

Question about derivatives, Jacobian Matrix and non-linear least square curve fitting

I am working with a software package that performs non-linear least squares curve fitting. The goal of curve fitting is to calculate the coefficients of an equation given x,y points. The software ...
0
votes
2answers
316 views

Regression analysis on temperature/sensor data

Looking for a solution to what I thought should be an easy problem, but has me running in circles somehow... I'm working with two sets of data: he first set is raw values from a sensor ...
2
votes
1answer
60 views

Basic Multilinear regression question for finding examples or counterexamples.

Hello Wise mathematicians! I have few quenstions about Multi linear regresstion. I've been asked from my friend, but I have very weak knowledge background from that field. It seems my friend is in ...
0
votes
1answer
46 views

Curve Fitting and Multiple Experiments

Say I do an an experiment 5 times, each of which gives you a list of data points. Do I fit a curve to each one separately and then average the parameters and their uncertainties? Or do I take the ...
0
votes
1answer
679 views

Show posterior probability takes the form of the logistic function

Suppose you have a D-dimensional data vector $x$ = ($x_1$, ..., $x_n$) and associated class variable $y \in \{0, 1\}$, which is Bernoulli with parameter $\alpha$. Assume the dimensions of $x$ are ...
0
votes
0answers
279 views

What does the superscript T refer to in the following equations regarding non-linear least squares curve fitting?

In the paper, http://people.duke.edu/~hpgavin/ce281/lm.pdf, equations 1), 2), and 3) refer to the chi-squared error criterion. Equation 2) is reproduced below. $$ \chi^2({\mathbf p}) = (\mathbf y- ...
0
votes
1answer
82 views

Statistics - Covariance and variance question

Please fill in the intermediate steps $$\sum_{i=1}^nx_i(x_i-\bar x)=\sum_{i=1}^n(x_i-\bar x)^2$$ and $$\sum_{i=1}^nx_i(y_i-\bar y)=\sum_{i=1}^n(x_i-\bar x)(y_i-\bar y)$$
0
votes
1answer
317 views

Quadratic form as a ratio of determinants

I am looking for some hints to prove the following equality: $y^{\top}y - y^{\top}X(X^{\top}X)^{-1}X^{\top}y = \dfrac{\det(L^{\top}L)}{\det(X^{\top}X)},$ where $y$ is a $n\times 1$ vector, $X$ is a ...
1
vote
2answers
94 views

Is there a way to fit an even function using only odd functions?

I was wondering if there is a way to make an infinite series of odd functions equal to an even function. For example, I would like to know if the next equation is valid ...
1
vote
1answer
261 views

Mathematical model building with dependent and independent variables

I have been working with data and building models on data. I have developed models in regression using cubic and power series. It works fine for variables with one dependent and one independent ...
3
votes
1answer
130 views

Constraining estimated linear regression coefficients over several regressions

I'm trying to run a series of simultaneous linear regressions, and I want to constrain the regression coefficients. For the standard ordinary least squares regression, the specification of the ...
1
vote
0answers
146 views

Multivariate regression model

Supposing I have a graph involving time (x axis time in seconds) and log of nubers of bacteria: I would like to adjust a model given by a formula involving 5 coefficients like: $Number = Log\left ...
0
votes
1answer
25 views

Formula completion

I have, for v and x this values: ...
1
vote
1answer
51 views

Iterative Power Regression

If I have a set of data points that would fit inside a power equation of the form y = a*x^b, what is the best ITERATIVE method to find the values of 'a' and 'b'. I thought I could compute the error ...
2
votes
1answer
124 views

Why does the regression line of $x$ on $y$ and $y$ on $x$ meet at $\bar{x}$ and $\bar{y}$?

Why does the least squares regression line of $x$ on $y$ and $y$ on $x$ intersect at $\bar{x}$ and $\bar{y}$? Also, why are the form of regression lines as they are? For the general form ...
1
vote
1answer
2k views

Mean response in linear regression

What does mean response in linear regession mean? I don't understand the definition given in wikipedia. This is the definition: Mean response is an estimate of the mean of the $y$ population ...
0
votes
1answer
168 views

Minimizing squared error between several datasets

I'm just starting to get back into math for some computer programs i am writing and I've run into a complex regression-like problem. Its been a long time since grade school and i don't even know which ...
0
votes
1answer
69 views

Proving the equality of two expressions

The context of the following identity is in the Classical Normal Linear Regression Model, ie, $\boldsymbol{y} = \boldsymbol{X}\boldsymbol{\beta}+ \boldsymbol{u}$ where $\boldsymbol{u}$ is a $n \times ...
24
votes
3answers
8k views

derivative of cost function for Logistic Regression

I am going over the lectures on Machine Learning at Coursera. I am struggling with the following. How can the partial derivative of ...
2
votes
1answer
66 views

Probit model question (regression)

I'm reading a thesis and I need your help to understand the equation below. $$\Pr(\text{failure}=1 \mid X_1,X_3,X_3,X_4)=\int_{-\infty}^z \varphi(k) \, dk\tag{1}$$ $\varphi(k)$ is the standard ...