Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

3
votes
2answers
30 views

Distance between a plane and set of points

Suppose $m$ data points belonging to a class represented by matrix $A$. Therefore, the size of matrix $A$ is $m\times n$. In addition, suppose $w\cdot x + b=0$ be equation of a plane in ...
0
votes
0answers
4 views

Is there a standard formula for calculating a project portfolio elasticity?

My project portfolio has a lot of variable numbers in it which could drastically change the cost, schedule and resource requirements of the portfolio. I was wondering if there is already some kind of ...
2
votes
0answers
19 views

combining multiple regression outputs

Suppose I have multiple regressions, along with their r-squares, standard-errors, etc.: $y(t) = \alpha_1 + \beta_1 x(t) + e_1$, where $t \in (\tau_0, \tau_1)$ $y(t) = \alpha_2 + \beta_2 x(t) + ...
-1
votes
0answers
23 views

Averaging between different formula of same kind

There is a dataset of 8 books. We examined the similarity of each book, using a technique, with other 19 books (8*19 number of distinct books). We stored the data and then, we used ...
-2
votes
0answers
12 views

how to find the corelation or regression between two tables [closed]

I have two tables , I need to find the dependence or relation between these two tables table one ... $3$ sport v/s four regions with value as sales table two .. same $3$ sport v/s same four regions ...
0
votes
0answers
14 views

tree models - adaptive basis functions

i am self-studying adaptive basis functions - and came across the following text on classification and regression trees. I am wondering: as the final region in a tree might be result of multiple ...
1
vote
1answer
20 views

Fit Quantized Piecewise Constant Function to Another Piecewise Constant Function

I have a situation where I have a function $$f(x) : [r_1,r_2]\in\mathbb{R} \rightarrow [r_3,r_4]\in\mathbb{R}$$ and I need to fit a function $$g(x) : [r_1,r_2]\in\mathbb{R} \rightarrow ...
0
votes
0answers
40 views

On a modified least square.

Given a vector $y \in \mathbb R^n$ and real constants $x_{ij}$ ($i=1,\dots,n$, $j=1,\dots,p$), we consider a vector $\beta = (\beta_0,\dots,\beta_p)$ which minimize ...
0
votes
0answers
15 views

Is total least square solution only valid for isotropic error

Let $\mathbf{y} = \mathbf{Ax}$ represent a system of equation where, $\mathbf{y}\in\mathbb{R}^n, \mathbf{A}\in\mathbb{R}^{n\times m}$. However due to error in sensor, what we observe is the following ...
1
vote
2answers
50 views

Minimizing a summation?

I have absolutely no idea how to approach this problem. I've been looking through notes, and I think I missed this when my professor discussed this in class. $$ \text{Consider the data}\\ i\: x_i\: ...
1
vote
2answers
37 views

Expected value and the standard simple regression model

Given the standard simple regression model: $y_i = β_0 + β_1 x_i + u_i$ What is the expected value of the estimator $\hat\beta_1$in terms of $x_i, \beta_0$ and $\beta_1$ when $\hat\beta_1=\sum x_i ...
0
votes
2answers
40 views

Approximate/Find Function

I have got values $x_{i}$ and targets $z_{i}$. Now I want to find a function $f(x)=z$, which approximates the mapping of my value $x_{i}$ to its targetvalue $z_{i}$ as good as possible for every ...
0
votes
0answers
30 views

Simple curve fitting smoothing algorithm

Is there any conditionally simple algorithm to obtain a best fit (equation) for points set? I have a few points for which I want to have a "smooth" function is constructed that approximately fits the ...
1
vote
1answer
24 views

Detrending sine waves accurately

I am doing some data analysis where I look at electricity demand over the course of a day, but need to separate the intra-day (constant and periodic) components from daily changes (assumed linear). At ...
3
votes
1answer
24 views

Quantile Regression - Linear Loss Minimization

I'm currently reading Quantile Regression by Roger Koenker, and for some reason I'm having a lot of trouble deriving one of his equation (sect. 1.3, p. 5-6). He goes on to demonstrate that $\hat{x}$ ...
2
votes
0answers
22 views

GLM for Poisson Regression for Soccer Ratings Not Converging

I have been trying to formulate a model of soccer matches to help me predict the outcomes. The model I'm trying to formulate involves using Poisson regression to assign attack and defence ratings to ...
0
votes
0answers
31 views

How to find a perfect regression fit in R?

I have a set of points, which I know can be described with some equation. How can I find this equation? The scatter plot for this set looks like this: I look at the plot and assume that I can use a ...
0
votes
0answers
52 views

How to calculate relation between Beats (as in BPM) and variance of the energies?

I'm working on an implementation of a Beat Detection algorithm and I can't find a relation between the statistical variance of the energies in 1 second of audio decoded as PCM and the threshold value ...
0
votes
0answers
36 views

Understanding Piecewise Logistic Function

I am trying to understand the use and application of the piecewise logistic function/regression and have recently been shown how to apply it after getting some help at another forum. However, the ...
2
votes
1answer
75 views

Least-squares solution to a matrix equation?

Suppose I have $n$ observations of $m$ dependent variables $y_1,\dots,y_m$, and I believe they follow some model wherein they can all be written as linear combinations of some underlying variables ...
3
votes
0answers
19 views

Regression with many discrete and continuous predictors and few rows

I want to do regression on a dataset. It has one continuous dependent variable that I want to predict. It has many categorical and some continuous predictors. It only has a few rows. A simplified ...
1
vote
3answers
53 views

Transposing matrix when differentiating it

Hi so I am trying to understand the solution of linear regression with matrices (found at the following link) and an confused about how on page 10 he says the derivative of $2Y'XB$ with respect to $B$ ...
0
votes
0answers
10 views

Softmax Regression Gradient Derivation

I'm implementing softmax regression and am deriving the max-log-likelihood update for gradient descent by hand first. Coming from the Stanford UFLDL site, they show the gradient of the cost function ...
1
vote
0answers
17 views

LRT and Wald Test in Multivariate Linear Regression significance

I've been researching all afternoon trying to get a better idea of what the Likelihood ratio test and the Wald test are actually doing. I have a bunch of covariates and I'm testing out like 30 ...
0
votes
0answers
19 views

Finding the curvature from a set of datapoints

I have a set of 1. 1-d 2. 2-d data. I want to find the curvature at each single point. Till now I was using difference technique to find out the curvature, i.e, central difference at middle and ...
1
vote
1answer
25 views

Significance of dummy variables in probit regression

I am an undergraduate student working on some projects using probit regression. I have a question on dummy variables that I was hoping someone could help me with (which I think stems from an ...
1
vote
1answer
14 views

Find a distribution for this plot

Please help me find a formula that fits the distribution. It does not need to be exact, a simple approximation would suffice. Bonus points if you can tell me which predefined distribution in the ...
0
votes
1answer
29 views

Least squares with known error in y

so I want to do a linear least squares regression on my data, however I have known experimental error on my data points in $y$ and relatively few numbers of points so I would like to use values ...
0
votes
1answer
31 views

How do I solve for vector $P$ in the matrix equation $s=A'B^{-1}A$?

I would like to rearrange the matrix equation $s=A'B^{-1}A$ into the form $A=f(s,B)$ (i.e., some function of $s$ and $B$), where s is scalar, $A$ is $n\times 1$, $A'$ is the transpose of $A$, and $B$ ...
0
votes
1answer
23 views

finding a question about constrained regression - had a side constraint $x \geq y^*z$

I saw a question that asked about solving a non-negative least squares problem with $3$ unknowns, $(x,y,z)$. But there was an additional constraint, $x \geq y^*z$. Would appreciate getting the ...
0
votes
1answer
24 views

Getting VAR parameters from a research paper.

Many econometrics papers provide the parameters used in their VAR model. If I notate my VAR model as $$z_{t+1} = c + B z_{t} + \Sigma \epsilon_{t+1}$$ where $\epsilon \sim N(0, I)$, then I need to ...
1
vote
3answers
51 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
0
votes
1answer
9 views

In a simple regression model estimated using OLS, the covariance between the estimated errors and regressors is zero by construction

Is this statement true or false? I seem to remember that this relationship does not hold when the regression has no intercept, however my teacher said that this was true regardless of whether we ...
-1
votes
0answers
21 views

(x/y) v (y/x) as predictor in regression

I am trying to predict a variable 'score' with x and y. I believe it is related to (x/y)/z and z/(x/y), but I'm not sure which. Here is some data: ...
0
votes
1answer
27 views

Polynomial least squares fit — restrictions on order?

If we're finding an interpolating polynomial for 10 data pairs, the order of the polynomial has to be 9. In class, my professor said that when doing a polynomial least squares fit, if you have 10 ...
0
votes
2answers
79 views

Non-linear regression fit

I'm trying to fit my data to the following equation: $$ Y = A(1-2e^{bx}) $$ What I tried to do was transform the equation to a linear form via the following steps: \begin{align*} & A-Y = ...
3
votes
0answers
28 views

How to perform nonlinear regression with regressors affected by gaussian error?

I am trying to calibrate a sensor and I have a data set consisting of several observations of a 3-dimensional vector $X_i$, with $X_i=w_i + \epsilon_i$ where $w_i$ is the value that the sensor ...
1
vote
2answers
65 views

Fit exponential with constant

I have data whic would fit to an exponential function with a constant. So y=aexp(bt) + c Now I can solve an exponential without a constant using least square by taking log of y and making the ...
0
votes
1answer
11 views

Least Squares Estimators Derivation (Bi-variate)

To derive least square estimators: We have $SS(\alpha,\beta)= \sum(y_i-\alpha-bx_i)^2$ and find partials for each. The answer I get is: $\beta = \frac{\sum y_i-\bar{y}}{\sum x_i-\bar{x}}$, but the ...
1
vote
1answer
28 views

Multiple Linear Regression in Matrix Form

I am currently studying for my exams and came across the following question: State the multiple linear regression equation in matrix form. Write down the order of each matrix and explain what the ...
2
votes
0answers
23 views

Can I adjust linear growth of a a subpopulation to a linear decay of the general population?

I need to estimate the amount of CF patients in Poland in the next four years. I have: estimations of the Polish population for the future years a CF patients' register for the last couple of years ...
0
votes
2answers
26 views

Calculating the correlation coefficient between least square estimates

PROBLEM STATEMENT: Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean $0$ and variance $1$; $$y_i = α + β(x_i − \bar ...
0
votes
1answer
24 views

Why is the SSE <= SST?

I can think of a regression line producing a larger sum of squared errors (SSE) than the total sum of squares (SST). I read that should not be possible, how come? My understanding is that the ...
2
votes
1answer
59 views

Outlier detection with robust multiple regression model

I have a set of features (eg, location, income, budget, education) that I use to predict a continuous variable (say, amount spent per day on the internet). I am interested in detecting outliers. I ...
0
votes
0answers
22 views

Transpose is just the way of generalizing a dot product?

It seems like $a^Tb$ is the same as writing $a \cdot b$ in matrix form. 1) Why is $n \times 1$ and $n \times 1$ matrix multiplication undefined? 2) Is this just a generalization of the dot ...
1
vote
1answer
13 views

Do we only use T distribution for confidence interval of Beta for linear regression or we can also normal distribution?

Do we only use T distribution for confidence interval of Beta for linear regression or we can also normal distribution? Is it that when sample size is less than 30 then we use T distribution else ...
4
votes
5answers
43 views

calculating least squares fit

I read this thread talking about 'why we use least squares' for curve fitting Why do we use a Least Squares fit? One answer by Chris Taylor begins with the assumption that we should look for $$ ...
1
vote
4answers
50 views

How to estimate $\alpha$ in $y=(1-\exp(-\alpha x))/(1+\exp(-\alpha x))$?

I have a function $$y=\dfrac{1-\exp(-\alpha x)}{1+\exp(-\alpha x)}$$ where $y$ is not binary. The range of this function is $[-1,1)$. So this does not fit into either logit or probit models. How ...
0
votes
1answer
47 views

Identical observations in linear regression

I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
0
votes
1answer
20 views

Method for ?not quite? weighted least squares fitting for more realistic results

I need a linear least squares type of fitting algorithm that understands how to weight the probability of a response coming from certain functions over another. To explain, given the standard linear ...