Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
20 views

Is total least square solution only valid for isotropic error

Let $\mathbf{y} = \mathbf{Ax}$ represent a system of equation where, $\mathbf{y}\in\mathbb{R}^n, \mathbf{A}\in\mathbb{R}^{n\times m}$. However due to error in sensor, what we observe is the following ...
0
votes
0answers
96 views

Simple curve fitting smoothing algorithm

Is there any conditionally simple algorithm to obtain a best fit (equation) for points set? I have a few points for which I want to have a "smooth" function is constructed that approximately fits the ...
0
votes
0answers
59 views

How to find a perfect regression fit in R?

I have a set of points, which I know can be described with some equation. How can I find this equation? The scatter plot for this set looks like this: I look at the plot and assume that I can use a ...
0
votes
0answers
64 views

How to calculate relation between Beats (as in BPM) and variance of the energies?

I'm working on an implementation of a Beat Detection algorithm and I can't find a relation between the statistical variance of the energies in 1 second of audio decoded as PCM and the threshold value ...
0
votes
0answers
40 views

Understanding Piecewise Logistic Function

I am trying to understand the use and application of the piecewise logistic function/regression and have recently been shown how to apply it after getting some help at another forum. However, the ...
0
votes
0answers
40 views

Finding the curvature from a set of datapoints

I have a set of 1. 1-d 2. 2-d data. I want to find the curvature at each single point. Till now I was using difference technique to find out the curvature, i.e, central difference at middle and ...
0
votes
1answer
50 views

How do I solve for vector $P$ in the matrix equation $s=A'B^{-1}A$?

I would like to rearrange the matrix equation $s=A'B^{-1}A$ into the form $A=f(s,B)$ (i.e., some function of $s$ and $B$), where s is scalar, $A$ is $n\times 1$, $A'$ is the transpose of $A$, and $B$ ...
0
votes
1answer
26 views

finding a question about constrained regression - had a side constraint $x \geq y^*z$

I saw a question that asked about solving a non-negative least squares problem with $3$ unknowns, $(x,y,z)$. But there was an additional constraint, $x \geq y^*z$. Would appreciate getting the ...
0
votes
1answer
29 views

Getting VAR parameters from a research paper.

Many econometrics papers provide the parameters used in their VAR model. If I notate my VAR model as $$z_{t+1} = c + B z_{t} + \Sigma \epsilon_{t+1}$$ where $\epsilon \sim N(0, I)$, then I need to ...
0
votes
1answer
15 views

In a simple regression model estimated using OLS, the covariance between the estimated errors and regressors is zero by construction

Is this statement true or false? I seem to remember that this relationship does not hold when the regression has no intercept, however my teacher said that this was true regardless of whether we ...
0
votes
2answers
95 views

Non-linear regression fit

I'm trying to fit my data to the following equation: $$ Y = A(1-2e^{bx}) $$ What I tried to do was transform the equation to a linear form via the following steps: \begin{align*} & A-Y = ...
0
votes
1answer
16 views

Least Squares Estimators Derivation (Bi-variate)

To derive least square estimators: We have $SS(\alpha,\beta)= \sum(y_i-\alpha-bx_i)^2$ and find partials for each. The answer I get is: $\beta = \frac{\sum y_i-\bar{y}}{\sum x_i-\bar{x}}$, but the ...
0
votes
2answers
38 views

Calculating the correlation coefficient between least square estimates

PROBLEM STATEMENT: Consider the following 2-variable linear regression where the error $e_i$ 's are independently and identically distributed with mean $0$ and variance $1$; $$y_i = α + β(x_i − \bar ...
0
votes
0answers
28 views

Transpose is just the way of generalizing a dot product?

It seems like $a^Tb$ is the same as writing $a \cdot b$ in matrix form. 1) Why is $n \times 1$ and $n \times 1$ matrix multiplication undefined? 2) Is this just a generalization of the dot ...
0
votes
1answer
36 views

Method for ?not quite? weighted least squares fitting for more realistic results

I need a linear least squares type of fitting algorithm that understands how to weight the probability of a response coming from certain functions over another. To explain, given the standard linear ...
0
votes
0answers
32 views

Relation between the Coefficient of Multiple Correlation and Coefficient of Simple Correlation

Consider the regression model $Y=\beta_1 X_1+\beta_2 X_2+\epsilon$, with a sample of size $n$, $Y_i=\beta_1 X_{i1}+\beta_2 X_{i2}+\epsilon_i$, $\epsilon_i \in N(0,\sigma^2)$. Suppossing ...
0
votes
1answer
39 views

Quadratic Form Matrices

How do I know if a matrix in quadratic form, e.g. D'MD is positive or negative (semi)definite? M here is the residual maker matrix for X, so I know that it is symmetric. I know what the definitions ...
0
votes
0answers
82 views

Solving constrained linear programming problem

For the variable $t$, problem is to find best multipliers $k$ which minimizes the objective function. Time: $t_1$, $t_2$, $t_3$,... given in input Multiplier $k_1$, $k_2$, $k_3$,... (These are ...
0
votes
0answers
20 views

If the null hypothesis is true, how will the test statistic be distributed?

I went with T~(50-6) The question goes.... "A regression is estimated with 50 observations, five explanatory variables and with a constant. Suppose You want to test the following hypothesis $H_0: ...
0
votes
0answers
18 views

Where can I find formulas for the multiclass logistic regression with bias term?

In most of the books and web sources and papers the multiclass logistic regression is introduced and discussed without bias terms. I am looking for generalised formulas using bias terms. The standard ...
0
votes
2answers
90 views

Rational function regression without poles in a interval, or polynomial regression with positivity constraint

I have some sets of experimental data for some functions $f_i$ from $I=[0,1]$ onto itself, which should satisfy the following physical constraints: $f_i(0)=1$ $f_i(x) \in I= [0,1] \; \forall x \in I ...
0
votes
0answers
65 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!
0
votes
1answer
59 views

How to find Parameters in nonlinear Regression Model?

I have a nonlinear Regression Model with eleven observations of $x,y$. How do I find the parameters $a,b,c,d$ of the model: $ y=f(x)=a + b \sin cx e^{dx}$ by using the function: $$\Phi(a, b, c, ...
0
votes
0answers
39 views

Correlation/Regression for Continuous and Discrete data

I want to correlate a data where one axis is continuous (ranging from 0 to 1), other axis is discrete. Discrete axis scale is 1 to 5 (1 is for Strongly Disagree and 5 is for Strongly agree). How ...
0
votes
0answers
27 views

Linear regression with rounded down dependable variable.

I have a problem where I need to find the underlying linear relationship between an independent variable and it's dependent variable. However, I know that the dependent variable is being rounded down ...
0
votes
1answer
25 views

Does scatterplot matrix “work” with quadratic variables?

basically I want to plot a scatterplot matrix using a few variables. For simplicity lets say my model is: $$z=\alpha_0 + \alpha_1w+\alpha_2x+\alpha_3y+\alpha_4y^2 + \epsilon$$ When I plot the matrix, ...
0
votes
0answers
34 views

Statistical Multiple Linear Regression Log Transformation

If for example we have a multiple linear regression as follows: $$hydrcarb=x_1+x_2tanktemp+x_3disptemp+x_4tankpres+x_5disppres+x_6tankpres^2+x_7dispres^2$$ And I am trying to do a backward ...
0
votes
0answers
33 views

The correlation between alpha and beta

Consider the following 2-variable linear regression where error $e_i$'s are independently and identically distributed with mean 0 and variance 1; $$ y_i=\alpha + \beta (x_i - \bar {x}) + e_i$$ where ...
0
votes
0answers
14 views

Is it always possible to find a logistic regression model that yields zero training error on any dataset?

I am leaning towards no. A logit regression model is just one function, and there is no way its coefficients can accurately predict an entire dataset, outliers and all. Is this the correct intuition?
0
votes
1answer
53 views

Maximum and minimum penalty in lasso regression family

I am trying to adjust penalty, lambda, in group lasso regression, but I have no idea about it. Just to clarify, group lasso regression is a kind of multiple linear regression which use penalties on ...
0
votes
1answer
38 views

Show ARIMA(1,1) with mean $\mu$ is an ARMA process

I am trying to show that an ARIMA(1,1) process with mean $\mu$ is an ARMA process, as well as to show if it causal and/or invertible. The set up is: Let $X_t$ be a causal and invertible ARMA(1,1) ...
0
votes
0answers
41 views

OLS: Estimation with negative coefficients

I have probably an easy problem, however I'm not really sure how to do it: Basically, I would like to estimate a linear regression with OLS. So far so easy. However, the model that I would like to ...
0
votes
0answers
34 views

Intuitive understanding about LSE

Let y= X$\beta$ + $\epsilon$ where y is $n \times 1$ vector, X is $n \times p+1$ matrix, $\beta$ is $p+1 \times 1$ vector, $\epsilon$ ~ $N(0, \sigma^2I_{n}$) now, let $y_{i}$ = $\hat{y_{i}}$ for some ...
0
votes
0answers
36 views

minimum orders in linear regression to get a perfect fit

The problem is that, $(X_i, Y_i), i = 1,\ldots, n$ is an i.i.d. bivariate sample. Show that it is possible to fit a polynomial model using least square such that the fitted values are equal to the ...
0
votes
0answers
25 views

Question about ridge estimator

I have tried to show that ridge estimator is the solution to following problem min $(\beta- \hat{\beta})^t$$X^t X$$(\beta- \hat{\beta})$ subject to $\beta^t \beta =< d^2$ and $\beta$ is a $p$ x 1 ...
0
votes
0answers
41 views

two way ANOVA and linear regression model.

I know that Analysis of variance model can be written as a linear regression model using indicator regressors. For, one way ANOVA, I can write down the regression model. But for two way ANOVA ...
0
votes
1answer
58 views

Power form of regression equation which is not centered at x=0?

For a given set of data, the power form of the regression equation is given by $$y=b\cdot x^{m}$$ where $$m=\frac{n(\sum \mathrm{ln}(x_i)\mathrm{ln}(y_i))-(\sum \mathrm{ln}(x_i))(\sum ...
0
votes
1answer
23 views

standard error for the parameters of a linear regression model

Given a linear model $\mathbf{y} = \beta \mathbf{X} + \epsilon$, it is well known that the estimate for $\beta$ that gives the minimum residual sum of squares (RSS) is given by $\hat{\beta} = ...
0
votes
1answer
44 views

How to report significant digits in coefficient of determination?

Say that I fit some data with some model, for instance a linear function $y = mx+b$. What is the proper way to report the fitted coefficients and the goodness of fit? Specifically, if I do the fit in ...
0
votes
0answers
88 views

Chi Square Formula and Degrees of Freedom Questions

I have a population sample with 200 points of data and 3 degrees of freedom so am I supposed to do a chi square formula with all 200 points of data? I believe that is what I'm supposed to do but I'm ...
0
votes
0answers
12 views

Difference Between Three Similar Error Reducing Algorithms

I found a Least Square Error Recognition algorithm that finds the least mean square error from a 2-d matrix element by element. Logistic regression from this site, on the other hand, seeks to ...
0
votes
0answers
22 views

L1 regression statistics

Consider fitting the below dataset using L1 regression: x: 8.3 8.3 12.1 12.1 17.0 17.0 17.0 24.3 24.3 24.3 33.6 y: 224 312 362 521 640 539 728 945 738 759 663 Why do the regression estimates ...
0
votes
0answers
3k views

How do I find Sxx in a Simple linear regression model?

In a Simple linear regression model, I have only Sxy and Syy data with me. How shall I derive Sxx, linking Sxy and Syy based on first principles? I know the formulas separately. I want to find Sxx, ...
0
votes
0answers
51 views

Classical Regression Model: Combining linearity and strict exogeneity

I am studying the Classical Regression Model for random samples. Hence consider the random sample $(y_i,\mathbf{x_i})$ Where: ...
0
votes
0answers
31 views

How to derive F statistic for general linear hypothesis

I want to derive the F statistic for general linear hypothesis $H_0$ : T$\beta$ = c vs $H_1$ : T$\beta$ $\not =$ c where T is $p * q$ matrix with rank q I have tried to express $SS_{Res} (RM)$ - ...
0
votes
1answer
163 views

how to apply weighting factor to linear regression

Say if I have two sets of data, x and y. And I am required to apply a weighting factor,1/x, to the regression line. Does that mean I should plot 1/y versus 1/x and then get the regression? Could ...
0
votes
0answers
26 views

Difference in coefficient estimates

I have simple regression $y_i=\beta_0+\beta_1x+\epsilon$. Then I have estimates $\hat{\beta_0},\hat{\beta_1}$ computed from n observation and estimates $\beta_0',\beta_1'$ computed from n+1 ...
0
votes
1answer
33 views

Questions about Multi linear regression model.

I have two questions about multi linear regression model. First question. Suppose 2 independent samples Sample1 : $y_1$, ... $y_{n_1}$ and $x_1$, ..., $x_{n_1}$ Sample2 : $y_{n_1 +1}$, ... $y_{n_1 ...
0
votes
0answers
14 views

relation within Gauss-Newton method for minimization

If we study model fit on a nonlinear regression model $Y_i=f(z_i,\theta)+\epsilon_i$, $i=1,...,n$, and in the Gauss-Newton method, the update on the parameter $\theta$ from step $t$ to $t+1$ is to ...
0
votes
0answers
22 views

Poisson distribution with normal informative priors

I'm Jia, a student of economics and finance. I was wondering if someone could help in understanding this problem. I've just started to attend a new course "Financial and nonlinear econometrics" and ...