Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
0answers
9 views

Approximation technique when data is missing?

I am doing some statistical studies and I would appreciate some guidance to some approximation techniques when not all data is available. I have a model that takes certain input parameters (discrete, ...
0
votes
0answers
11 views

Statistical Multiple Linear Regression Log Transformation

If for example we have a multiple linear regression as follows: $$hydrcarb=x_1+x_2tanktemp+x_3disptemp+x_4tankpres+x_5disppres+x_6tankpres^2+x_7dispres^2$$ And I am trying to do a backward ...
0
votes
2answers
27 views

How we can linearize this equation?

I have an equation that it seems to be a non-linear equation. I want to compute the parameters a1 till a4.I want to simply do a linear regression to find the parameters, which is much easier than a ...
0
votes
1answer
21 views

X - axis of a linearized polynomial.

The other day in my Physics class we had some sample data that we wanted to linearize. The graph resembled a root curve. So to linearize it, we took the square root of all the x data and replotted ...
0
votes
1answer
11 views

Regression project in octave/matlab

I'm trying to establish a polynomial model to adjust the variation of the dollar throughout the year. Suppose hypothetically that I have the following data ...
0
votes
0answers
15 views

The correlation between alpha and beta

Consider the following 2-variable linear regression where error $e_i$'s are independently and identically distributed with mean 0 and variance 1; $$ y_i=\alpha + \beta (x_i - \bar {x}) + e_i$$ where ...
1
vote
0answers
24 views

Does gradient descent and normal equation give the same answer?

I tried to optimize for a linear regression model using both approaches and they gave me two completely different answers. My sample data set was: df <- data.frame(c(1,5,6),c(3,5,6),c(4,6,8)) ...
0
votes
0answers
10 views

Is it always possible to find a logistic regression model that yields zero training error on any dataset?

I am leaning towards no. A logit regression model is just one function, and there is no way its coefficients can accurately predict an entire dataset, outliers and all. Is this the correct intuition?
0
votes
1answer
14 views

Maximum and minimum penalty in lasso regression family

I am trying to adjust penalty, lambda, in group lasso regression, but I have no idea about it. Just to clarify, group lasso regression is a kind of multiple linear regression which use penalties on ...
1
vote
1answer
33 views

finding column vectors - linear transformations

$L:\mathbb{R}^3\rightarrow \mathbb{R}^2$ with bases $\mathcal{S}=\left\{\left(-1,1,0\right),\left(0,1,1\right),\left(1,0,0\right)\right\} \: \text{for} \:\mathbb{R}^3 \:\text{and} \\ ...
0
votes
0answers
21 views

Show ARIMA(1,1) with mean $\mu$ is an ARMA process

I am trying to show that an ARIMA(1,1) process with mean $\mu$ is an ARMA process, as well as to show if it causal and/or invertible. The set up is: Let $X_t$ be a causal and invertible ARMA(1,1) ...
0
votes
0answers
28 views

OLS: Estimation with negative coefficients

I have probably an easy problem, however I'm not really sure how to do it: Basically, I would like to estimate a linear regression with OLS. So far so easy. However, the model that I would like to ...
0
votes
0answers
31 views

Intuitive understanding about LSE

Let y= X$\beta$ + $\epsilon$ where y is $n \times 1$ vector, X is $n \times p+1$ matrix, $\beta$ is $p+1 \times 1$ vector, $\epsilon$ ~ $N(0, \sigma^2I_{n}$) now, let $y_{i}$ = $\hat{y_{i}}$ for some ...
0
votes
1answer
39 views

Math notation clarification

I'm working on learning more about logistic regression and I came across an equation with some confusing notation that I've never seen before: $$ \frac{\delta}{\delta \theta_{y'}^{(j)}} l(\theta) = ...
0
votes
0answers
14 views

minimum orders in linear regression to get a perfect fit

The problem is that, $(X_i, Y_i), i = 1,\ldots, n$ is an i.i.d. bivariate sample. Show that it is possible to fit a polynomial model using least square such that the fitted values are equal to the ...
1
vote
1answer
44 views

Significance level for a hypothesis test for a linear regression

Consider linear regression model $Y_i=a+b\cdot x_i+\epsilon_i$, $i=1,2,3,4,5$, where $a,b\in\mathbb{R}$ are unknown and $x_1=x_2=1,x_3=3,x_4=x_5=5$, $\epsilon_i$ are iid, normally distributed with ...
1
vote
1answer
12 views

contour plot in multiple linear regression

I have recently saw some examples about contour plots and multiple linear regression, for what I know a countour plot is obtained for having a graphical view of how the weights in a linear regression ...
3
votes
1answer
84 views

Find x,y & z (xyz+xyz=zyx)

I saw this problem the other day at work and found it pretty interesting: $$xyz + xyz = zyx$$ Find $x, y, z$ and the base(s) which this is true. Note that $x,y,z$ are simply digits concatenated, ...
0
votes
0answers
25 views

Question about ridge estimator

I have tried to show that ridge estimator is the solution to following problem min $(\beta- \hat{\beta})^t$$X^t X$$(\beta- \hat{\beta})$ subject to $\beta^t \beta =< d^2$ and $\beta$ is a $p$ x 1 ...
1
vote
1answer
29 views

Comparison of parameter: two different populations

I was wondering what the best way is to check for the equality of two parameters for a regression with no constant including possibly a confidence interval and p-value. $$H_0:\beta_1=\beta_2\ \vert\ ...
-1
votes
0answers
30 views

Curve Fitting - When to use Interpolation and/or Best-Fit?

This is my first time posting on Mathematics Stack. Nice to meet you all. I have a question regarding curve fitting, interpolation, and best-fit approximation. My supervisor wants me to write a ...
0
votes
0answers
24 views

two way ANOVA and linear regression model.

I know that Analysis of variance model can be written as a linear regression model using indicator regressors. For, one way ANOVA, I can write down the regression model. But for two way ANOVA ...
0
votes
0answers
16 views

Intuition for “weights” in simple linear regression [migrated]

Suppose we have data $\{x_i,y_i\}_{i=1}^n$ where $x_i \in \mathbb{R}$ and $y_i \in \mathbb{R}$ and we model $$ y_i=\beta x_i + \varepsilon_i $$ The ordinary least squares estimate of $\beta$ is $$ ...
0
votes
1answer
34 views

Power form of regression equation which is not centered at x=0?

For a given set of data, the power form of the regression equation is given by $$y=b\cdot x^{m}$$ where $$m=\frac{n(\sum \mathrm{ln}(x_i)\mathrm{ln}(y_i))-(\sum \mathrm{ln}(x_i))(\sum ...
0
votes
1answer
16 views

standard error for the parameters of a linear regression model

Given a linear model $\mathbf{y} = \beta \mathbf{X} + \epsilon$, it is well known that the estimate for $\beta$ that gives the minimum residual sum of squares (RSS) is given by $\hat{\beta} = ...
0
votes
1answer
15 views

How to report significant digits in coefficient of determination?

Say that I fit some data with some model, for instance a linear function $y = mx+b$. What is the proper way to report the fitted coefficients and the goodness of fit? Specifically, if I do the fit in ...
1
vote
1answer
15 views

Interpreting linear regression.

I'm not very versed in statistics or anything so I'm in the dark for this. For my biology (Grade 12) class I've been looking at journals and papers and I've seen a lot of graphs expressed in the form ...
1
vote
0answers
26 views

Odd Ratio and Logistic Regression

Windy and Play Tennis = 9 Windy and Not Playing Tennis = 8 Not Windy and Play Tennis = 14 Not Windy and Not Playing Tennis = 8 I performed logistic regression in Weka and got odd ratio as 0.3448 for ...
0
votes
0answers
25 views

Chi Square Formula and Degrees of Freedom Questions

I have a population sample with 200 points of data and 3 degrees of freedom so am I supposed to do a chi square formula with all 200 points of data? I believe that is what I'm supposed to do but I'm ...
2
votes
1answer
39 views

Is it possible to have two lines of best fit?

Could you rig a data set to have two lines of equally good (and best) fit? Or is it impossible?
1
vote
0answers
24 views

Increase the probability of correct prediction using multiple regression

First off let me begin by saying that I'm brand new to statistics and I would appreciate it if you could dumb down any answers for my problem. I am trying to create a general prediction of how much a ...
0
votes
0answers
11 views

How do I get the proper p-value of a time series average of regression coefficients?

I have run cross-sectional regression on the returns of 50 companies on 16 regressors, for 128 days. The regression output looks something like this: ...
0
votes
2answers
26 views

How to find more than two coefficient for single variable nonlinear equation?

I don't have good knowledge on mathematics, but now I faced one problem with maths. That is, I have a data set which contains only one independent and one dependent variable. Now I have a equation ...
0
votes
0answers
19 views

Logistic regression - How to interpret a graph?

Can somebody help me to understand these two graphs. I don't know how to interpret them correctly. Thanks!
1
vote
0answers
32 views

Linear regression of time series data - moving linear regression

Situation A suitable analogy for my real-world problem would be a shop - customers arrive, spend a random amount of time in the shop and leave. The arrival behaviour of customers follows a Poisson ...
0
votes
0answers
8 views

Difference Between Three Similar Error Reducing Algorithms

I found a Least Square Error Recognition algorithm that finds the least mean square error from a 2-d matrix element by element. Logistic regression from this site, on the other hand, seeks to ...
0
votes
0answers
20 views

L1 regression statistics

Consider fitting the below dataset using L1 regression: x: 8.3 8.3 12.1 12.1 17.0 17.0 17.0 24.3 24.3 24.3 33.6 y: 224 312 362 521 640 539 728 945 738 759 663 Why do the regression estimates ...
0
votes
0answers
71 views

How do I find Sxx in a Simple linear regression model?

In a Simple linear regression model, I have only Sxy and Syy data with me. How shall I derive Sxx, linking Sxy and Syy based on first principles? I know the formulas separately. I want to find Sxx, ...
0
votes
0answers
25 views

Classical Regression Model: Combining linearity and strict exogeneity

I am studying the Classical Regression Model for random samples. Hence consider the random sample $(y_i,\mathbf{x_i})$ Where: ...
0
votes
0answers
13 views

How to derive F statistic for general linear hypothesis

I want to derive the F statistic for general linear hypothesis $H_0$ : T$\beta$ = c vs $H_1$ : T$\beta$ $\not =$ c where T is $p * q$ matrix with rank q I have tried to express $SS_{Res} (RM)$ - ...
0
votes
1answer
18 views

how to apply weighting factor to linear regression

Say if I have two sets of data, x and y. And I am required to apply a weighting factor,1/x, to the regression line. Does that mean I should plot 1/y versus 1/x and then get the regression? Could ...
0
votes
0answers
19 views

Difference in coefficient estimates

I have simple regression $y_i=\beta_0+\beta_1x+\epsilon$. Then I have estimates $\hat{\beta_0},\hat{\beta_1}$ computed from n observation and estimates $\beta_0',\beta_1'$ computed from n+1 ...
0
votes
1answer
19 views

Questions about Multi linear regression model.

I have two questions about multi linear regression model. First question. Suppose 2 independent samples Sample1 : $y_1$, ... $y_{n_1}$ and $x_1$, ..., $x_{n_1}$ Sample2 : $y_{n_1 +1}$, ... $y_{n_1 ...
0
votes
0answers
11 views

relation within Gauss-Newton method for minimization

If we study model fit on a nonlinear regression model $Y_i=f(z_i,\theta)+\epsilon_i$, $i=1,...,n$, and in the Gauss-Newton method, the update on the parameter $\theta$ from step $t$ to $t+1$ is to ...
1
vote
1answer
76 views

Using the observation vector $ \vec{y}$ instead of the centered observation vector $ \vec{y_{d}} $ doesn't change the projection $\vec{\hat y}$

I'm wondering why the two statements below are equal regardless of using $\vec{y}$ in deviation form/mean-deviaton/centered form or not. In other words, why isn't the result changed when you use the ...
0
votes
0answers
20 views

Poisson distribution with normal informative priors

I'm Jia, a student of economics and finance. I was wondering if someone could help in understanding this problem. I've just started to attend a new course "Financial and nonlinear econometrics" and ...
0
votes
0answers
39 views

$\left(\sum_{i=1}^n1\right)\left(\sum_{i=1}^n x_i^2\right)-\left(\sum_{i=1}^n x_i\right)^2=\frac{1}{2}\sum_{i=1}^n\sum_{k=1}^n\left(x_i-x_j\right)^2$?

I am reading Widder's Advanced Calculus and on page 130 he states that \begin{align}\left(\sum_{i=1}^n 1\right)\left(\sum_{i=1}^n x_i^2\right)-\left(\sum_{i=1}^n ...
1
vote
1answer
19 views

An algebraic identity: $M(X_2)-M(X)=P(M(X_2)X_1)$?

Notation: for a matrix $Z$ of full column rank, we define $P(Z)=Z(Z'Z)^{-1}Z'$ and $M(Z)=I-P(Z)$. ($I$ is the identity of matrix with as many rows as $Z$.) Let's consider the linear regression model ...
0
votes
1answer
27 views

Advice to solve a system of 8th order univariate polynomials

I am struggling to solve a least square problem in which the tedious part is the initialization. Grid search methods are out of question. The initial problem I've stated my problem in a previous ...
0
votes
0answers
16 views

Fit derivative to a set of points

Let's say I have a set of discrete values $X = {x_1, x_2, x_3, ..., x_n}$ from the sampling at a rate $f_s$ of a continuous function. I scale some values in $X$ (in a different manner for each one), ...