Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
47 views

Identical observations in linear regression

I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
0
votes
1answer
21 views

Method for ?not quite? weighted least squares fitting for more realistic results

I need a linear least squares type of fitting algorithm that understands how to weight the probability of a response coming from certain functions over another. To explain, given the standard linear ...
1
vote
1answer
15 views

Assumption of Normal Distribution

I have a problem and I do not know when it is crucial and when it is NOT crucial to assume a normal distribution regarding linear regression, for estimates, t-tests, f-tests, confidence intervals and ...
1
vote
1answer
36 views

Best percentage change for trend

Consider the revenue of a company for the last five year and you want to to know whether there is an upward, downward or no trend. How would you calculate the "optimal" percentage change? I have an ...
1
vote
1answer
28 views

Is there such a thing as a weighted multiple regression?

I'm new to linear algebra, but I know how multiple linear regressions work. What I want to do is something slightly different. As an example, let's say that I have a list of nutrients I want to get ...
1
vote
0answers
35 views

Confusion with Bayesian Linear Regression

In the book Gaussian Processes for Machine Learning in Chapter 2 p. 11 (see http://www.gaussianprocess.org/gpml/chapters/RW2.pdf), eq. 2.9 states: $p(f_* | X, y) = \int p(f_* | x_*,w) p(w|X, y)dw$ ...
2
votes
1answer
40 views

Calculate trend and represent in text?

First off, I'm terrible at math. I'm writing a script that monitors transactions from clients daily over a 7 day period. Given a set of numbers like below, I would like to calculate a trend and ...
0
votes
0answers
22 views

Relation between the Coefficient of Multiple Correlation and Coefficient of Simple Correlation

Consider the regression model $Y=\beta_1 X_1+\beta_2 X_2+\epsilon$, with a sample of size $n$, $Y_i=\beta_1 X_{i1}+\beta_2 X_{i2}+\epsilon_i$, $\epsilon_i \in N(0,\sigma^2)$. Suppossing ...
1
vote
1answer
23 views

Logistic regression coefficients problem

I'm using logistic regression model to do a multi-class classification (4 classes). I want to look at the logistic regression coefficients to see the importance of different features. I got model ...
0
votes
1answer
28 views

Distribution of e, least squares residuals

I have the model $y=X{\beta}+{\epsilon}$ and $E({\epsilon})=0$ and $Var({\epsilon})={\sigma}^2I_n$ The vector y can be written: $y=X\hat{\beta}+{e}$ If ${\epsilon}$~$N(0, {\sigma}^2I_n)$ how is ...
0
votes
1answer
29 views

Quadratic Form Matrices

How do I know if a matrix in quadratic form, e.g. D'MD is positive or negative (semi)definite? M here is the residual maker matrix for X, so I know that it is symmetric. I know what the definitions ...
2
votes
0answers
28 views

Show $\hat{\beta}$ and $s^2$ are independent?

I have the model: $y=X{\beta}+{\epsilon}$ I know $\hat{\beta}=(X'X)^{-1}X'y$ and that it is an unbiased estimator of ${\beta}$ and that $s^2=\hat{\epsilon}'\hat{\epsilon}/(n-k)$ and is an unbiased ...
0
votes
1answer
35 views

Multiple linear regression inconsistency?

I've got a linear model: $y_i=β_1x_{i1}+β_2x_{i2}+ε_i$ where E($ε_i$)=0 and Var($ε_i$)= $σ^2I_n$ for i=1,...,n Supposed we don't have the data for $x_{i2}$ and we estimate: $y_i=β_1x_{i1}+ε_i$ for ...
1
vote
0answers
48 views

Minimize correlation between input and output of a linear system

I am not sure if "minimize correlation" is the right title for this issue but I could not find a better sentence to describe what I would like to achieve. Let's say that I have a black box with ...
1
vote
1answer
29 views

I would like to know how to do log transformation of hyperparameters in Gaussian Process Classification.

I am using Gaussian Process classification and I want to do log transform of the hyperparameters so that they are all positive. From this www.lce.hut.fi/research/mm/gpstuff/GPstuffDoc.pdf document, I ...
0
votes
0answers
64 views

Solving constrained linear programming problem

For the variable $t$, problem is to find best multipliers $k$ which minimizes the objective function. Time: $t_1$, $t_2$, $t_3$,... given in input Multiplier $k_1$, $k_2$, $k_3$,... (These are ...
0
votes
0answers
16 views

If the null hypothesis is true, how will the test statistic be distributed?

I went with T~(50-6) The question goes.... "A regression is estimated with 50 observations, five explanatory variables and with a constant. Suppose You want to test the following hypothesis $H_0: ...
0
votes
1answer
27 views

If $\ln(y) = 5 - 0.1X $what is the elasticity of $Y$ with respect to $X$, when $X=10$?

So i got the following model $\ln(y) = 5 - 0.1* X$ The elasticity of Y with respect to X, when $X=10$ i said -0.1 but apparently i'm wrong Isn't the coefficient of X the elasticity of y when the ...
1
vote
1answer
15 views

Class or type variables as features in polynomial regression algrorithm

I am new in machine learning area, and trying to use polynomial regression for my problem. I have data - advertisements of the cars from kolesa.kz website. Data contains mark, model, mileage, engine ...
1
vote
1answer
12 views

Linear or Non-Linear Model

I have the following regression equation \begin{align*} y_i = \alpha + \gamma\cdot\beta\cdot x_i+ \varepsilon_i, \end{align*} where $y_i$, $x_i$ and $\varepsilon_i$ are $n\times 1$ vectors, ...
0
votes
0answers
12 views

Where can I find formulas for the multiclass logistic regression with bias term?

In most of the books and web sources and papers the multiclass logistic regression is introduced and discussed without bias terms. I am looking for generalised formulas using bias terms. The standard ...
0
votes
1answer
56 views

Confidence Interval for Nonlinear Regression using F-Test - lmfit

I am trying to understand the implementation for the lmfit confidence interval calculation - in the docs it is stated: "The F-test is used to compare our null model, which is the best fit we have ...
1
vote
1answer
33 views

Gauss-Markov Theorem: How can you show that $\Lambda^T (X^TX)^gX^TX(X^TX)^{g^T} \Lambda$ = $\Lambda^T(X^TX)^g\Lambda$?

I'm stuck on how to go from the first line to the second line in this equation related to the Gauss-Markov model where $\mathbf{y}=X\mathbf{b}+\mathbf{e}$, $E(\mathbf{e})=0$, and ...
0
votes
2answers
49 views

Rational function regression without poles in a interval, or polynomial regression with positivity constraint

I have some sets of experimental data for some functions $f_i$ from $I=[0,1]$ onto itself, which should satisfy the following physical constraints: $f_i(0)=1$ $f_i(x) \in I= [0,1] \; \forall x \in I ...
2
votes
1answer
29 views

Prove a result in multiple linear regression

This arises in multiple linear regression. Given $m, n \in \mathbb{N}$ and matrices $X \in \mathbb{R}^{m \times (n+1)} (m > n + 1), H = X(X'X)^{-1}X' \in \mathbb{R}^{m\times m}, I = I_m$ and $J ...
1
vote
0answers
12 views

How is genetic programming used in Symbolic regression

I am in highschool and have not taken any courses on this. Rather I am working on an outside project. I don't quite understand how Genetic Programming could be used effectively to generate a set of ...
0
votes
0answers
25 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!
2
votes
1answer
13 views

Problem with verifying variance of residual

I am supposed to show the following: $$Var(e_{ij}) = \sigma^{2}\left(1-\frac{1}{n_i}\right)$$ Where the follwing is known: $$y_{ij} = \mu + \alpha_{i} + \varepsilon_{ij}$$ $$e_{ij} = y_{ij} - ...
0
votes
1answer
25 views

Linear regression custom fit function, calculate A and B using system of linear equations

Good afternoon! As a part of solved examples from previous year examination, there is a following bi-dimensional table of frequencies: ...
1
vote
1answer
25 views

Linear Regression quadratic terms

I have a hard time understanding the term 'linear regression'. For what I know, linear means polynomial of degree 1. But then, I found that in one of my lectures, the lecturers are saying that this ...
-1
votes
1answer
25 views

Using least squares regression for line of best fit

Use the least square approximation to find the closest line (the line of "Best Fit") to the points: $$(-6,-1), \quad (-2,2), \quad (1,1), \quad (7,6)$$ I'm attempting to use the least squares ...
0
votes
1answer
68 views

What's the most efficient way to fit a surface to three or more points?

Say I have a function of the form $s=b-mp+at$, where $p$ and $t$ are the independent variables, and I have 3 or more points of the form $(p,t,s)$. I want to find the best values for $b$, $m$, and $a$ ...
0
votes
1answer
49 views

How to find Parameters in nonlinear Regression Model?

I have a nonlinear Regression Model with eleven observations of $x,y$. How do I find the parameters $a,b,c,d$ of the model: $ y=f(x)=a + b \sin cx e^{dx}$ by using the function: $$\Phi(a, b, c, ...
0
votes
0answers
23 views

Correlation/Regression for Continuous and Discrete data

I want to correlate a data where one axis is continuous (ranging from 0 to 1), other axis is discrete. Discrete axis scale is 1 to 5 (1 is for Strongly Disagree and 5 is for Strongly agree). How ...
0
votes
0answers
37 views

Mathematical equivalent to curve fit between polynomials

I am adapting a calculation done in an Excel workbook to code. Right now, we are predicting a variable based on three other variables, say $x,y,z$. We are creating six functions of $x$ and $y$ at ...
0
votes
0answers
16 views

Linear regression with rounded down dependable variable.

I have a problem where I need to find the underlying linear relationship between an independent variable and it's dependent variable. However, I know that the dependent variable is being rounded down ...
0
votes
1answer
22 views

Does scatterplot matrix “work” with quadratic variables?

basically I want to plot a scatterplot matrix using a few variables. For simplicity lets say my model is: $$z=\alpha_0 + \alpha_1w+\alpha_2x+\alpha_3y+\alpha_4y^2 + \epsilon$$ When I plot the matrix, ...
1
vote
1answer
66 views

Why do the components of an equivalent kernel sum to 1?

Let $\textbf{x} = (x_1, \dots, x_n)^T \in \mathbb{R}^n$ and $k \in \mathbb{N}$. We define $$ X := \begin{pmatrix} 1 & x_1 & \cdots & x_1^k \\ \vdots & \vdots & & ...
1
vote
0answers
15 views

Approximation technique when data is missing?

I am doing some statistical studies and I would appreciate some guidance to some approximation techniques when not all data is available. I have a model that takes certain input parameters (discrete, ...
0
votes
0answers
32 views

Statistical Multiple Linear Regression Log Transformation

If for example we have a multiple linear regression as follows: $$hydrcarb=x_1+x_2tanktemp+x_3disptemp+x_4tankpres+x_5disppres+x_6tankpres^2+x_7dispres^2$$ And I am trying to do a backward ...
0
votes
2answers
34 views

How we can linearize this equation?

I have an equation that it seems to be a non-linear equation. I want to compute the parameters a1 till a4.I want to simply do a linear regression to find the parameters, which is much easier than a ...
0
votes
1answer
31 views

X - axis of a linearized polynomial.

The other day in my Physics class we had some sample data that we wanted to linearize. The graph resembled a root curve. So to linearize it, we took the square root of all the x data and replotted ...
0
votes
1answer
21 views

Regression project in octave/matlab

I'm trying to establish a polynomial model to adjust the variation of the dollar throughout the year. Suppose hypothetically that I have the following data ...
0
votes
0answers
22 views

The correlation between alpha and beta

Consider the following 2-variable linear regression where error $e_i$'s are independently and identically distributed with mean 0 and variance 1; $$ y_i=\alpha + \beta (x_i - \bar {x}) + e_i$$ where ...
1
vote
0answers
31 views

Does gradient descent and normal equation give the same answer?

I tried to optimize for a linear regression model using both approaches and they gave me two completely different answers. My sample data set was: df <- data.frame(c(1,5,6),c(3,5,6),c(4,6,8)) ...
0
votes
0answers
10 views

Is it always possible to find a logistic regression model that yields zero training error on any dataset?

I am leaning towards no. A logit regression model is just one function, and there is no way its coefficients can accurately predict an entire dataset, outliers and all. Is this the correct intuition?
0
votes
1answer
26 views

Maximum and minimum penalty in lasso regression family

I am trying to adjust penalty, lambda, in group lasso regression, but I have no idea about it. Just to clarify, group lasso regression is a kind of multiple linear regression which use penalties on ...
1
vote
1answer
39 views

finding column vectors - linear transformations

$L:\mathbb{R}^3\rightarrow \mathbb{R}^2$ with bases $\mathcal{S}=\left\{\left(-1,1,0\right),\left(0,1,1\right),\left(1,0,0\right)\right\} \: \text{for} \:\mathbb{R}^3 \:\text{and} \\ ...
0
votes
1answer
34 views

Show ARIMA(1,1) with mean $\mu$ is an ARMA process

I am trying to show that an ARIMA(1,1) process with mean $\mu$ is an ARMA process, as well as to show if it causal and/or invertible. The set up is: Let $X_t$ be a causal and invertible ARMA(1,1) ...
0
votes
0answers
31 views

OLS: Estimation with negative coefficients

I have probably an easy problem, however I'm not really sure how to do it: Basically, I would like to estimate a linear regression with OLS. So far so easy. However, the model that I would like to ...