Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
13 views

I would like to know how to do log transformation of hyperparameters in Gaussian Process Classification.

I am using Gaussian Process classification and I want to do log transform of the hyperparameters so that they are all positive. From this www.lce.hut.fi/research/mm/gpstuff/GPstuffDoc.pdf document, I ...
-1
votes
0answers
11 views

Find a way to regression between two data which are NOT paralleled

I have variables X and Y need to be regressed,shown in the following screenshot. I need to apply regression between X and Y. My problem is how to organize data to make the regression. For ...
0
votes
0answers
40 views

Solving constrained linear programming problem

For the variable $t$, problem is to find best multipliers $k$ which minimizes the objective function. Time: $t_1$, $t_2$, $t_3$,... given in input Multiplier $k_1$, $k_2$, $k_3$,... (These are ...
0
votes
0answers
15 views

If the null hypothesis is true, how will the test statistic be distributed?

I went with T~(50-6) The question goes.... "A regression is estimated with 50 observations, five explanatory variables and with a constant. Suppose You want to test the following hypothesis $H_0: ...
0
votes
1answer
23 views

If $\ln(y) = 5 - 0.1X $what is the elasticity of $Y$ with respect to $X$, when $X=10$?

So i got the following model $\ln(y) = 5 - 0.1* X$ The elasticity of Y with respect to X, when $X=10$ i said -0.1 but apparently i'm wrong Isn't the coefficient of X the elasticity of y when the ...
1
vote
1answer
13 views

Class or type variables as features in polynomial regression algrorithm

I am new in machine learning area, and trying to use polynomial regression for my problem. I have data - advertisements of the cars from kolesa.kz website. Data contains mark, model, mileage, engine ...
0
votes
1answer
11 views

Linear or Non-Linear Model

I have the following regression equation \begin{align*} y_i = \alpha + \gamma\cdot\beta\cdot x_i+ \varepsilon_i, \end{align*} where $y_i$, $x_i$ and $\varepsilon_i$ are $n\times 1$ vectors, ...
0
votes
0answers
7 views

Where can I find formulas for the multiclass logistic regression with bias term?

In most of the books and web sources and papers the multiclass logistic regression is introduced and discussed without bias terms. I am looking for generalised formulas using bias terms. The standard ...
0
votes
1answer
30 views

Confidence Interval for Nonlinear Regression using F-Test - lmfit

I am trying to understand the implementation for the lmfit confidence interval calculation - in the docs it is stated: "The F-test is used to compare our null model, which is the best fit we have ...
0
votes
1answer
26 views

Pearson coefficient and rate of change [on hold]

Given that two series $(x_1,....x_n)$ and $(y_1,....y_n)$ are linearly correlated How can I measure the change in the increment ($i\leq j$ ) $y_j−y_i$ as a function of $x_j−x_i$ , Pearson's r and ...
0
votes
2answers
34 views

Rational function regression without poles in a interval, or polynomial regression with positivity constraint

I have some sets of experimental data for some functions $f_i$ from $I=[0,1]$ onto itself, which should satisfy the following physical constraints: $f_i(0)=1$ $f_i(x) \in I= [0,1] \; \forall x \in I ...
1
vote
1answer
19 views

Gauss-Markov Theorem: How can you show that $\Lambda^T (X^TX)^gX^TX(X^TX)^{g^T} \Lambda$ = $\Lambda^T(X^TX)^g\Lambda$?

I'm stuck on how to go from the first line to the second line in this equation related to the Gauss-Markov model where $\mathbf{y}=X\mathbf{b}+\mathbf{e}$, $E(\mathbf{e})=0$, and ...
2
votes
1answer
27 views

Prove a result in multiple linear regression

This arises in multiple linear regression. Given $m, n \in \mathbb{N}$ and matrices $X \in \mathbb{R}^{m \times (n+1)} (m > n + 1), H = X(X'X)^{-1}X' \in \mathbb{R}^{m\times m}, I = I_m$ and $J ...
2
votes
1answer
83 views

Prove that $E(\mathbf{u}|\mathbf{X})=\mathbf{0}$ implies $Cov(\mathbf{x},\mathbf{u})=\mathbf{0}$

Let \begin{equation} \mathbf{y}=\mathbf{X}\mathbf{\beta}+\mathbf{u} \end{equation} where $\mathbf{y}=\begin{bmatrix}y_1 \\ \vdots \\ y_n\end{bmatrix}$, $\mathbf{X}=\begin{bmatrix}X_{11} & ...
2
votes
1answer
11 views

Problem with verifying variance of residual

I am supposed to show the following: $$Var(e_{ij}) = \sigma^{2}\left(1-\frac{1}{n_i}\right)$$ Where the follwing is known: $$y_{ij} = \mu + \alpha_{i} + \varepsilon_{ij}$$ $$e_{ij} = y_{ij} - ...
1
vote
0answers
10 views

How is genetic programming used in Symbolic regression

I am in highschool and have not taken any courses on this. Rather I am working on an outside project. I don't quite understand how Genetic Programming could be used effectively to generate a set of ...
0
votes
0answers
13 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!
0
votes
1answer
28 views

Show ARIMA(1,1) with mean $\mu$ is an ARMA process

I am trying to show that an ARIMA(1,1) process with mean $\mu$ is an ARMA process, as well as to show if it causal and/or invertible. The set up is: Let $X_t$ be a causal and invertible ARMA(1,1) ...
0
votes
1answer
22 views

Linear regression custom fit function, calculate A and B using system of linear equations

Good afternoon! As a part of solved examples from previous year examination, there is a following bi-dimensional table of frequencies: ...
0
votes
0answers
28 views

Mathematical equivalent to curve fit between polynomials

I am adapting a calculation done in an Excel workbook to code. Right now, we are predicting a variable based on three other variables, say $x,y,z$. We are creating six functions of $x$ and $y$ at ...
1
vote
1answer
22 views

Linear Regression quadratic terms

I have a hard time understanding the term 'linear regression'. For what I know, linear means polynomial of degree 1. But then, I found that in one of my lectures, the lecturers are saying that this ...
-1
votes
1answer
18 views

Using least squares regression for line of best fit

Use the least square approximation to find the closest line (the line of "Best Fit") to the points: $$(-6,-1), \quad (-2,2), \quad (1,1), \quad (7,6)$$ I'm attempting to use the least squares ...
0
votes
1answer
65 views

What's the most efficient way to fit a surface to three or more points?

Say I have a function of the form $s=b-mp+at$, where $p$ and $t$ are the independent variables, and I have 3 or more points of the form $(p,t,s)$. I want to find the best values for $b$, $m$, and $a$ ...
1
vote
2answers
1k views

Log-likelihood gradient and Hessian

Considering a binary classification problem with data $D = \{(x_i,y_i)\}_{i=1}^n$, $x_i \in \mathbb{R}^d$ and $y_i \in \{0,1\}$. Given the following definitions: $f(x) = x^T \beta$ $p(x) = ...
0
votes
0answers
15 views

Correlation/Regression for Continuous and Discrete data

I want to correlate a data where one axis is continuous (ranging from 0 to 1), other axis is discrete. Discrete axis scale is 1 to 5 (1 is for Strongly Disagree and 5 is for Strongly agree). How ...
0
votes
1answer
29 views

How to find Parameters in nonlinear Regression Model?

I have a nonlinear Regression Model with eleven observations of $x,y$. How do I find the parameters $a,b,c,d$ of the model: $ y=f(x)=a + b \sin cx e^{dx}$ by using the function: $$\Phi(a, b, c, ...
-1
votes
0answers
17 views

X correlates with Y and Y correlates with Z when p-values are known

"How can I calculate the range of correlation of two variables X and Z given I have the correlations of X and Y, and Y and Z?" These are useful resources: ...
0
votes
0answers
10 views

Linear regression with rounded down dependable variable.

I have a problem where I need to find the underlying linear relationship between an independent variable and it's dependent variable. However, I know that the dependent variable is being rounded down ...
0
votes
1answer
20 views

Does scatterplot matrix “work” with quadratic variables?

basically I want to plot a scatterplot matrix using a few variables. For simplicity lets say my model is: $$z=\alpha_0 + \alpha_1w+\alpha_2x+\alpha_3y+\alpha_4y^2 + \epsilon$$ When I plot the matrix, ...
1
vote
0answers
43 views

Why do the components of an equivalent kernel sum to 1?

Let $\textbf{x} = (x_1, \dots, x_n)^T \in \mathbb{R}^n$ and $k \in \mathbb{N}$. We define $$ X := \begin{pmatrix} 1 & x_1 & \cdots & x_1^k \\ \vdots & \vdots & & ...
0
votes
2answers
310 views

Extrapolation with exponential curve

I would like to extrapolate time series using exponential curve while getting the parameters via linear regression. Exponential curve is given as $g=e^{~a + b \cdot t}$. Since I want to use linear ...
0
votes
0answers
26 views

Statistical Multiple Linear Regression Log Transformation

If for example we have a multiple linear regression as follows: $$hydrcarb=x_1+x_2tanktemp+x_3disptemp+x_4tankpres+x_5disppres+x_6tankpres^2+x_7dispres^2$$ And I am trying to do a backward ...
1
vote
0answers
14 views

Approximation technique when data is missing?

I am doing some statistical studies and I would appreciate some guidance to some approximation techniques when not all data is available. I have a model that takes certain input parameters (discrete, ...
0
votes
1answer
23 views

X - axis of a linearized polynomial.

The other day in my Physics class we had some sample data that we wanted to linearize. The graph resembled a root curve. So to linearize it, we took the square root of all the x data and replotted ...
0
votes
2answers
30 views

How we can linearize this equation?

I have an equation that it seems to be a non-linear equation. I want to compute the parameters a1 till a4.I want to simply do a linear regression to find the parameters, which is much easier than a ...
0
votes
1answer
14 views

Regression project in octave/matlab

I'm trying to establish a polynomial model to adjust the variation of the dollar throughout the year. Suppose hypothetically that I have the following data ...
0
votes
0answers
18 views

The correlation between alpha and beta

Consider the following 2-variable linear regression where error $e_i$'s are independently and identically distributed with mean 0 and variance 1; $$ y_i=\alpha + \beta (x_i - \bar {x}) + e_i$$ where ...
1
vote
0answers
24 views

Does gradient descent and normal equation give the same answer?

I tried to optimize for a linear regression model using both approaches and they gave me two completely different answers. My sample data set was: df <- data.frame(c(1,5,6),c(3,5,6),c(4,6,8)) ...
0
votes
0answers
10 views

Is it always possible to find a logistic regression model that yields zero training error on any dataset?

I am leaning towards no. A logit regression model is just one function, and there is no way its coefficients can accurately predict an entire dataset, outliers and all. Is this the correct intuition?
1
vote
1answer
321 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
0
votes
1answer
17 views

Maximum and minimum penalty in lasso regression family

I am trying to adjust penalty, lambda, in group lasso regression, but I have no idea about it. Just to clarify, group lasso regression is a kind of multiple linear regression which use penalties on ...
1
vote
1answer
35 views

finding column vectors - linear transformations

$L:\mathbb{R}^3\rightarrow \mathbb{R}^2$ with bases $\mathcal{S}=\left\{\left(-1,1,0\right),\left(0,1,1\right),\left(1,0,0\right)\right\} \: \text{for} \:\mathbb{R}^3 \:\text{and} \\ ...
2
votes
1answer
308 views

Understanding Regularization parameters in Machine Learning/Statistics

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...
0
votes
0answers
34 views

Why use the expected value of $y$, $E(y)$, in simple linear regression.

I am learning about linear regression and I have ran into a bit of confusion. I'm trying to relate what I've learned in my probability and mathematical statistics course (in particular, expected ...
1
vote
1answer
47 views

Significance level for a hypothesis test for a linear regression

Consider linear regression model $Y_i=a+b\cdot x_i+\epsilon_i$, $i=1,2,3,4,5$, where $a,b\in\mathbb{R}$ are unknown and $x_1=x_2=1,x_3=3,x_4=x_5=5$, $\epsilon_i$ are iid, normally distributed with ...
0
votes
0answers
29 views

OLS: Estimation with negative coefficients

I have probably an easy problem, however I'm not really sure how to do it: Basically, I would like to estimate a linear regression with OLS. So far so easy. However, the model that I would like to ...
0
votes
0answers
32 views

Intuitive understanding about LSE

Let y= X$\beta$ + $\epsilon$ where y is $n \times 1$ vector, X is $n \times p+1$ matrix, $\beta$ is $p+1 \times 1$ vector, $\epsilon$ ~ $N(0, \sigma^2I_{n}$) now, let $y_{i}$ = $\hat{y_{i}}$ for some ...
0
votes
1answer
39 views

Math notation clarification

I'm working on learning more about logistic regression and I came across an equation with some confusing notation that I've never seen before: $$ \frac{\delta}{\delta \theta_{y'}^{(j)}} l(\theta) = ...
2
votes
1answer
309 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: $y$=value, $x_1$=material, $x_2$=weight, $x_3$=color $x_1$ and $x_2$ are interacting variables but $x_3$ ...
0
votes
0answers
22 views

minimum orders in linear regression to get a perfect fit

The problem is that, $(X_i, Y_i), i = 1,\ldots, n$ is an i.i.d. bivariate sample. Show that it is possible to fit a polynomial model using least square such that the fitted values are equal to the ...