Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

-1
votes
0answers
8 views

Find a way to regression between two data which are NOT paralleled

I have variables X and Y need to be regressed,shown in the following screenshot. I need to apply regression between X and Y. My problem is how to organize data to make the regression. For ...
0
votes
0answers
39 views

Solving constrained linear programming problem

For the variable $t$, problem is to find best multipliers $k$ which minimizes the objective function. Time: $t_1$, $t_2$, $t_3$,... given in input Multiplier $k_1$, $k_2$, $k_3$,... (These are ...
0
votes
0answers
15 views

If the null hypothesis is true, how will the test statistic be distributed?

I went with T~(50-6) The question goes.... "A regression is estimated with 50 observations, five explanatory variables and with a constant. Suppose You want to test the following hypothesis $H_0: ...
0
votes
1answer
23 views

If $\ln(y) = 5 - 0.1X $what is the elasticity of $Y$ with respect to $X$, when $X=10$?

So i got the following model $\ln(y) = 5 - 0.1* X$ The elasticity of Y with respect to X, when $X=10$ i said -0.1 but apparently i'm wrong Isn't the coefficient of X the elasticity of y when the ...
1
vote
1answer
13 views

Class or type variables as features in polynomial regression algrorithm

I am new in machine learning area, and trying to use polynomial regression for my problem. I have data - advertisements of the cars from kolesa.kz website. Data contains mark, model, mileage, engine ...
0
votes
1answer
10 views

Linear or Non-Linear Model

I have the following regression equation \begin{align*} y_i = \alpha + \gamma\cdot\beta\cdot x_i+ \varepsilon_i, \end{align*} where $y_i$, $x_i$ and $\varepsilon_i$ are $n\times 1$ vectors, ...
0
votes
0answers
7 views

Where can I find formulas for the multiclass logistic regression with bias term?

In most of the books and web sources and papers the multiclass logistic regression is introduced and discussed without bias terms. I am looking for generalised formulas using bias terms. The standard ...
0
votes
1answer
25 views

Pearson coefficient and rate of change [on hold]

Given that two series $(x_1,....x_n)$ and $(y_1,....y_n)$ are linearly correlated How can I measure the change in the increment ($i\leq j$ ) $y_j−y_i$ as a function of $x_j−x_i$ , Pearson's r and ...
0
votes
1answer
30 views

Confidence Interval for Nonlinear Regression using F-Test - lmfit

I am trying to understand the implementation for the lmfit confidence interval calculation - in the docs it is stated: "The F-test is used to compare our null model, which is the best fit we have ...
1
vote
1answer
19 views

Gauss-Markov Theorem: How can you show that $\Lambda^T (X^TX)^gX^TX(X^TX)^{g^T} \Lambda$ = $\Lambda^T(X^TX)^g\Lambda$?

I'm stuck on how to go from the first line to the second line in this equation related to the Gauss-Markov model where $\mathbf{y}=X\mathbf{b}+\mathbf{e}$, $E(\mathbf{e})=0$, and ...
0
votes
2answers
34 views

Rational function regression without poles in a interval, or polynomial regression with positivity constraint

I have some sets of experimental data for some functions $f_i$ from $I=[0,1]$ onto itself, which should satisfy the following physical constraints: $f_i(0)=1$ $f_i(x) \in I= [0,1] \; \forall x \in I ...
2
votes
1answer
27 views

Prove a result in multiple linear regression

This arises in multiple linear regression. Given $m, n \in \mathbb{N}$ and matrices $X \in \mathbb{R}^{m \times (n+1)} (m > n + 1), H = X(X'X)^{-1}X' \in \mathbb{R}^{m\times m}, I = I_m$ and $J ...
1
vote
0answers
10 views

How is genetic programming used in Symbolic regression

I am in highschool and have not taken any courses on this. Rather I am working on an outside project. I don't quite understand how Genetic Programming could be used effectively to generate a set of ...
0
votes
0answers
13 views

advantage and disadvantage of using SVD to solve least square problems

I usually just use $AA^T$ or QR decomposition of A to solve least square problems. But SVD seems to be the popular way to solve the problem. what is the advantage and disadvantage of SVD? thanks!
2
votes
1answer
11 views

Problem with verifying variance of residual

I am supposed to show the following: $$Var(e_{ij}) = \sigma^{2}\left(1-\frac{1}{n_i}\right)$$ Where the follwing is known: $$y_{ij} = \mu + \alpha_{i} + \varepsilon_{ij}$$ $$e_{ij} = y_{ij} - ...
0
votes
1answer
22 views

Linear regression custom fit function, calculate A and B using system of linear equations

Good afternoon! As a part of solved examples from previous year examination, there is a following bi-dimensional table of frequencies: ...
1
vote
1answer
22 views

Linear Regression quadratic terms

I have a hard time understanding the term 'linear regression'. For what I know, linear means polynomial of degree 1. But then, I found that in one of my lectures, the lecturers are saying that this ...
-1
votes
1answer
18 views

Using least squares regression for line of best fit

Use the least square approximation to find the closest line (the line of "Best Fit") to the points: $$(-6,-1), \quad (-2,2), \quad (1,1), \quad (7,6)$$ I'm attempting to use the least squares ...
0
votes
1answer
65 views

What's the most efficient way to fit a surface to three or more points?

Say I have a function of the form $s=b-mp+at$, where $p$ and $t$ are the independent variables, and I have 3 or more points of the form $(p,t,s)$. I want to find the best values for $b$, $m$, and $a$ ...
0
votes
1answer
29 views

How to find Parameters in nonlinear Regression Model?

I have a nonlinear Regression Model with eleven observations of $x,y$. How do I find the parameters $a,b,c,d$ of the model: $ y=f(x)=a + b \sin cx e^{dx}$ by using the function: $$\Phi(a, b, c, ...
0
votes
0answers
14 views

Correlation/Regression for Continuous and Discrete data

I want to correlate a data where one axis is continuous (ranging from 0 to 1), other axis is discrete. Discrete axis scale is 1 to 5 (1 is for Strongly Disagree and 5 is for Strongly agree). How ...
-1
votes
0answers
17 views

X correlates with Y and Y correlates with Z when p-values are known

"How can I calculate the range of correlation of two variables X and Z given I have the correlations of X and Y, and Y and Z?" These are useful resources: ...
0
votes
0answers
28 views

Mathematical equivalent to curve fit between polynomials

I am adapting a calculation done in an Excel workbook to code. Right now, we are predicting a variable based on three other variables, say $x,y,z$. We are creating six functions of $x$ and $y$ at ...
0
votes
0answers
10 views

Linear regression with rounded down dependable variable.

I have a problem where I need to find the underlying linear relationship between an independent variable and it's dependent variable. However, I know that the dependent variable is being rounded down ...
0
votes
1answer
20 views

Does scatterplot matrix “work” with quadratic variables?

basically I want to plot a scatterplot matrix using a few variables. For simplicity lets say my model is: $$z=\alpha_0 + \alpha_1w+\alpha_2x+\alpha_3y+\alpha_4y^2 + \epsilon$$ When I plot the matrix, ...
1
vote
0answers
43 views

Why do the components of an equivalent kernel sum to 1?

Let $\textbf{x} = (x_1, \dots, x_n)^T \in \mathbb{R}^n$ and $k \in \mathbb{N}$. We define $$ X := \begin{pmatrix} 1 & x_1 & \cdots & x_1^k \\ \vdots & \vdots & & ...
1
vote
0answers
14 views

Approximation technique when data is missing?

I am doing some statistical studies and I would appreciate some guidance to some approximation techniques when not all data is available. I have a model that takes certain input parameters (discrete, ...
0
votes
0answers
26 views

Statistical Multiple Linear Regression Log Transformation

If for example we have a multiple linear regression as follows: $$hydrcarb=x_1+x_2tanktemp+x_3disptemp+x_4tankpres+x_5disppres+x_6tankpres^2+x_7dispres^2$$ And I am trying to do a backward ...
0
votes
2answers
30 views

How we can linearize this equation?

I have an equation that it seems to be a non-linear equation. I want to compute the parameters a1 till a4.I want to simply do a linear regression to find the parameters, which is much easier than a ...
0
votes
1answer
23 views

X - axis of a linearized polynomial.

The other day in my Physics class we had some sample data that we wanted to linearize. The graph resembled a root curve. So to linearize it, we took the square root of all the x data and replotted ...
0
votes
1answer
14 views

Regression project in octave/matlab

I'm trying to establish a polynomial model to adjust the variation of the dollar throughout the year. Suppose hypothetically that I have the following data ...
0
votes
0answers
18 views

The correlation between alpha and beta

Consider the following 2-variable linear regression where error $e_i$'s are independently and identically distributed with mean 0 and variance 1; $$ y_i=\alpha + \beta (x_i - \bar {x}) + e_i$$ where ...
1
vote
0answers
24 views

Does gradient descent and normal equation give the same answer?

I tried to optimize for a linear regression model using both approaches and they gave me two completely different answers. My sample data set was: df <- data.frame(c(1,5,6),c(3,5,6),c(4,6,8)) ...
0
votes
0answers
10 views

Is it always possible to find a logistic regression model that yields zero training error on any dataset?

I am leaning towards no. A logit regression model is just one function, and there is no way its coefficients can accurately predict an entire dataset, outliers and all. Is this the correct intuition?
0
votes
1answer
17 views

Maximum and minimum penalty in lasso regression family

I am trying to adjust penalty, lambda, in group lasso regression, but I have no idea about it. Just to clarify, group lasso regression is a kind of multiple linear regression which use penalties on ...
1
vote
1answer
35 views

finding column vectors - linear transformations

$L:\mathbb{R}^3\rightarrow \mathbb{R}^2$ with bases $\mathcal{S}=\left\{\left(-1,1,0\right),\left(0,1,1\right),\left(1,0,0\right)\right\} \: \text{for} \:\mathbb{R}^3 \:\text{and} \\ ...
0
votes
1answer
27 views

Show ARIMA(1,1) with mean $\mu$ is an ARMA process

I am trying to show that an ARIMA(1,1) process with mean $\mu$ is an ARMA process, as well as to show if it causal and/or invertible. The set up is: Let $X_t$ be a causal and invertible ARMA(1,1) ...
0
votes
0answers
29 views

OLS: Estimation with negative coefficients

I have probably an easy problem, however I'm not really sure how to do it: Basically, I would like to estimate a linear regression with OLS. So far so easy. However, the model that I would like to ...
0
votes
0answers
32 views

Intuitive understanding about LSE

Let y= X$\beta$ + $\epsilon$ where y is $n \times 1$ vector, X is $n \times p+1$ matrix, $\beta$ is $p+1 \times 1$ vector, $\epsilon$ ~ $N(0, \sigma^2I_{n}$) now, let $y_{i}$ = $\hat{y_{i}}$ for some ...
0
votes
1answer
39 views

Math notation clarification

I'm working on learning more about logistic regression and I came across an equation with some confusing notation that I've never seen before: $$ \frac{\delta}{\delta \theta_{y'}^{(j)}} l(\theta) = ...
0
votes
0answers
22 views

minimum orders in linear regression to get a perfect fit

The problem is that, $(X_i, Y_i), i = 1,\ldots, n$ is an i.i.d. bivariate sample. Show that it is possible to fit a polynomial model using least square such that the fitted values are equal to the ...
1
vote
1answer
47 views

Significance level for a hypothesis test for a linear regression

Consider linear regression model $Y_i=a+b\cdot x_i+\epsilon_i$, $i=1,2,3,4,5$, where $a,b\in\mathbb{R}$ are unknown and $x_1=x_2=1,x_3=3,x_4=x_5=5$, $\epsilon_i$ are iid, normally distributed with ...
1
vote
1answer
15 views

contour plot in multiple linear regression

I have recently saw some examples about contour plots and multiple linear regression, for what I know a countour plot is obtained for having a graphical view of how the weights in a linear regression ...
3
votes
1answer
98 views

Find x,y & z (xyz+xyz=zyx)

I saw this problem the other day at work and found it pretty interesting: $$xyz + xyz = zyx$$ Find $x, y, z$ and the base(s) which this is true. Note that $x,y,z$ are simply digits concatenated, ...
0
votes
0answers
25 views

Question about ridge estimator

I have tried to show that ridge estimator is the solution to following problem min $(\beta- \hat{\beta})^t$$X^t X$$(\beta- \hat{\beta})$ subject to $\beta^t \beta =< d^2$ and $\beta$ is a $p$ x 1 ...
1
vote
1answer
30 views

Comparison of parameter: two different populations

I was wondering what the best way is to check for the equality of two parameters for a regression with no constant including possibly a confidence interval and p-value. $$H_0:\beta_1=\beta_2\ \vert\ ...
0
votes
0answers
25 views

two way ANOVA and linear regression model.

I know that Analysis of variance model can be written as a linear regression model using indicator regressors. For, one way ANOVA, I can write down the regression model. But for two way ANOVA ...
0
votes
1answer
35 views

Power form of regression equation which is not centered at x=0?

For a given set of data, the power form of the regression equation is given by $$y=b\cdot x^{m}$$ where $$m=\frac{n(\sum \mathrm{ln}(x_i)\mathrm{ln}(y_i))-(\sum \mathrm{ln}(x_i))(\sum ...
0
votes
1answer
17 views

standard error for the parameters of a linear regression model

Given a linear model $\mathbf{y} = \beta \mathbf{X} + \epsilon$, it is well known that the estimate for $\beta$ that gives the minimum residual sum of squares (RSS) is given by $\hat{\beta} = ...
0
votes
1answer
15 views

How to report significant digits in coefficient of determination?

Say that I fit some data with some model, for instance a linear function $y = mx+b$. What is the proper way to report the fitted coefficients and the goodness of fit? Specifically, if I do the fit in ...