Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
1answer
20 views

Cubic Polynomial fitting with defined ranges for coefficients

Is there a way, given a set of values $(x,y)$, to find a cubic polynomial $f(x)$ that fits the values? My cubic polynomial is defined as $c_0 + c_1x +\frac {1}{2} c_2 x^2 +\frac {1}{6}c_3 x^3$ ...
2
votes
0answers
12 views

Derive the Hat Matrix to map actual response to estimated resposne

In order to measure the quality of a regression we can calculate the Hat Matrix. Using it we can estimate the response variable as if we used the predictor variables to regress them. For linear ...
0
votes
0answers
20 views

MLE of heteroscedastic model?

Given the regression model where our and are identically and independently distributed. I'm trying to find the MLE B-hat and the unbiased estimator sigma-hat^2. I haven't dealt with any models in ...
0
votes
1answer
33 views

Showing sum of squared residuals is zero?

I have the model $$y_i = B_0+\sum\limits_{i=0}^pB_kX_{ik} + e_i$$ I'm looking to show the sum of squared residuals is zero if $p = (n-1)$. I have tried expanding it quite in depth and I haven't been ...
1
vote
1answer
39 views

Showing Residual Sum of Squares for Multiple Linear Regression is 0

Problem: I have the linear regression model: $y_i=\beta_0+\sum_{k=1}^p \beta_kx_{ik}+\epsilon_i$ where $\epsilon_i\sim N(0,\sigma^2)$, for $i = 1,2,\ldots ,n$. I want to prove that the residual sum ...
3
votes
2answers
2k views

Assumption of a Random error term in a regression

In one of my recent statistics courses, our teacher introduced the linear regression model. The typical $y=\alpha + \beta X + \epsilon$, where $\epsilon$ is a "random" error term. The teacher then ...
1
vote
0answers
15 views

Name of the field of study that details extrapolation of a series based on subset sample data

Apologies if my notation and/or terminology is way off - I'm not well versed in mathematics. I'm looking for the name of the field of mathematics that might help me solve my problem. Here's my problem:...
1
vote
1answer
25 views

Derivative of dot product of Residual Sum Square in matrix notation

I am trying to derive the following expression w.r.t. $\beta$: \begin{equation} RSS(\beta) = (\mathbf{y} - \mathbf{X} \beta)^T (\mathbf{y} - \mathbf{X} \beta) \end{equation} I know that the ...
3
votes
0answers
73 views

Two dimensional (discrete) orthogonal polynomials for regression

This question How to work out orthogonal polynomials for regression model and the answer http://math.stackexchange.com/a/354807/51020 explain how to build orthogonal polynomials for regression. ...
0
votes
1answer
19 views

error term in time-series Seasonal AR model

I am reading a paper related to timeseries forecasting in which I have a question regarding the seasonal AR model described in equation (1.2) namely: $log(y_t)$~$log(y_{t-1}) + log(y_{t-12}) + x^{(1)}...
1
vote
1answer
42 views

Can I ignore multicolinearity problem if all the regression coefficients are highly significant?

Can I ignore multicolinearity problem if all the regression coefficients are highly significant? My data is large enough and all the resulting coefficients are significant enough in less than 0.01 ...
1
vote
0answers
16 views

Confused about solution to the piecewise constant regression model

I am confused about the solution to the following solution to fitting piecewise constants: Specifically, are we minimising the sum of squares, that is, finding the vector $\beta = (\beta_1,\beta_2, ...
0
votes
0answers
19 views

Is there a way to determine the most most orthagonal variable or the “most powerful variable” in a Logistical Regression in the statstical software R?

I am curently working on a Logistical (Binary) Regression, and I am using R to create ROC curves based on the data. I cannot seem to determine exactly how I can determine variables to change to change ...
-2
votes
0answers
10 views

Interpretation and prediction interval for OLS regression

I have some questions about an exam problem in regression analysis from an exam I recently took. The question is as follows: You use a data set to compute a prediction interval for the mileage (...
0
votes
1answer
33 views

How do I find first two steps of Newton's method?

How to find given by the points $(x_0,y_0)=(0.8,2.1)$ first two steps of newton method ,in order to approximate for $f(x,y)=x^3+14x+x^2y^2-5y$ one result of system of equation $\nabla f(x,y)=(0,0)$ ?
0
votes
0answers
15 views

How to get best fitting model decision for data X and Y in e.g. Matlab?

I have two sources of data, X and Y, which are basically counts, from 23 individual origins (3D ROIs in my case). For example: ...
-1
votes
0answers
21 views

Linear regression equation

I am doing a project for a college math course and am stumped about something. I had to provide an (x,y) data table and then a linear regression scatter plot from the data. It was the winning times of ...
0
votes
1answer
1k views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
0
votes
1answer
531 views

Convert odds ratio based on unit change to several unit changes

Imagine to have two groups of people, the first one more strongly exposed to a pollutant than the second one, and the first one developing a certain disease more often. Having measurements of the ...
2
votes
2answers
862 views

connection between PCA and linear regression

Is there a formal link between linear regression and PCA? The goal of PCA is to decompose a matrix into a linear combination of variables that contain most of the information in the matrix. Suppose ...
0
votes
0answers
22 views

Software to run non-linear regression

We currently use a very old version of StatGraphics and unfortunately it doesn't run on x64 systems. So I am looking for open-source software that may do the same thing as a replacement. Namely we run ...
0
votes
1answer
26 views

coefficient of determination: absence of cross products [closed]

With regard to the coefficient of determination, why is the total variation equal to the sum of the explained variation and the unexplained variation and there are no cross-products?
1
vote
0answers
101 views

showing SSE of simple regression model is larger than or equal to SSE of multiple regression model

Lets say we have 2 linear regression models: $y_i = B_0 + B_1x_{i1} + \epsilon_i,$ where $\epsilon_i$ follows $N(0,σ_1^2)$ $y_i = B_0 + B_1x_{i1} + B_2x_{i2} + \lambda_i,$ where $ \lambda_i$ follows ...
1
vote
0answers
14 views

Numerical Method for fitting parameters of an explicit integration to actual data

I have a heat transfer system described by, $$\{\dot{T}\} = [C^{-1}]\left([K]\{T\} + \{F\} \right)$$ where ${T}$ is a vector of the nodal temperatures of the system. From initial conditions I am able ...
0
votes
0answers
27 views

A computer test of a very fast primarily test

Fermat's little theorem states that if $p$ is a prime and $a$ is any integer not divisible by $p$, then $a^{p-1} - 1$ is divisible by $p$. $$a^{p-1}\equiv 1\pmod p$$ This can be used to test if a ...
0
votes
0answers
18 views

predicted values combined with LDA

Suppose that we transform the original predictors X to Yˆ by taking the predicted values under linear regression. Show that LDA using Yˆ is identical to using LDA in the original space.
1
vote
1answer
31 views

Logistic regression MLE example. What is this “logistic function”?

This is a basic question (I think). I am trying to grasp the idea behind this example, where we define a "logistic function" and use that to work towards the maximum likelihood estimate (MLE). We ...
0
votes
0answers
13 views

Proof of Random Treatment's Effect on the Causal Regression Function

Consider $C(x)$ to be the outcome a subject would have if they recieve dose $x$ on some real interval. the observed response is given by the random variable relationship $Y = C(X)$. We treat ...
0
votes
1answer
32 views

Unbiased Estimate of Variance

Consider a simple linear regression model for $n$ observations where $$Y_i = \beta_1 X_i + \epsilon_i$$ where $\epsilon_i \sim N(0,\sigma^2).$ I want to show that $$\hat{\sigma}^2 = \frac{1}{n-2} \...
0
votes
0answers
28 views

Minimize the sum of distances between a sample and two “centers”

Suppose we have a set of readings $\{X_{i}\}$, each of which is a real number. What I want is to find 2 numbers, $a$ and $b$, such that minimize the sum of distances between each $X_{i}$ and ...
0
votes
0answers
8 views
0
votes
0answers
13 views

Weighted Linear Regression

I am performing linear regression analysis on a time-series of data. Data contains some missing values, My question is, if I impute the missing values using mean of all the values and I want to ...
0
votes
1answer
15 views

Question on optimization algorithm to train peculiar regression

I've been in my operations research course, and we have been working on optimization in particular problems within regression. We hypothesize that for variables $h,s,d,t,$ there is this set ...
1
vote
1answer
25 views

Best fit with unknown functional form

I have data from a simulation of a certain function $F(x)$ for a discrete set of values $\{x_i\}$. I know that $$F(x) = A \ f(x) + B \ g(x)$$ where $A,B$ are (real, positive) constants and $f,g$ ...
1
vote
1answer
37 views

Create a strictly increasing sequence following criterias

Problem Let y be a sequence of real numbers (of length $n$) bounded in the range [0,1]. I am trying to calculate the sequence x ...
-2
votes
1answer
24 views

multiple linear regression model: scale the dependent variable y by a factor $c ∈ \mathbb{R}, c \neq 0$ [on hold]

In the multiple linear regression model $y = Xβ + u$, if you scale the dependent variable $y$ by a factor $c ∈ \mathbb{R}$, $c \neq 0$, how does the LS estimator $\hat{β}$ change? Does such a change ...
2
votes
1answer
24 views

Variance of Least Squares Estimator

Suppose a fit a line using the method of least squares to $n$ points, all the standard statistical assumptions hold, and I want to estimate that line at a new point, $x_0$. Denoting that value by $\...
2
votes
1answer
678 views

Linear Regression Model, linearity in parameters/ variables

I am confusing with the wording here. I was reading a book on linear regression. "The primary concern for linear models is that they display linearity in the parameters. Therefore, when we refer to a ...
4
votes
1answer
494 views

Determine whether ARMA(p,q) is stationary and/or invertible?

Determine whether an ARMA(p,q) process is stationary and invertible such that $y_t = \sum_{i=1}^{p} \phi_i y_{t-i} + \sum_{i=1}^{p} \theta_{i} \epsilon_{t-i}$ with the restriction that $\theta_{0} = ...
0
votes
1answer
21 views

Regression of y/x on x

I have a simple question but I do not manage to be sure! I would be very grateful if you can confirm me! Do we have the possibility to estimate the following model : $$\frac{y}{x}= \alpha+\beta x+\...
1
vote
1answer
29 views

Can I run a regression when both independent and dependent variables are all dichotomous?

I have conducted a survey where all my questions are asked in a dichotomous manner (Yes/No). Eg IV:"Are you a smoker?", "Are you obese", "Is your gender male/Female" etc. DV: "Have you ever had a ...
0
votes
1answer
422 views

Fast way of finding RSS of Multiple Linear Regression

Is there any smarter way to compute Residual Sum of Squares(RSS) in Multiple Linear Regression other then fitting the model -> find coefficients -> find fitted values -> find residuals -> find norm of ...
0
votes
1answer
6 views

Calculating person product moment correlation coefficient on a 3 X 3 table

Usually we are given problems that only involve 2 rows (x and y), but recently saw a problem asking how to compute the correlation coefficient on a table of data that has 3 rows and am not sure how to ...
0
votes
0answers
13 views

What are the differences between stochastic v.s. fixed regressors in linear regression model?

If we have stochastic regressors, we are drawing random pairs $(y_i,\vec{x}_i)$ for a bunch of $i$, the so-called random sample, from a fixed but unknown probabilistic distribution $(y,\vec{x})$. ...
0
votes
1answer
26 views

Degrees of freedom of t-test in multiple regression .

Formula of t-test in regression is, $ t=\frac{\hat{\beta}-\beta}{se (\hat{\beta})} $ and the degrees of freedom of t-test is (n-k) because we estimate $\hat{\sigma}^2$ from RSS and the RSS has (n-k) ...
0
votes
0answers
13 views

Can the dependent variable in a multivariate linear regression be binary when the independent variables are continuous?

Can the dependent variable in a multivariate linear regression be binary when the independent variables are continuous?
0
votes
1answer
33 views

How to fit a set of 3D points to a helical curve?

suppose I have a set of points in $\mathbb{R}^3$, and I want to find an arbitrary helix which best approximates these points. An arbitrary helix in $\mathbb{R}^3$ can be parametrized as $$\vec{r}(t)...
6
votes
4answers
5k views

Correlation Coefficient and Determination Coefficient

I'm really new to linear regression and am trying to teach myself. In my textbook there's a problem that asks why $R^{2}$ in the regression of $Y$ on $X =$ the sample correlation between X and Y the ...
1
vote
1answer
54 views

How to Fit a Curve to a Given Model with Constraints?

The input are triples $\left\{ x,y,v\right\}$ where $x,y,v \in \mathbb{R} ^{+}$ I need to find function $f(x,y) = v$ by finding parameters of the following model $f(x,y) = a + bx^c + dy^e $ ...