Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
2 views

Showing Residual Sum of Squares for Multiple Linear Regression is 0

Problem: I have the linear regression model: $y_i=\beta_0+\sum_{k=1}^p \beta_kx_{ik}+\epsilon_i$ where $\epsilon_i$~N(0,$\sigma^2$), for i = 1,2,··· ,n. I want to prove that the residual sum of ...
1
vote
0answers
15 views

Name of the field of study that details extrapolation of a series based on subset sample data

Apologies if my notation and/or terminology is way off - I'm not well versed in mathematics. I'm looking for the name of the field of mathematics that might help me solve my problem. Here's my problem:...
1
vote
1answer
23 views

Derivative of dot product of Residual Sum Square in matrix notation

I am trying to derive the following expression w.r.t. $\beta$: \begin{equation} RSS(\beta) = (\mathbf{y} - \mathbf{X} \beta)^T (\mathbf{y} - \mathbf{X} \beta) \end{equation} I know that the ...
1
vote
1answer
42 views

Can I ignore multicolinearity problem if all the regression coefficients are highly significant?

Can I ignore multicolinearity problem if all the regression coefficients are highly significant? My data is large enough and all the resulting coefficients are significant enough in less than 0.01 ...
0
votes
0answers
18 views

Is there a way to determine the most most orthagonal variable or the “most powerful variable” in a Logistical Regression in the statstical software R?

I am curently working on a Logistical (Binary) Regression, and I am using R to create ROC curves based on the data. I cannot seem to determine exactly how I can determine variables to change to change ...
-2
votes
0answers
9 views

Interpretation and prediction interval for OLS regression

I have some questions about an exam problem in regression analysis from an exam I recently took. The question is as follows: You use a data set to compute a prediction interval for the mileage (...
1
vote
0answers
15 views

Confused about solution to the piecewise constant regression model

I am confused about the solution to the following solution to fitting piecewise constants: Specifically, are we minimising the sum of squares, that is, finding the vector $\beta = (\beta_1,\beta_2, ...
0
votes
1answer
18 views

error term in time-series Seasonal AR model

I am reading a paper related to timeseries forecasting in which I have a question regarding the seasonal AR model described in equation (1.2) namely: $log(y_t)$~$log(y_{t-1}) + log(y_{t-12}) + x^{(1)}...
0
votes
0answers
15 views

How to get best fitting model decision for data X and Y in e.g. Matlab?

I have two sources of data, X and Y, which are basically counts, from 23 individual origins (3D ROIs in my case). For example: ...
2
votes
0answers
54 views
+50

Two dimensional (discrete) orthogonal polynomials for regression

This question How to work out orthogonal polynomials for regression model and the answer http://math.stackexchange.com/a/354807/51020 explain how to build orthogonal polynomials for regression. ...
-1
votes
0answers
21 views

Linear regression equation

I am doing a project for a college math course and am stumped about something. I had to provide an (x,y) data table and then a linear regression scatter plot from the data. It was the winning times of ...
0
votes
1answer
33 views

How do I find first two steps of Newton's method?

How to find given by the points $(x_0,y_0)=(0.8,2.1)$ first two steps of newton method ,in order to approximate for $f(x,y)=x^3+14x+x^2y^2-5y$ one result of system of equation $\nabla f(x,y)=(0,0)$ ?
0
votes
0answers
22 views

Software to run non-linear regression

We currently use a very old version of StatGraphics and unfortunately it doesn't run on x64 systems. So I am looking for open-source software that may do the same thing as a replacement. Namely we run ...
1
vote
1answer
97 views

showing SSE of simple regression model is larger than or equal to SSE of multiple regression model

Lets say we have 2 linear regression models: $y_i = B_0 + B_1x_{i1} + \epsilon_i,$ where $\epsilon_i$ follows $N(0,σ_1^2)$ $y_i = B_0 + B_1x_{i1} + B_2x_{i2} + \lambda_i,$ where $ \lambda_i$ follows ...
1
vote
0answers
14 views

Numerical Method for fitting parameters of an explicit integration to actual data

I have a heat transfer system described by, $$\{\dot{T}\} = [C^{-1}]\left([K]\{T\} + \{F\} \right)$$ where ${T}$ is a vector of the nodal temperatures of the system. From initial conditions I am able ...
0
votes
0answers
27 views

A computer test of a very fast primarily test

Fermat's little theorem states that if $p$ is a prime and $a$ is any integer not divisible by $p$, then $a^{p-1} - 1$ is divisible by $p$. $$a^{p-1}\equiv 1\pmod p$$ This can be used to test if a ...
0
votes
0answers
17 views

predicted values combined with LDA

Suppose that we transform the original predictors X to Yˆ by taking the predicted values under linear regression. Show that LDA using Yˆ is identical to using LDA in the original space.
1
vote
1answer
31 views

Logistic regression MLE example. What is this “logistic function”?

This is a basic question (I think). I am trying to grasp the idea behind this example, where we define a "logistic function" and use that to work towards the maximum likelihood estimate (MLE). We ...
0
votes
0answers
12 views

Proof of Random Treatment's Effect on the Causal Regression Function

Consider $C(x)$ to be the outcome a subject would have if they recieve dose $x$ on some real interval. the observed response is given by the random variable relationship $Y = C(X)$. We treat ...
0
votes
1answer
31 views

Unbiased Estimate of Variance

Consider a simple linear regression model for $n$ observations where $$Y_i = \beta_1 X_i + \epsilon_i$$ where $\epsilon_i \sim N(0,\sigma^2).$ I want to show that $$\hat{\sigma}^2 = \frac{1}{n-2} \...
0
votes
0answers
8 views

How to make a covariance matrix from multiple observations of different objects?

I have $N$ objects. From each object, I sample $M$ values $(x,y)$ like so: ...
0
votes
0answers
13 views

Weighted Linear Regression

I am performing linear regression analysis on a time-series of data. Data contains some missing values, My question is, if I impute the missing values using mean of all the values and I want to ...
0
votes
0answers
28 views

Minimize the sum of distances between a sample and two “centers”

Suppose we have a set of readings $\{X_{i}\}$, each of which is a real number. What I want is to find 2 numbers, $a$ and $b$, such that minimize the sum of distances between each $X_{i}$ and ...
0
votes
1answer
26 views

coefficient of determination: absence of cross products [closed]

With regard to the coefficient of determination, why is the total variation equal to the sum of the explained variation and the unexplained variation and there are no cross-products?
1
vote
1answer
25 views

Best fit with unknown functional form

I have data from a simulation of a certain function $F(x)$ for a discrete set of values $\{x_i\}$. I know that $$F(x) = A \ f(x) + B \ g(x)$$ where $A,B$ are (real, positive) constants and $f,g$ ...
1
vote
1answer
26 views

How to find point in polynomial regresion

I have the following data set: ...
1
vote
1answer
36 views

Create a strictly increasing sequence following criterias

Problem Let y be a sequence of real numbers (of length $n$) bounded in the range [0,1]. I am trying to calculate the sequence x ...
0
votes
1answer
14 views

Question on optimization algorithm to train peculiar regression

I've been in my operations research course, and we have been working on optimization in particular problems within regression. We hypothesize that for variables $h,s,d,t,$ there is this set ...
2
votes
1answer
23 views

Variance of Least Squares Estimator

Suppose a fit a line using the method of least squares to $n$ points, all the standard statistical assumptions hold, and I want to estimate that line at a new point, $x_0$. Denoting that value by $\...
0
votes
1answer
21 views

Regression of y/x on x

I have a simple question but I do not manage to be sure! I would be very grateful if you can confirm me! Do we have the possibility to estimate the following model : $$\frac{y}{x}= \alpha+\beta x+\...
-1
votes
1answer
21 views

multiple linear regression model: scale the dependent variable y by a factor $c ∈ \mathbb{R}, c \neq 0$

In the multiple linear regression model $y = Xβ + u$, if you scale the dependent variable $y$ by a factor $c ∈ \mathbb{R}$, $c \neq 0$, how does the LS estimator $\hat{β}$ change? Does such a change ...
1
vote
1answer
28 views

Can I run a regression when both independent and dependent variables are all dichotomous?

I have conducted a survey where all my questions are asked in a dichotomous manner (Yes/No). Eg IV:"Are you a smoker?", "Are you obese", "Is your gender male/Female" etc. DV: "Have you ever had a ...
0
votes
0answers
13 views

What are the differences between stochastic v.s. fixed regressors in linear regression model?

If we have stochastic regressors, we are drawing random pairs $(y_i,\vec{x}_i)$ for a bunch of $i$, the so-called random sample, from a fixed but unknown probabilistic distribution $(y,\vec{x})$. ...
0
votes
1answer
26 views

Degrees of freedom of t-test in multiple regression .

Formula of t-test in regression is, $ t=\frac{\hat{\beta}-\beta}{se (\hat{\beta})} $ and the degrees of freedom of t-test is (n-k) because we estimate $\hat{\sigma}^2$ from RSS and the RSS has (n-k) ...
0
votes
0answers
13 views

Can the dependent variable in a multivariate linear regression be binary when the independent variables are continuous?

Can the dependent variable in a multivariate linear regression be binary when the independent variables are continuous?
0
votes
1answer
6 views

Calculating person product moment correlation coefficient on a 3 X 3 table

Usually we are given problems that only involve 2 rows (x and y), but recently saw a problem asking how to compute the correlation coefficient on a table of data that has 3 rows and am not sure how to ...
0
votes
1answer
31 views

How to fit a set of 3D points to a helical curve?

suppose I have a set of points in $\mathbb{R}^3$, and I want to find an arbitrary helix which best approximates these points. An arbitrary helix in $\mathbb{R}^3$ can be parametrized as $$\vec{r}(t)...
1
vote
0answers
21 views

Local quadratic approximation

I wanted to implement some penalized regression parameter estimation algorithm by Fan&Li (http://sites.stat.psu.edu/~rli/research/penlike.pdf, section 3.3), but cannot catch the idea of some ...
1
vote
1answer
54 views

How to Fit a Curve to a Given Model with Constraints?

The input are triples $\left\{ x,y,v\right\}$ where $x,y,v \in \mathbb{R} ^{+}$ I need to find function $f(x,y) = v$ by finding parameters of the following model $f(x,y) = a + bx^c + dy^e $ ...
1
vote
0answers
22 views

What would be the standard errors of this transformed regression model, given that I know the standard errors of the original model

Say I have the following regression model: $\ln\left(\dfrac{y_i}{x_{2i}}\right)=\alpha_1+\alpha_2\ln(x_{2i}) + \alpha_3\ln(x_{3i}) +e_i$ where I know the values of the regression coefficients and ...
0
votes
0answers
25 views

Linear or nonlinear modell

Given those three modells and the assignment to decide whether or not those modells can be transformed into linear modells: (a) $Y_i = \beta_0 + \dfrac{\beta_1}{x_{i1}}+\beta_2x_{i2}+\beta_3x_{i1}^{...
1
vote
0answers
18 views

Does increasing sample size have any effect on omitted variable bias?

Say I have a multiple linear regression model, where two of the variables are positively correlated, and I omit one of these variables from the model. First question - if I increase the sample size, ...
0
votes
0answers
24 views

Coefficient Correlation r of Exponential Functions Regression

I'm writing an exponent regression calculator $Ae^{Bx}$ Sample Data Set (X,Y) is (9, 1) (7, 10) (6,11) (20, 10) (15, 1) A = 5.287 and B = -0.0232. So $F(x) = 5....
0
votes
0answers
11 views

What are the causes of overfitting in regression/classification for statistical data?

Say I have some n-dimensional data, and I want to come up with some hypothesis function which generalizes that data for future predictions in the model. "Overfitness" of my hypothesis function is a ...
0
votes
0answers
19 views

$n$th order Polynomial for $(n+1)$ points

I was reading about Polynomial Fitting and found this sentence: How can one reach this conclusion and prove it?
0
votes
0answers
17 views

What is the equivalent of $R^2$ ( coefficient of determination in linear regression) for non linear regression?

I have a dataset with two correlating variables. The relation cannot be described as: $y= a+bx$. Therefore I told my math programm to calculate a nonlinear regression line. But unlike in linear ...
0
votes
0answers
16 views

Using optim and fitdistr in R to find parameters

I am using R to fit distributions. I have been given the data and have been asked to find the optimised parameters(for lognormal, weibull, exponential and gamma functions) using: 1) fitdistr and 2) ...
0
votes
0answers
9 views

What does fixed regressor say about our linearity condition?

The linearity condition states that $y_i=(\vec{x}_i)^{T}\vec{\beta}$ for all $i$. Now, if we have fixed regressors, $\{\vec{x}_1,\vec{x}_2,\cdots\}$, our linearity condition only says for those $\vec{...
0
votes
0answers
14 views

An example where pearson is wildy different to spearman? [duplicate]

Im looking to spearman and pearson, and from what i understand spearman is better at looking at curves. Can i see an example of a small set of data (10 or less) where this difference is large.