Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

3
votes
1answer
74 views

Why is $\sum x^2 _t \times \text{Var}(\beta)=\frac{\sum x^2 _t \times \sigma^2}{ \sum x^2 _t} = \sigma^2$?

I do not get this connection. Is is reliable to divide this equation by $\sum x^2 _t$ to get just $\sigma^2$ ? $$\sum x^2 _t \times E(\hat \beta - \beta)^2=\sum x^2 _t \times ...
3
votes
2answers
122 views

Can I consider shooting% as an independent variable

First time poster in the math section (a few posts in the stats section) and I am looking for clarification on a variable query that I have. Basically I enjoy sports and enjoy putting a mathematical ...
3
votes
2answers
540 views

Confidence interval of a random variable for an ordinary linear regression

I have a small problem. With my limited stats background I am not sure I am getting this one right. After fitting an ordinary linear regression model I get ...
3
votes
1answer
46 views

Weighted least squares with angular data

Suppose I have a system whose state is $\Theta=(\theta_1,\theta_2,\ldots,\theta_n)$, where $\theta_i\in[-\pi,\pi)$ (i.e., they are angles). I'd like to determine the most likely estimate of $\Theta$ ...
3
votes
2answers
104 views

Regression with error coming from rounding

I am looking at the following model: $c$ is a fixed vector in $\mathbb{R}_+^n$ and for any $x \in \mathbb{R}_+^n$ we obtain a value $y =[c^Tx]$, i.e. rounding $c^Tx$ to the nearest integer. I want ...
3
votes
0answers
35 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
3
votes
0answers
86 views

Determine whether ARMA(p,q) is stationary and/or invertible?

Determine whether an ARMA(p,q) process is stationary and invertible such that $y_t = \sum_{i=1}^{p} \phi_i y_{t-i} + \sum_{i=1}^{p} \theta_{i} \epsilon_{t-i}$ with the restriction that ...
3
votes
1answer
33 views

Logarithmic Functions algebra question

This is my first post and I honestly just want a second opinion on my answer to a question I got incorrect on an exam before I go arguing over it with my professor. Basically, is this mathematically ...
3
votes
0answers
73 views

A* vs D* vs Dijkstra [closed]

I understand the basis of A* as being a derivative of Dijkstra, however, I recently found out about D*. From wikipedia, I can understand the algorithm. What I do not understand is why I would use D* ...
3
votes
0answers
132 views

Can the sigmoid function approximate any function (or relation) where 0<y<1

I'm studying Machine Learning and Artificial Neural Networks. Some basic principles of Machine Learning are linear regression, multivariate linear regression, and nonlinear regression. The last of ...
3
votes
0answers
134 views

Multilinear or Tensor Regression?

Given input data $x_t\in \mathbb{R}^n$ and output data $y_t\in\mathbb{R}^m$, the closed form solution to $\min_A \sum_t \|y_t - Ax_t\|^2_2$ is given by $A = (XX^T)^{-1}XY^T$ where $x_t$ form the ...
3
votes
0answers
144 views

How to perform nonlinear regression with correlated errors?

I have a nonlinear least squares problem, but the errors are correlated. I could use R's nls function to do the regression if the errors were independent, but I don't know the right way to handle ...
3
votes
0answers
273 views

Least Square Method with Positive Parameters

this is my first post here in the Stack Exchange. A friend told me about this forum and I'm giving it a try. I searched a bit past threads, but couldn't find what I wanted, so I'm posting the problem ...
3
votes
0answers
868 views

Correlation and Regression Question

Two separate tests are designed to measure a students ability to solve problems. Several students are randomly selected to take both tests and the results are: $$ \begin{matrix} \text{Test A}(x) ...
3
votes
1answer
449 views

Fitting a sine function to data

I have a sequence of $n$ points $(x_i,y_i)$, for $i=1,\dots,n$. I would like to find the function, of the form $y=V\sin(x+\phi)$, which best fits the points. Which numerical method could I use? I have ...
3
votes
1answer
136 views

Bound of linear regression's object function

Randomly uniformly select $n$ numbers from a set $\{1,2,...,U\}$ with/without replacement, $y_i$ is the $i$th number selected, and $x_i$ is the rank of $y_i$ in the $n$ numbers. The rank is the order ...
2
votes
1answer
598 views

Polynomial fitting - how to fit and what is _polynomial fitting_

I don't understand what is polynomial fitting. Can anyone explain me how to fit a curve to given points?
2
votes
3answers
183 views

least squares regression in 3space

robjohn is giving me a hand with this, but in case anybody else knows... I need to do a least-squares regression for linearity on a set of coordinates in 3space. If the dataset is linear, I need to ...
2
votes
1answer
621 views

Why can we assume that the expected value of the error term is zero? [closed]

Why can we assume that the expected value of the error term in a linear regression model is zero? This is with regard to a simple linear regression.
2
votes
4answers
302 views

Fitting curve for exponential: $y = A - B\mathrm{e}^{-t/\tau}$

I have some data that follows a saturation or charging profile such as $y = A - B\mathrm{e}^{-t/\tau}$. To begin with, is there a proper name for this function? I have seen it many times, including: ...
2
votes
2answers
236 views

Closed form for coefficients in Multiple Regression model

I want to find $\hat{\beta}$ in ordinary least squares s.t. $\hat{Y} = \hat{\beta}_0 + \hat{\beta}_1 X_1 + \cdots + \hat{\beta}_n X_n $. I know the way to do this is through the normal equation using ...
2
votes
2answers
282 views

design matrices

Given a linear model $Y = X\beta + \epsilon$ with three treatments and six subjects where $X$ is the design matrix, suppose $X = \begin{matrix}1 & 1 & 0\\ 1 & 1 & 0\\ 1 & 0 ...
2
votes
1answer
602 views

Linear Regression with 3x3 Matrices

Here's my Homework Problem: We can generalize the least squares method to other polynomial curves. To find the quadratic equation $y=a x^2+b x+c$ that best fits the points $(-1, −3)$, $(0, 0)$, ...
2
votes
1answer
256 views

How do I do a least squares fit of $a x + b y = 1$?

How do I do a least squares fit of the line equation $a x + b y = 1$, so that the points are as close to the line as possible? (Not just vertically close) If I use the matrices $$X = ...
2
votes
2answers
2k views

Examples of logistic regression and multinomial logistic regression

I've been studying to understand the concept of logistic regression and I think I understand the idea more or less, but there are still some gaps to fill. What I'm looking for is an example of ...
2
votes
3answers
5k views

Fitting exponential curve to data

If I have a collection of data points that follow an exponential curve relationship, how can I manually construct the equation that defines the best-fit exponential curve for the data?
2
votes
2answers
109 views

Linear Models - Regression Analysis

As a student learning Applied Regression Analysis, I come from a background with very little information about this topic. I understand that given $y = \beta_0 + \beta_1x_1 + \epsilon$ $E(y\mid x) = ...
2
votes
1answer
644 views

What is $Cov(\hat{Y},Y)$?

If $\hat{Y}$ is the OLS linear regression model for $Y$, what can I say about $Cov(\hat{Y},Y)$? Is this value 0?
2
votes
1answer
1k views

Hat Matrix Identities in Regression

I need to show that $\bar h= \sum{h_{ii}/n} = \operatorname{Tr}[H]/n = (p+1)/n$ Using the fact that $\operatorname{Tr}[AB]=\operatorname{Tr}[BA]$ and $H=X(X^TX)^{-1}X^T$. But I have no idea how to ...
2
votes
3answers
367 views

Residuals of regression model

Let's suppose I do a regression between earnings and age (and suppose I do not know the distribution of earnings). Would it be possible for the residuals to be normally distributed? I am thinking it ...
2
votes
2answers
415 views

Calculate Line Of Best Fit Using Exponential Weighting?

I know how to calculate a line of best fit with a set of data. I want to be able to exponentially weight the data that is more recent so that the more recent data has a greater effect on the line. ...
2
votes
1answer
187 views

How to handle constant term in Least Squares Regression?

In the well known matrix form of a least squares regression where I am trying to solve for B in Y = B1X1 + B2X2 + B3 I might be given X and Y sample data as something like $X$ = $\begin{bmatrix} ...
2
votes
1answer
33 views

How to take the derivative of Matrices

I was browsing the derivation of the Least Squares estimates and stumbled about this problem. It said that: $$E = (Y + XB)^2$$ $$\frac{dE}{dB} = -X^TY + X^TXB$$ It is to my understanding that the ...
2
votes
1answer
55 views

Isn't the Hat Matrix just an identity matrix?

In Linear regression $y = X\beta + \epsilon$ The Hat matrix is defined to be $H = X(X^TX)^{-1} X^T$ . However. If I compute the equation for Hat matrix, I just get an identity matrix. My calculation ...
2
votes
1answer
43 views

How can I plot this?

Given a bunch of data $x_i$ , $y_i$, how do I plot $$f(\theta_2,\theta_2)= \frac{1}{2M} \sum_{i=1}^{M} (\theta_1\cdot x_i -\theta_2 y_i)^2$$ in matlab? I know it should be parabolic, but my code ...
2
votes
1answer
71 views

Fitting a simple linear regression

A professor in the School of Business in a university polled a dozen colleagues about the number of professional meetings they attended in the past five years $x$ and the number of papers they ...
2
votes
2answers
124 views

connection between PCA and linear regression

Is there a formal link between linear regression and PCA? The goal of PCA is to decompose a matrix into a linear combination of variables that contain most of the information in the matrix. Suppose ...
2
votes
1answer
53 views

Choosing the right regression

I'm trying to analyze my sleep using regression analysis. Each night is rated (dependent variable). I'm trying to explain this rating with, for example, my sleep duration and each night's bed time's ...
2
votes
1answer
29 views

Determine which parameter has correlation with result and which is not

sorry for probably silly question, it's the first time when I need to do such work. I have large data set with regarding clicks on some element on web page. It contains some characteristics of such ...
2
votes
4answers
98 views

Condition for $\det(A^{T}A)=0$

Is it always true that $\det(A^{T}A)=0$, $\hspace{0.5mm}$ for $A=n \times m$ matrix with $n<m$? From some notes I am reading on Regression analysis, and from some trials, it would appear this is ...
2
votes
1answer
354 views

Arriving at the Logistic function from a Binomial Distribution and Maximum Likelihood

I've been trying to understand the origin of the Logistic function in Logistic regression: $$\Pr(Y=1|x;\theta)=\frac{1}{1+e^{-\theta x}}$$ I was lead to beilive that one could somehow arrive at this ...
2
votes
1answer
91 views

Find data to perform regression analysis

I'm trying to find some data (two continuous variables that I believe are correlated) online for which I can perform a regression anaylsis, my assignment sheet says: The data may be found anywhere ...
2
votes
1answer
96 views

Linear relationship of a company's profit

Assume a linear relationship for a company that has several shops is not known. Let $Y_i$ be the profit the shop number $i$ makes in the coming year. Let $x_i$ be the size of the shop number ...
2
votes
4answers
87 views

Seeking a function based on its level set

I'm trying to create a function for a research project, but I fear my math knowledge is insufficient to derive it from the attached diagram I've created showing its desired behavior. I'm hoping ...
2
votes
3answers
428 views

How to deal with Linear Regression model with some data aggregated

Lets say I am trying to find a linear regression between Weight and Height of a person. $W=b_0+b_1 H+e$ The data I have gathered from 8 people is like this: ...
2
votes
1answer
3k views

How to calculate hyperbola from data points?

I have 4 data points, from which I want to calculate a hyperbola. It seems that the Excel trendline feature can't do it for me, so how do I find the relationship? The points are: (x,y) (3, 0.008) ...
2
votes
1answer
24 views

Analytic solution for matrix factorization using alternating least squares

The standard form for ridge regression aims to minimize the following cost function. $$ \min\ \ \sum_i(y_i-x_i^T\beta)^2 + \lambda\sum_j\beta^2_j $$ As described here, it's possible to differentiate ...
2
votes
1answer
35 views

Standard Error in OLS Regression

Assuming I have the following linear regression set-up: $y_i = \alpha + x_i * \beta + \epsilon_i$ for $i = 1,2,..., n$. When I run the regession, I get a $\beta$ and $\alpha$ estimates, along ...
2
votes
1answer
30 views

Gaussian prior favors values closest to zero?

I am reading an article on Bayesian Logistic Regression, where they're using Logistic Regression, imposing a Gaussian prior (with mean = 0) on its parameters. They state that a Gaussian prior favors ...
2
votes
1answer
394 views

What is the difference between Curve Fitting and Regression(Machine Learning)?

I know that Machine Learning regression algorithms try to find the function of the data. That is, if we have 1000 data points (x,y), to find a general continuous function that follows the trends of ...