Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
2answers
25 views

Errors and Residual

Why are errors independent but residuals dependent? As far i know the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also ...
1
vote
2answers
21 views

Simple linear regression seems off

I have the following datapoints: $$p1(52,730)$$ $$p2(53,409)$$ $$p3(52,250)$$ $$p4(52,90)$$ Now I want to find the best fitting line between these points. When I use simple linear regression I get $$y ...
2
votes
2answers
53 views
+100

Regression with error coming from rounding

I am looking at the following model: $c$ is a fixed vector in $\mathbb{R}_+^n$ and for any $x \in \mathbb{R}_+^n$ we obtain a value $y =[c^Tx]$, i.e. rounding $c^Tx$ to the nearest integer. I want ...
0
votes
0answers
15 views

Random variable variance

I have the model yi=β1+β2Xi+ui where ui∼iid N(0,σ2). I estimate β1 and β2 by drawing a straight line between the first (x1,y1) and last dot (xn,yn). So, β̂ 2 will be the slope of this straight line. ...
2
votes
1answer
286 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: y=value a1=material a2=weight a3=color a1 and a2 are interacting variables but a3 is not. Right now I'm ...
0
votes
2answers
50 views

Equation going infinitely towards y = 10

I'm programming a site on which I sell services, and the more the customers spends, the more discount they will have. Please have a look at the diagram below. Spending 700 USD will result in 5% ...
3
votes
1answer
41 views

Weighted least squares with angular data

Suppose I have a system whose state is $\Theta=(\theta_1,\theta_2,\ldots,\theta_n)$, where $\theta_i\in[-\pi,\pi)$ (i.e., they are angles). I'd like to determine the most likely estimate of $\Theta$ ...
1
vote
2answers
30 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
0
votes
0answers
21 views

numerical (script) fit of function with 2 arguments

I would like to find the least-square fit for a 1D-function that takes two arguments. m(x,y) = d * (x-x0)^2 / (y-y0)^2 I would like to write a c++ routine to ...
1
vote
1answer
26 views

time-series regression with missing data

I have a regression as follows for time-series data (e.g. stock prices versus other variables): $$ Y = b \cdot X + b_1 \cdot X_1 + e$$ where $X_1$ will be missing based on pre-determined dates ...
2
votes
1answer
44 views

Is there a site that will allow me to calculate a best fit for a set of data?

I have a bunch of x's and their corresponding y values, but do not have a Wolfram Pro account. Is there another site where I can input my dataset and have it spit out a best-fit regression (be it ...
0
votes
1answer
16 views

Minimum number of observations in non-linear regression

Are 5 observations enough to verify the following non-linear regression model in the form: $ Y= C K_0^{\alpha_0}K_1^{\alpha_1}K_2^{\alpha_2}$ And in general how many observations do I need for models ...
0
votes
0answers
14 views

factor models and using cross sectional regression?

I have been doing some reading on factor models. In the literature it mentions that when creating a portfolio that maximises particular attributes it may lead to unwanted bias to other factors. I ...
1
vote
1answer
60 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...
1
vote
2answers
124 views

How do I find equation of this curve?

I need to find equation of the curve as shown below, for which, I need to find equation for upper part. lower part is half circle. upper part is a constant distance from circle with line passing ...
1
vote
2answers
24 views

Conditional Expectation, Orthogonality, and Correlation

I know that if $\epsilon$ and $x$ are independent, then $E[\epsilon|x]=E[\epsilon]$ and Cov$(\epsilon,x)=0$. However, $E[\epsilon|x]=E[\epsilon]=0$ implies Cov$(\epsilon,x)=0$ iff $\epsilon$ and $x$ ...
1
vote
3answers
48 views

Linear regression throug the origin versus mean?

Assume that I have data that can be described by: $y_i = \beta x_i + \epsilon_i, \epsilon_i \sim (0,\sigma_{\epsilon})$, then the least squares estimator is given by $\hat{\beta_1} = ...
1
vote
0answers
12 views

Non-Linear Regression involving the maximum function

How do you calculate the regression of this model? I know Minitab and MATLAB, so if you guide me with these software I would totally appreciate it. $$Y=c+\max(X^{-n}, 0.34) $$ Here c and n are ...
1
vote
0answers
14 views

Good MSE doesnt imply good prediction in logistic regression?

I am writing some code for regularized logistic regression. I observe this interesting phenomena and wonder if it is a normal thing or just my code is wrong. For loss function, I am using the ...
0
votes
0answers
13 views

Nearest points / residuals on a total least squares parabola

Consider fitting a parabola $y = a + bx + cx^2$ to 2d data $X_i, Y_i$ with noise in both X and Y, using the the singular value decomposition as in Total_least_squares (TLS): $\qquad X = [ 1\ \ Xdata\ ...
1
vote
1answer
24 views

Finding best predictors of a classification function

I have a large dataset where each element has a number of "input" categories that are either present or not (or if you like, true or false, 1 or 0 etc). Each one also has an output category, again a ...
2
votes
0answers
10 views

Showing Hat matrix equal specific values

Consider a one way layout model $y_{ij}$ = $\mu_i + e_{ij}$ (1 $\leq$ i $\leq$ a, 1 $\leq$ j $\leq$ $n_i$) where a = 3 and $n_1$ = 2, $n_2$ = 3, $n_3$ = 4. Show that the hat matrix for this design ...
0
votes
1answer
57 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
0
votes
0answers
8 views

How to build a prediction model for exam score based on previous scores [migrated]

I am trying to construct a formula, which will take student's previous exam results (for ex: SAT) taken at particular dates and predict his future test result, given that he continues to study with ...
0
votes
2answers
41 views

How to do non-linear regression with this function

I have observed that my data matches the function : $ a e^{bx}+c $ I want to get the parameters a ,b and c. I know how to solve this problem if c equals 0. But how to solve it when c involves in?
0
votes
1answer
31 views

Regression with Mean, Standard Deviation, Range and Correlation

A research team collected data on students in a statistics course. Their dependent variable was the student’s score on the final examination, which ranged from 200 to 800 points. The observed average ...
0
votes
0answers
10 views

Finding rounding rules underlying rounded function results

Here's the problem I've been stuck on for two weeks now. I basically have a black-box function, which takes into input an integer and returns an integer, and I'm trying to determine it. Here are the ...
0
votes
1answer
37 views

4 points, how to know if it's growing over time?

I've an array of 4 points, which formula should I use to detect their growth ? ...
0
votes
1answer
13 views

Deriving estimators for the parameters a and b that minimize the random error - setting up linear regression variables?

I'm reviewing old notes, and I know I solved this way back when, but can't remember how to know: Consider the simple linear regression model: $$Y_i = a + bX_i + \epsilon_i$$ where $Y_i$ is the ...
1
vote
0answers
20 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
0
votes
2answers
32 views

Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
1
vote
0answers
69 views

Finding the best fitting square

I have to find the best fitting square using the total least squares method. First we we had to find the best fitting rectangle using the following equations: s1: $c1+ax+by=0$, $a^2 +b^2 =1$ s2: ...
1
vote
1answer
395 views

Fitting a sine function to data

I have a sequence of $n$ points $(x_i,y_i)$, for $i=1,\dots,n$. I would like to find the function, of the form $y=V\sin(x+\phi)$, which best fits the points. Which numerical method could I use? I have ...
0
votes
0answers
34 views

Quantile as solution to minimization problem

I'm studying basics of quantile regression now and I have trouble prooving that $\tau-$th quantile of real-valued random variable $Y$ is a solution to the following minimization problem (in the ...
0
votes
0answers
22 views

How to calculate the coordinate of a point which depends on other points on a plane with specific distances

I have $8$ points on a plane $(x_1,y_1)....(x_8,y_8)$ among these $8$ points I know the coordinates for $7$ points and I have to find the $8^{th}$ point. Each points has the difference between all ...
0
votes
1answer
17 views

Regression when the variance of the residuals depends on the independent variable

When the residuals follow a normal distribution, the most likely function that fits the data is found using least squares. In that case: $y = f(x_i) + r_i, \quad r\sim\mathcal{N}(0, \sigma^2)$ ...
0
votes
0answers
9 views

Ordered logistic regression with likert scales

I'm currently have a bit of difficulty determining how to analyze this data via logistic regression analysis. Q18 = DV (satisfaction score ranging from 1-10) Q10_1 = IV (Customer Service likert ...
5
votes
2answers
2k views

Correlation Coefficient and Determination Coefficient

I'm really new to linear regression and am trying to teach myself. In my textbook there's a problem that asks why R^2 in the regression of Y on X = the sample correlation between X and Y the whole ...
0
votes
1answer
15 views

Contribution of each variable in multiple linear regression

What will be the best measure of the contribution of a variable in multiple linear regression? I was thinking of using the coefficient ratio as a marker of a variable's contribution. For example: ...
4
votes
2answers
68 views

Rewriting the matrix equation $AX = YB$ as $Y = CX$?

Is it possible in general, if $A,B,C,X,Y$ are square and of the same dimensions? If so, does it generalize to non-square matrices (using a pseudoinverse)? I'm doing some curve fitting in which I have ...
1
vote
0answers
8 views

Finding posterior of normal distributions and logistic regression.

$P(w_0 | x) = \frac{1}{1 + e^{-log\frac{P(x|w_0)}{P(x|w_1)}-log\frac{P(w_0)}{P(w_1)}}}$ Note: x = $[x_1, \dots, x_d]^T$; a $d$ dimensional vector. $w$ can take on one of two values: $w_0$ or $w_1$. ...
1
vote
1answer
38 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
10
votes
5answers
11k views

regression vs classification

THis is more machine learning questions, but perhaps someone will be able to help. I would like to know what is the diference between regression and classification when we try to generate output for a ...
0
votes
0answers
12 views

Spatially model

Can someone explain to me what a 'spatially lagged autoregressive model' is ? I came across this 'model' by searching new techniques for modeling geographical data.
0
votes
0answers
21 views

Estimation of real estate

I'm working on a project on estimating the price value of real estate. First I have collected a lot of data (500 000 instances) with details such as postal code, number of bedrooms, build year, ...
0
votes
1answer
24 views

Find optimal least square solution to the normal equation

What is the optimal solution for $\beta_1$ and $\beta_2$ in the following normal equation: $$\beta _{ 1 }\sum _{ i=1 }^{ n }{ { x }_{ i } } +\beta _{ 0 }=\sum _{ i=1 }^{ n }{ { y }_{ i } } $$ EDIT ...
1
vote
1answer
35 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
3
votes
0answers
33 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
7
votes
2answers
2k views

derivative of cost function for Logistic Regression

I am going over the lectures on Machine Learning at Coursera. I am struggling with the following. How can the partial derivative of ...
1
vote
0answers
20 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...