Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

1
vote
1answer
13 views

Simple linear regression seems off

I have the following datapoints: $$p1(52,730)$$ $$p2(53,409)$$ $$p3(52,250)$$ $$p4(52,90)$$ Now I want to find the best fitting line between these points. When I use simple linear regression I get $$y ...
0
votes
0answers
15 views

Random variable variance

I have the model yi=β1+β2Xi+ui where ui∼iid N(0,σ2). I estimate β1 and β2 by drawing a straight line between the first (x1,y1) and last dot (xn,yn). So, β̂ 2 will be the slope of this straight line. ...
3
votes
1answer
41 views

Weighted least squares with angular data

Suppose I have a system whose state is $\Theta=(\theta_1,\theta_2,\ldots,\theta_n)$, where $\theta_i\in[-\pi,\pi)$ (i.e., they are angles). I'd like to determine the most likely estimate of $\Theta$ ...
1
vote
1answer
38 views
+100

Regression with error coming from rounding

I am looking at the following model: $c$ is a fixed vector in $\mathbb{R}_+^n$ and for any $x \in \mathbb{R}_+^n$ we obtain a value $y =[c^Tx]$, i.e. rounding $c^Tx$ to the nearest integer. I want ...
0
votes
0answers
21 views

numerical (script) fit of function with 2 arguments

I would like to find the least-square fit for a 1D-function that takes two arguments. m(x,y) = d * (x-x0)^2 / (y-y0)^2 I would like to write a c++ routine to ...
1
vote
2answers
30 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
2
votes
1answer
41 views

Is there a site that will allow me to calculate a best fit for a set of data?

I have a bunch of x's and their corresponding y values, but do not have a Wolfram Pro account. Is there another site where I can input my dataset and have it spit out a best-fit regression (be it ...
0
votes
1answer
16 views

Minimum number of observations in non-linear regression

Are 5 observations enough to verify the following non-linear regression model in the form: $ Y= C K_0^{\alpha_0}K_1^{\alpha_1}K_2^{\alpha_2}$ And in general how many observations do I need for models ...
0
votes
0answers
14 views

factor models and using cross sectional regression?

I have been doing some reading on factor models. In the literature it mentions that when creating a portfolio that maximises particular attributes it may lead to unwanted bias to other factors. I ...
0
votes
2answers
50 views

Equation going infinitely towards y = 10

I'm programming a site on which I sell services, and the more the customers spends, the more discount they will have. Please have a look at the diagram below. Spending 700 USD will result in 5% ...
1
vote
1answer
59 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...
1
vote
0answers
12 views

Non-Linear Regression involving the maximum function

How do you calculate the regression of this model? I know Minitab and MATLAB, so if you guide me with these software I would totally appreciate it. $$Y=c+\max(X^{-n}, 0.34) $$ Here c and n are ...
1
vote
3answers
48 views

Linear regression throug the origin versus mean?

Assume that I have data that can be described by: $y_i = \beta x_i + \epsilon_i, \epsilon_i \sim (0,\sigma_{\epsilon})$, then the least squares estimator is given by $\hat{\beta_1} = ...
1
vote
2answers
124 views

How do I find equation of this curve?

I need to find equation of the curve as shown below, for which, I need to find equation for upper part. lower part is half circle. upper part is a constant distance from circle with line passing ...
1
vote
0answers
14 views

Good MSE doesnt imply good prediction in logistic regression?

I am writing some code for regularized logistic regression. I observe this interesting phenomena and wonder if it is a normal thing or just my code is wrong. For loss function, I am using the ...
0
votes
0answers
13 views

Nearest points / residuals on a total least squares parabola

Consider fitting a parabola $y = a + bx + cx^2$ to 2d data $X_i, Y_i$ with noise in both X and Y, using the the singular value decomposition as in Total_least_squares (TLS): $\qquad X = [ 1\ \ Xdata\ ...
1
vote
1answer
26 views

time-series regression with missing data

I have a regression as follows for time-series data (e.g. stock prices versus other variables): $$ Y = b \cdot X + b_1 \cdot X_1 + e$$ where $X_1$ will be missing based on pre-determined dates ...
2
votes
0answers
10 views

Showing Hat matrix equal specific values

Consider a one way layout model $y_{ij}$ = $\mu_i + e_{ij}$ (1 $\leq$ i $\leq$ a, 1 $\leq$ j $\leq$ $n_i$) where a = 3 and $n_1$ = 2, $n_2$ = 3, $n_3$ = 4. Show that the hat matrix for this design ...
0
votes
1answer
56 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
0
votes
0answers
8 views

How to build a prediction model for exam score based on previous scores [migrated]

I am trying to construct a formula, which will take student's previous exam results (for ex: SAT) taken at particular dates and predict his future test result, given that he continues to study with ...
0
votes
2answers
41 views

How to do non-linear regression with this function

I have observed that my data matches the function : $ a e^{bx}+c $ I want to get the parameters a ,b and c. I know how to solve this problem if c equals 0. But how to solve it when c involves in?
0
votes
1answer
31 views

Regression with Mean, Standard Deviation, Range and Correlation

A research team collected data on students in a statistics course. Their dependent variable was the student’s score on the final examination, which ranged from 200 to 800 points. The observed average ...
0
votes
0answers
10 views

Finding rounding rules underlying rounded function results

Here's the problem I've been stuck on for two weeks now. I basically have a black-box function, which takes into input an integer and returns an integer, and I'm trying to determine it. Here are the ...
0
votes
1answer
37 views

4 points, how to know if it's growing over time?

I've an array of 4 points, which formula should I use to detect their growth ? ...
0
votes
1answer
13 views

Deriving estimators for the parameters a and b that minimize the random error - setting up linear regression variables?

I'm reviewing old notes, and I know I solved this way back when, but can't remember how to know: Consider the simple linear regression model: $$Y_i = a + bX_i + \epsilon_i$$ where $Y_i$ is the ...
1
vote
0answers
19 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
0
votes
2answers
32 views

Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
1
vote
1answer
24 views

Finding best predictors of a classification function

I have a large dataset where each element has a number of "input" categories that are either present or not (or if you like, true or false, 1 or 0 etc). Each one also has an output category, again a ...
0
votes
0answers
34 views

Quantile as solution to minimization problem

I'm studying basics of quantile regression now and I have trouble prooving that $\tau-$th quantile of real-valued random variable $Y$ is a solution to the following minimization problem (in the ...
0
votes
0answers
22 views

How to calculate the coordinate of a point which depends on other points on a plane with specific distances

I have $8$ points on a plane $(x_1,y_1)....(x_8,y_8)$ among these $8$ points I know the coordinates for $7$ points and I have to find the $8^{th}$ point. Each points has the difference between all ...
0
votes
0answers
9 views

Ordered logistic regression with likert scales

I'm currently have a bit of difficulty determining how to analyze this data via logistic regression analysis. Q18 = DV (satisfaction score ranging from 1-10) Q10_1 = IV (Customer Service likert ...
0
votes
1answer
17 views

Regression when the variance of the residuals depends on the independent variable

When the residuals follow a normal distribution, the most likely function that fits the data is found using least squares. In that case: $y = f(x_i) + r_i, \quad r\sim\mathcal{N}(0, \sigma^2)$ ...
0
votes
1answer
15 views

Contribution of each variable in multiple linear regression

What will be the best measure of the contribution of a variable in multiple linear regression? I was thinking of using the coefficient ratio as a marker of a variable's contribution. For example: ...
4
votes
2answers
68 views

Rewriting the matrix equation $AX = YB$ as $Y = CX$?

Is it possible in general, if $A,B,C,X,Y$ are square and of the same dimensions? If so, does it generalize to non-square matrices (using a pseudoinverse)? I'm doing some curve fitting in which I have ...
1
vote
0answers
8 views

Finding posterior of normal distributions and logistic regression.

$P(w_0 | x) = \frac{1}{1 + e^{-log\frac{P(x|w_0)}{P(x|w_1)}-log\frac{P(w_0)}{P(w_1)}}}$ Note: x = $[x_1, \dots, x_d]^T$; a $d$ dimensional vector. $w$ can take on one of two values: $w_0$ or $w_1$. ...
0
votes
0answers
12 views

Spatially model

Can someone explain to me what a 'spatially lagged autoregressive model' is ? I came across this 'model' by searching new techniques for modeling geographical data.
0
votes
0answers
21 views

Estimation of real estate

I'm working on a project on estimating the price value of real estate. First I have collected a lot of data (500 000 instances) with details such as postal code, number of bedrooms, build year, ...
0
votes
1answer
24 views

Find optimal least square solution to the normal equation

What is the optimal solution for $\beta_1$ and $\beta_2$ in the following normal equation: $$\beta _{ 1 }\sum _{ i=1 }^{ n }{ { x }_{ i } } +\beta _{ 0 }=\sum _{ i=1 }^{ n }{ { y }_{ i } } $$ EDIT ...
1
vote
1answer
34 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
3
votes
0answers
33 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
1
vote
0answers
20 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
0
votes
0answers
12 views

Derivative and linear fitting model

Let $V=v_1,v_2,\ldots,v_n$ be the measured velocities and $A=a_1,a_2,\ldots,a_n$ be the measured accelerations of a vehicle at times $T=t_1,t_2,\ldots,t_n$. Let $Y=c_1+c_2t+c_3t^2$ be the best ...
1
vote
1answer
22 views

Model selection in regression: Estimated parameters seem to be “non-significant”

I have conducted an experiment which manipulated three factors (Factor 1: 3 levels, Factor 2: 2 levels, Factor 3: 2 levels). The response variable is binomially distributed (1 = correct or 0 = not ...
1
vote
1answer
38 views

Variance- covariance matrix

Consider $H$ denotes hat matrix and $e$ denotes residual. In the book Applied regression Analysis by Draper/Smith, it is written that : $\mathbb V(e_i)$ is given ...
1
vote
1answer
15 views

Dummy recoding for more than two categorical variables

Say I am doing a study with 3 different types of fruit and I want to make a regression depending on the type that tries to predict the amount sold. I know that I could make 2 dummy variables: orange ...
1
vote
1answer
35 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
0
votes
1answer
32 views

Linear regression as $\dim(\beta) \rightarrow \infty$

Consider the linear regression, $$ Y_i = X_i\beta + U_i \qquad E[X_i'U_i]=0 $$ where $X_i=(1,W_{i},W_{i}^2,..\ldots,W_i^K)$ and $\beta \in \mathbb{R}^{K+1}$. The joint distribution of $(X_i,Y_i)$ is ...
0
votes
1answer
22 views

How to create a model with high multicollinearity

I am going to create a model strictly for predictive purposes. Some of my independent variables are highly correlated. When I try to create the regression with all of the variables together, then, I ...
0
votes
1answer
15 views

Can I make one of my predictors categorical and continuous?

I would like to add age as a predictor for my regression, but I would also like to make it a binary categorical predictor with a cutoff of 18 years of age. I would like to do this because I suspect ...
0
votes
0answers
25 views

Metric for movement in 2D space

I have a set of points that represent the coordinates of an object moving in the 2D space at different points in time. Using this points I want to get a "measurement" that will describe the general ...