Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

0
votes
0answers
19 views

How to solve this linear algebra equation / regression?

$Y = X D$ and $Y$ is $n\times 1$ known matrix. $X$ is $n\times k$ unknown matrix. $D$ is $k\times 1$ known matrix. How to solve for $X$?
3
votes
3answers
61 views

linear solution of curve fitting on multiple linear functions differing by a multiplier

I recently posted this question here but I thought this could be of interest also in mathematics, given I found a partially related question here I am facing the following problem. I know nonlinear ...
0
votes
1answer
50 views

Examining the effect of a quantitative factor on response.

To examine the effect of a quantitative factor temperature on yield,the researcher has a plan to use the following model for the analysis: $$y_{ix}=\beta_0+\beta_1 x+\epsilon_{ix}$$ where $y_{ix}$ ...
0
votes
1answer
29 views

Constructing a while loop in R for Newton's method

I'm very, very new to R, and my instructor's example seemed like a special case (or I just don't know how to extrapolate his syntax to my problem). Here's the example we used: Approximate the square ...
1
vote
1answer
41 views

Least Squares Regression Matrix for Rational Functions

So first off no, this isn't a homework problem. Second, I'm trying to understand how this works, NOT find a program that will do it for me. Okay so I've known for a while how to use Gaussian-Jordan ...
0
votes
0answers
17 views

How to find a mapping function from n dimensional space to m dimensional space

What are different methods/approaches to find a function $T:x_1 \to x_2$ that minimizes a cost function defined as: $\text{cost}(T) = | Q(x_1) - P(x_2) | = | Q(x_1) - P(T(x_1) |$ where, $x_1$ has ...
1
vote
1answer
20 views

Regression Model with even function?

Is there any method to test if the mean function, $f(x)$, of a regression model $y=f(x)+\epsilon$ is even or not?
1
vote
0answers
19 views

Orthogonalization of Variables

Let's assume we have three collinear variables/factors, $X_1$, $X_2$, $X_3$. Would there be a method to orthogonalize these variables in a simultaneous way: in other word, orthogonalizing them in such ...
0
votes
0answers
24 views

What is distinction between Functional Linear Regression and Functional Linear Models?

Thanks in advance for the help. I want to make sure that I understand two concepts correctly. A functional linear model is a particular type of linear model while functional linear regression is the ...
0
votes
1answer
34 views

Given a sample of input/output data, predict new outputs

My problem is the following : I have a number of inputs with the corresponding deterministic outputs. There is no error on either input or output. The link between the two is completely unknown to me. ...
1
vote
2answers
41 views

Errors and Residual

Why are errors independent but residuals dependent? As far i know the sum of the residuals within a random sample is necessarily zero, and thus the residuals are necessarily not independent. But also ...
1
vote
2answers
26 views

Simple linear regression seems off

I have the following datapoints: $$p1(52,730)$$ $$p2(53,409)$$ $$p3(52,250)$$ $$p4(52,90)$$ Now I want to find the best fitting line between these points. When I use simple linear regression I get $$y ...
3
votes
1answer
45 views

Weighted least squares with angular data

Suppose I have a system whose state is $\Theta=(\theta_1,\theta_2,\ldots,\theta_n)$, where $\theta_i\in[-\pi,\pi)$ (i.e., they are angles). I'd like to determine the most likely estimate of $\Theta$ ...
3
votes
2answers
104 views

Regression with error coming from rounding

I am looking at the following model: $c$ is a fixed vector in $\mathbb{R}_+^n$ and for any $x \in \mathbb{R}_+^n$ we obtain a value $y =[c^Tx]$, i.e. rounding $c^Tx$ to the nearest integer. I want ...
0
votes
0answers
22 views

numerical (script) fit of function with 2 arguments

I would like to find the least-square fit for a 1D-function that takes two arguments. m(x,y) = d * (x-x0)^2 / (y-y0)^2 I would like to write a c++ routine to ...
1
vote
2answers
43 views

orthogonal matrices vs. orthogonal columns

I'm just reading a book on econometrics and now I'm stuck with a problem: There is a Theorem on "Orthogonal Partitioned Regression" which says: "In the multiple linear least squares regression of ...
2
votes
1answer
49 views

Is there a site that will allow me to calculate a best fit for a set of data?

I have a bunch of x's and their corresponding y values, but do not have a Wolfram Pro account. Is there another site where I can input my dataset and have it spit out a best-fit regression (be it ...
0
votes
1answer
20 views

Minimum number of observations in non-linear regression

Are 5 observations enough to verify the following non-linear regression model in the form: $ Y= C K_0^{\alpha_0}K_1^{\alpha_1}K_2^{\alpha_2}$ And in general how many observations do I need for models ...
0
votes
0answers
17 views

factor models and using cross sectional regression?

I have been doing some reading on factor models. In the literature it mentions that when creating a portfolio that maximises particular attributes it may lead to unwanted bias to other factors. I ...
0
votes
2answers
52 views

Equation going infinitely towards y = 10

I'm programming a site on which I sell services, and the more the customers spends, the more discount they will have. Please have a look at the diagram below. Spending 700 USD will result in 5% ...
1
vote
1answer
78 views

Modeling non-linear data using least squares best fit

I have some data for liquid viscosity as a function of pressure and temperature. I would like to learn how come up with a single equation that would determine this fluid's viscosity with pressure and ...
1
vote
0answers
16 views

Non-Linear Regression involving the maximum function

How do you calculate the regression of this model? I know Minitab and MATLAB, so if you guide me with these software I would totally appreciate it. $$Y=c+\max(X^{-n}, 0.34) $$ Here c and n are ...
1
vote
3answers
53 views

Linear regression throug the origin versus mean?

Assume that I have data that can be described by: $y_i = \beta x_i + \epsilon_i, \epsilon_i \sim (0,\sigma_{\epsilon})$, then the least squares estimator is given by $\hat{\beta_1} = ...
1
vote
2answers
136 views

How do I find equation of this curve?

I need to find equation of the curve as shown below, for which, I need to find equation for upper part. lower part is half circle. upper part is a constant distance from circle with line passing ...
1
vote
0answers
25 views

Good MSE doesnt imply good prediction in logistic regression?

I am writing some code for regularized logistic regression. I observe this interesting phenomena and wonder if it is a normal thing or just my code is wrong. For loss function, I am using the ...
0
votes
0answers
22 views

Nearest points / residuals on a total least squares parabola

Consider fitting a parabola $y = a + bx + cx^2$ to 2d data $X_i, Y_i$ with noise in both X and Y, using the the singular value decomposition as in Total_least_squares (TLS): $\qquad X = [ 1\ \ Xdata\ ...
1
vote
1answer
30 views

time-series regression with missing data

I have a regression as follows for time-series data (e.g. stock prices versus other variables): $$ Y = b \cdot X + b_1 \cdot X_1 + e$$ where $X_1$ will be missing based on pre-determined dates ...
2
votes
0answers
13 views

Showing Hat matrix equal specific values

Consider a one way layout model $y_{ij}$ = $\mu_i + e_{ij}$ (1 $\leq$ i $\leq$ a, 1 $\leq$ j $\leq$ $n_i$) where a = 3 and $n_1$ = 2, $n_2$ = 3, $n_3$ = 4. Show that the hat matrix for this design ...
0
votes
1answer
74 views

Why use regularization to reduce over-fitting

I'm having trouble understanding why should we use regularization for over-fitting when we can simply reduce the number of order to our polynomial function? Is it because it saves us time from having ...
0
votes
2answers
46 views

How to do non-linear regression with this function

I have observed that my data matches the function : $ a e^{bx}+c $ I want to get the parameters a ,b and c. I know how to solve this problem if c equals 0. But how to solve it when c involves in?
0
votes
1answer
55 views

Regression with Mean, Standard Deviation, Range and Correlation

A research team collected data on students in a statistics course. Their dependent variable was the student’s score on the final examination, which ranged from 200 to 800 points. The observed average ...
0
votes
0answers
11 views

Finding rounding rules underlying rounded function results

Here's the problem I've been stuck on for two weeks now. I basically have a black-box function, which takes into input an integer and returns an integer, and I'm trying to determine it. Here are the ...
0
votes
1answer
38 views

4 points, how to know if it's growing over time?

I've an array of 4 points, which formula should I use to detect their growth ? ...
0
votes
1answer
22 views

Deriving estimators for the parameters a and b that minimize the random error - setting up linear regression variables?

I'm reviewing old notes, and I know I solved this way back when, but can't remember how to know: Consider the simple linear regression model: $$Y_i = a + bX_i + \epsilon_i$$ where $Y_i$ is the ...
1
vote
0answers
40 views

Deriving cost function using MLE :Why use log function?

I am learning machine learning from Andrew Ng's open-class notes and coursera.org. I am trying to understand how the cost function for the logistic regression is derived. I will start with the cost ...
0
votes
2answers
41 views

Why divide by $2m$

I'm taking a machine learning course. The professor has a model for linear regression. Where $h_\theta$ is the hypothesis (proposed model. linear regression, in this case), $J(\theta_1)$ is the cost ...
1
vote
1answer
25 views

Finding best predictors of a classification function

I have a large dataset where each element has a number of "input" categories that are either present or not (or if you like, true or false, 1 or 0 etc). Each one also has an output category, again a ...
0
votes
0answers
45 views

Quantile as solution to minimization problem

I'm studying basics of quantile regression now and I have trouble prooving that $\tau-$th quantile of real-valued random variable $Y$ is a solution to the following minimization problem (in the ...
0
votes
0answers
27 views

How to calculate the coordinate of a point which depends on other points on a plane with specific distances

I have $8$ points on a plane $(x_1,y_1)....(x_8,y_8)$ among these $8$ points I know the coordinates for $7$ points and I have to find the $8^{th}$ point. Each points has the difference between all ...
0
votes
0answers
10 views

Ordered logistic regression with likert scales

I'm currently have a bit of difficulty determining how to analyze this data via logistic regression analysis. Q18 = DV (satisfaction score ranging from 1-10) Q10_1 = IV (Customer Service likert ...
0
votes
1answer
20 views

Regression when the variance of the residuals depends on the independent variable

When the residuals follow a normal distribution, the most likely function that fits the data is found using least squares. In that case: $y = f(x_i) + r_i, \quad r\sim\mathcal{N}(0, \sigma^2)$ ...
0
votes
1answer
16 views

Contribution of each variable in multiple linear regression

What will be the best measure of the contribution of a variable in multiple linear regression? I was thinking of using the coefficient ratio as a marker of a variable's contribution. For example: ...
4
votes
2answers
84 views

Rewriting the matrix equation $AX = YB$ as $Y = CX$?

Is it possible in general, if $A,B,C,X,Y$ are square and of the same dimensions? If so, does it generalize to non-square matrices (using a pseudoinverse)? I'm doing some curve fitting in which I have ...
1
vote
0answers
13 views

Finding posterior of normal distributions and logistic regression.

$P(w_0 | x) = \frac{1}{1 + e^{-log\frac{P(x|w_0)}{P(x|w_1)}-log\frac{P(w_0)}{P(w_1)}}}$ Note: x = $[x_1, \dots, x_d]^T$; a $d$ dimensional vector. $w$ can take on one of two values: $w_0$ or $w_1$. ...
0
votes
0answers
15 views

Spatially model

Can someone explain to me what a 'spatially lagged autoregressive model' is ? I came across this 'model' by searching new techniques for modeling geographical data.
0
votes
1answer
32 views

Find optimal least square solution to the normal equation

What is the optimal solution for $\beta_1$ and $\beta_2$ in the following normal equation: $$\beta _{ 1 }\sum _{ i=1 }^{ n }{ { x }_{ i } } +\beta _{ 0 }=\sum _{ i=1 }^{ n }{ { y }_{ i } } $$ EDIT ...
1
vote
1answer
89 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
3
votes
0answers
35 views

Bayesian linear regression cost function

I am studying classification using linear regression . Now, I want to map it in Bayesian regression. Let talk about binary classification using linear regression again. Assume that I have a set ...
1
vote
0answers
34 views

How to represent the parameters in logistic function

I want to find the parameters in logistic function. I read the guide at here. It very clear to explain. But it did not has final solution that I need. Now, we will consider a basis logistic function ...
0
votes
0answers
15 views

Derivative and linear fitting model

Let $V=v_1,v_2,\ldots,v_n$ be the measured velocities and $A=a_1,a_2,\ldots,a_n$ be the measured accelerations of a vehicle at times $T=t_1,t_2,\ldots,t_n$. Let $Y=c_1+c_2t+c_3t^2$ be the best ...