Questions on (linear or nonlinear) regression, the fitting of functions that best approximate empirical data.

learn more… | top users | synonyms

3
votes
2answers
33 views

Distance between a plane and set of points

Suppose $m$ data points belonging to a class represented by matrix $A$. Therefore, the size of matrix $A$ is $m\times n$. In addition, suppose $w\cdot x + b=0$ be equation of a plane in ...
0
votes
0answers
4 views

Is there a standard formula for calculating a project portfolio elasticity?

My project portfolio has a lot of variable numbers in it which could drastically change the cost, schedule and resource requirements of the portfolio. I was wondering if there is already some kind of ...
2
votes
0answers
19 views

combining multiple regression outputs

Suppose I have multiple regressions, along with their r-squares, standard-errors, etc.: $y(t) = \alpha_1 + \beta_1 x(t) + e_1$, where $t \in (\tau_0, \tau_1)$ $y(t) = \alpha_2 + \beta_2 x(t) + ...
-1
votes
0answers
23 views

Averaging between different formula of same kind

There is a dataset of 8 books. We examined the similarity of each book, using a technique, with other 19 books (8*19 number of distinct books). We stored the data and then, we used ...
-2
votes
0answers
12 views

how to find the corelation or regression between two tables [closed]

I have two tables , I need to find the dependence or relation between these two tables table one ... $3$ sport v/s four regions with value as sales table two .. same $3$ sport v/s same four regions ...
0
votes
0answers
14 views

tree models - adaptive basis functions

i am self-studying adaptive basis functions - and came across the following text on classification and regression trees. I am wondering: as the final region in a tree might be result of multiple ...
1
vote
1answer
20 views

Fit Quantized Piecewise Constant Function to Another Piecewise Constant Function

I have a situation where I have a function $$f(x) : [r_1,r_2]\in\mathbb{R} \rightarrow [r_3,r_4]\in\mathbb{R}$$ and I need to fit a function $$g(x) : [r_1,r_2]\in\mathbb{R} \rightarrow ...
5
votes
1answer
215 views

How to compare the similarity between functions?

I'm designing a web service that finds the regression function of a pattern within an image. I analyzed three images and found the following three regressions: 1) $f(x) = 74.7602 + 0.2005x - ...
0
votes
0answers
40 views

On a modified least square.

Given a vector $y \in \mathbb R^n$ and real constants $x_{ij}$ ($i=1,\dots,n$, $j=1,\dots,p$), we consider a vector $\beta = (\beta_0,\dots,\beta_p)$ which minimize ...
0
votes
0answers
15 views

Is total least square solution only valid for isotropic error

Let $\mathbf{y} = \mathbf{Ax}$ represent a system of equation where, $\mathbf{y}\in\mathbb{R}^n, \mathbf{A}\in\mathbb{R}^{n\times m}$. However due to error in sensor, what we observe is the following ...
1
vote
2answers
50 views

Minimizing a summation?

I have absolutely no idea how to approach this problem. I've been looking through notes, and I think I missed this when my professor discussed this in class. $$ \text{Consider the data}\\ i\: x_i\: ...
1
vote
2answers
37 views

Expected value and the standard simple regression model

Given the standard simple regression model: $y_i = β_0 + β_1 x_i + u_i$ What is the expected value of the estimator $\hat\beta_1$in terms of $x_i, \beta_0$ and $\beta_1$ when $\hat\beta_1=\sum x_i ...
0
votes
2answers
40 views

Approximate/Find Function

I have got values $x_{i}$ and targets $z_{i}$. Now I want to find a function $f(x)=z$, which approximates the mapping of my value $x_{i}$ to its targetvalue $z_{i}$ as good as possible for every ...
1
vote
1answer
24 views

Detrending sine waves accurately

I am doing some data analysis where I look at electricity demand over the course of a day, but need to separate the intra-day (constant and periodic) components from daily changes (assumed linear). At ...
0
votes
0answers
30 views

Simple curve fitting smoothing algorithm

Is there any conditionally simple algorithm to obtain a best fit (equation) for points set? I have a few points for which I want to have a "smooth" function is constructed that approximately fits the ...
3
votes
1answer
25 views

Quantile Regression - Linear Loss Minimization

I'm currently reading Quantile Regression by Roger Koenker, and for some reason I'm having a lot of trouble deriving one of his equation (sect. 1.3, p. 5-6). He goes on to demonstrate that $\hat{x}$ ...
0
votes
0answers
31 views

How to find a perfect regression fit in R?

I have a set of points, which I know can be described with some equation. How can I find this equation? The scatter plot for this set looks like this: I look at the plot and assume that I can use a ...
2
votes
0answers
22 views

GLM for Poisson Regression for Soccer Ratings Not Converging

I have been trying to formulate a model of soccer matches to help me predict the outcomes. The model I'm trying to formulate involves using Poisson regression to assign attack and defence ratings to ...
0
votes
0answers
52 views

How to calculate relation between Beats (as in BPM) and variance of the energies?

I'm working on an implementation of a Beat Detection algorithm and I can't find a relation between the statistical variance of the energies in 1 second of audio decoded as PCM and the threshold value ...
-1
votes
2answers
372 views

Extrapolation with exponential curve

I would like to extrapolate time series using exponential curve while getting the parameters via linear regression. Exponential curve is given as $g=e^{~a + b \cdot t}$. Since I want to use linear ...
0
votes
0answers
36 views

Understanding Piecewise Logistic Function

I am trying to understand the use and application of the piecewise logistic function/regression and have recently been shown how to apply it after getting some help at another forum. However, the ...
2
votes
1answer
75 views

Least-squares solution to a matrix equation?

Suppose I have $n$ observations of $m$ dependent variables $y_1,\dots,y_m$, and I believe they follow some model wherein they can all be written as linear combinations of some underlying variables ...
2
votes
1answer
311 views

Multiple linear regression with interaction

I'm doing a multiple linear regression with interacting variables. I'll give you an example: $y$=value, $x_1$=material, $x_2$=weight, $x_3$=color $x_1$ and $x_2$ are interacting variables but $x_3$ ...
1
vote
3answers
53 views

Transposing matrix when differentiating it

Hi so I am trying to understand the solution of linear regression with matrices (found at the following link) and an confused about how on page 10 he says the derivative of $2Y'XB$ with respect to $B$ ...
1
vote
1answer
394 views

Derivative of logistic loss function

I am using logistic in classification task. The task equivalents with find $\omega, b$ to minimize loss function: That means we will take derivative of L with respect to $\omega$ and $b$ (assume y ...
3
votes
0answers
19 views

Regression with many discrete and continuous predictors and few rows

I want to do regression on a dataset. It has one continuous dependent variable that I want to predict. It has many categorical and some continuous predictors. It only has a few rows. A simplified ...
2
votes
1answer
22 views

Multiple regression model

I have a multiple regression equation which as four quarters (maybe called them as parameters) ...
0
votes
0answers
10 views

Softmax Regression Gradient Derivation

I'm implementing softmax regression and am deriving the max-log-likelihood update for gradient descent by hand first. Coming from the Stanford UFLDL site, they show the gradient of the cost function ...
1
vote
2answers
56 views

properties of least square estimators in regression

$Y_i=\beta_0+\beta_1 X_i+\epsilon_i$ where $\epsilon_i$ is normally distributed with mean $0$ and variance $\sigma^2$ . The least square estimators of this model are $\hat\beta_0$ and $\hat\beta_1$. ...
1
vote
0answers
17 views

LRT and Wald Test in Multivariate Linear Regression significance

I've been researching all afternoon trying to get a better idea of what the Likelihood ratio test and the Wald test are actually doing. I have a bunch of covariates and I'm testing out like 30 ...
0
votes
0answers
19 views

Finding the curvature from a set of datapoints

I have a set of 1. 1-d 2. 2-d data. I want to find the curvature at each single point. Till now I was using difference technique to find out the curvature, i.e, central difference at middle and ...
1
vote
1answer
25 views

Significance of dummy variables in probit regression

I am an undergraduate student working on some projects using probit regression. I have a question on dummy variables that I was hoping someone could help me with (which I think stems from an ...
1
vote
1answer
14 views

Find a distribution for this plot

Please help me find a formula that fits the distribution. It does not need to be exact, a simple approximation would suffice. Bonus points if you can tell me which predefined distribution in the ...
0
votes
2answers
155 views

Non-linear regression - least square regression

I was trying to get some insight into how to solve non-linear regression model problems. Unfortunately, I've never attended a lecture on statistical math. Here is the link: In page number 4, they ...
0
votes
1answer
23 views

finding a question about constrained regression - had a side constraint $x \geq y^*z$

I saw a question that asked about solving a non-negative least squares problem with $3$ unknowns, $(x,y,z)$. But there was an additional constraint, $x \geq y^*z$. Would appreciate getting the ...
0
votes
1answer
24 views

Getting VAR parameters from a research paper.

Many econometrics papers provide the parameters used in their VAR model. If I notate my VAR model as $$z_{t+1} = c + B z_{t} + \Sigma \epsilon_{t+1}$$ where $\epsilon \sim N(0, I)$, then I need to ...
0
votes
2answers
79 views

Non-linear regression fit

I'm trying to fit my data to the following equation: $$ Y = A(1-2e^{bx}) $$ What I tried to do was transform the equation to a linear form via the following steps: \begin{align*} & A-Y = ...
0
votes
1answer
29 views

Least squares with known error in y

so I want to do a linear least squares regression on my data, however I have known experimental error on my data points in $y$ and relatively few numbers of points so I would like to use values ...
0
votes
1answer
31 views

How do I solve for vector $P$ in the matrix equation $s=A'B^{-1}A$?

I would like to rearrange the matrix equation $s=A'B^{-1}A$ into the form $A=f(s,B)$ (i.e., some function of $s$ and $B$), where s is scalar, $A$ is $n\times 1$, $A'$ is the transpose of $A$, and $B$ ...
1
vote
1answer
157 views

Optimizing Independent Variables to Maximize Dependent Variable

I looked around online and couldn't find anything that was answering my question so I thought I would take to the stack! I'm interested in knowing if there is a statistical or mathematical way of ...
1
vote
3answers
51 views

How come least square can have many solutions?

I know there always exists a least-square solution $\hat{x}$, regardless of the properties of the matrix $A$. However, I keep finding online that least-square can have infinitely many solutions, if ...
0
votes
1answer
9 views

In a simple regression model estimated using OLS, the covariance between the estimated errors and regressors is zero by construction

Is this statement true or false? I seem to remember that this relationship does not hold when the regression has no intercept, however my teacher said that this was true regardless of whether we ...
2
votes
1answer
395 views

Understanding Regularization parameters in Machine Learning/Statistics

Suppose I have the following $k$ degree polynomial regression model with a data set of size $n$ which includes a $k$-dimensional feature vector $x$ and an outcome denoted $t_i$ for each vector in the ...
-1
votes
0answers
21 views

(x/y) v (y/x) as predictor in regression

I am trying to predict a variable 'score' with x and y. I believe it is related to (x/y)/z and z/(x/y), but I'm not sure which. Here is some data: ...
0
votes
1answer
27 views

Polynomial least squares fit — restrictions on order?

If we're finding an interpolating polynomial for 10 data pairs, the order of the polynomial has to be 9. In class, my professor said that when doing a polynomial least squares fit, if you have 10 ...
1
vote
2answers
65 views

Fit exponential with constant

I have data whic would fit to an exponential function with a constant. So y=aexp(bt) + c Now I can solve an exponential without a constant using least square by taking log of y and making the ...
3
votes
0answers
28 views

How to perform nonlinear regression with regressors affected by gaussian error?

I am trying to calibrate a sensor and I have a data set consisting of several observations of a 3-dimensional vector $X_i$, with $X_i=w_i + \epsilon_i$ where $w_i$ is the value that the sensor ...
0
votes
1answer
11 views

Least Squares Estimators Derivation (Bi-variate)

To derive least square estimators: We have $SS(\alpha,\beta)= \sum(y_i-\alpha-bx_i)^2$ and find partials for each. The answer I get is: $\beta = \frac{\sum y_i-\bar{y}}{\sum x_i-\bar{x}}$, but the ...
1
vote
1answer
63 views

Why do the components of an equivalent kernel sum to 1?

Let $\textbf{x} = (x_1, \dots, x_n)^T \in \mathbb{R}^n$ and $k \in \mathbb{N}$. We define $$ X := \begin{pmatrix} 1 & x_1 & \cdots & x_1^k \\ \vdots & \vdots & & ...
2
votes
1answer
59 views

Outlier detection with robust multiple regression model

I have a set of features (eg, location, income, budget, education) that I use to predict a continuous variable (say, amount spent per day on the internet). I am interested in detecting outliers. I ...