# Variance of non-linear transformation of regression coefficients

Suppose we are doing multiple linear regression:

$\boldsymbol{Y} = X\boldsymbol{\beta}+\epsilon$

and

$\epsilon \sim N(0,\bar{\Sigma})$

Now, suppose that one of the coefficients is defined using some function, for example

$\beta_1 = ax+bx^2$

where $x$ is some quantity of interest, for example a x-coordinate or something.

Now we can estimate $a,b$ and any other coefficients by maximising the log-likelihood of the normal distribution with respect to these parameters. Using these estimates for $a$ and $b$ we can get estimates for $\beta_1$.

We could also find the covariance matrix between $a,b$ and all other coefficients except $\beta_1$ using the Information matrix. Let this covariance matrix be denoted $\Sigma$.

Now my question is how to we obtain the covariance matrix between all the beta coefficients?

If the transformation of $\beta_1$ is linear in $a$ and $b$ (like above) then we can find the desired covariance matrix using

$A\Sigma A^T$

where A is a matrix describing the linear transformation. However what do we do if the transformation is non-linear in $a$ and $b$?

I know the delta method exists and I can obtain the variance in the $\beta_1$ but how can this be extended to find the covariance between say $\beta_1$ and $\beta_2$?

Example:

Suppose instead of estimating $(\beta_0 \beta_1 \beta_2...,\beta_p)$ we want to estimate

$(\beta_0 , \beta_1=exp(ax^2) , \beta_2...,\beta_p)$

where data has been generated using $\beta_1=exp(ax^2)$. Using the log-likelihood we can estimate the most likely values of

$(\beta_0 , a , \beta_2...,\beta_p)$

and we can use the Information matrix to find the covariance of these parameters. My question is how to find the covariance matrix of

$(\beta_0 , \beta_1 , \beta_2...,\beta_p)$.

If $\beta_1$ was linear in a, we could use some transformation of the normal random variable by multiplying by a matrix A to get the covariance matrix $A\Sigma A^T$, but what now when $\beta_1$ is not linear in a?

A general delta method should get you the whole variance-covariance of $\hat{\beta}$, or if you're using MLE you could use the standard score function approach. For example, you can look up how the standard errors of a logistic or a probit regression are calculated, which are both canonical examples of equations that are non-linear in $\beta$.
More specifically, I'm not sure I understand your restriction on $\beta_1$. Could you edit your question to give a more concrete or clear example?