Hi apologies it's hard to type out the problem,
I have a lecture slide on neural networks. It says the fitting error gives the matrix:
N by M matrix of thi's multiplied by Mx1 weights minus Nx1 outputs (ys)
The Euclidean distance of this entire function is then found. Thi is a function, given at the bottom, w are the corresponding weights and y is the network output.
Following this the slide says "The above minimisation problem is usually solved using the least squares solution based on the pseudoinverse:
w = thi y thi has + above indicating pseudoinverse. "This pseudoinverse is efficiently solved using the singular value decomposition (SVD) techniques".
thi+ = (limit (thi_transpose * thi + lambda * Identity_Matrix)^(-1)) * thi_transpose
The limit is as lamba tends to zero. Lambda is a user-defined regularisation parameter, whereby 0 = no smoothness and infinity gives unreliable results.
I've been reading for ages and still dont understand this last line at all. Please could someone go through an example, or how to solve this? (use what values of thi u want)
Extra information I don't know if it's useful:
Thi(x) = exp( - euc_dist(x-ci) / 2*gamma_squared)
x = network input, c = centre of neuron, gamma = width.
I need to use the least square solution based on the pseudoinverse of an NxM thi matrix multiplied by Nx1 weight matrix minus Nx1 output matrix to find the Nx1 weight matrix.
w = pseudo_inverse_thi * output vector
where pseudo_inverse_thi = limit equation above
The SVD is somehow used to accomplish this. If I make no sense, I apologise, I'm utterly lost.