This coend actually does exist: it is just $0$. Morally, the idea is an Eilenberg swindle. Suppose we had a trace operation $tr$ defined for all vector spaces, and let $A$ be any endomorphism of a vector space.
Then if you take an infinite direct sum $B$ of copies of $A$, $tr(B)$ should be an infinite sum of copies of $tr(A)$. But if you add one more term $tr(A)$ to that infinite sum, it won't change the sum. But that means $tr(B)=tr(B)+tr(A)$ and so $tr(A)$ must be $0$.
To make this idea precise, let us recall that the coend (if it exists) is the initial vector space $V$ together with a linear map $tr_X:[X,X]\to V$ for each vector space $X$ such that for any pair of linear maps $A:X\to Y$ and $B:Y\to X$, $tr_X(BA)=tr_Y(AB)$. To prove that $V=0$ has this universal property, it suffices to show that any such collection of linear maps $tr_X:[X,X]\to V$ for any $V$ satisfies $tr_X=0$ for all $X$.
So, suppose we have a vector space $V$ and such linear maps $tr_X:[X,X]\to V$. Fix a vector space $X$ and $A\in [X,X]$; we will show that $tr_X(A)=0$. To do this, let $Y=X^{\oplus\mathbb{N}}$ and define $R:Y\to Y$ by $$R(x_0,x_1,x_2,\dots)=(0,Ax_0,Ax_1,Ax_2,\dots)$$ and $L:Y\to Y$ by $$L(x_0,x_1,x_2,\dots)=(x_1,x_2,x_3,\dots).$$ We then have $tr_Y(RL)=tr_Y(LR)$, so $tr_Y(LR-RL)=0$. However, notice that $LR-RL$ is given by the formula $$(LR-RL)(x_0,x_1,x_2,\dots)=(Ax_0,0,0,\dots).$$ Now let $i:X\to Y$ be the inclusion of the first summand and $p:Y\to X$ be the projection onto the first summand. We have $$tr_X(piA)=tr_Y(iAp).$$ But $piA=A$ and $iAp=LR-RL$ by the formula above, so this means $$tr_X(A)=tr_Y(LR-RL)=0,$$ as desired.