Calculation with small values of $n,d,k\geq 0$ shows the following is valid
\begin{align*}
\sum_{j=0}^{d}(-1)^{d-j}\binom{d}{k}\binom{n-j-1}{n-d-1}\binom{n-d}{j-k}
=
\begin{cases}
1&\qquad 0\leq k=d<n\\
0&\qquad \text{otherwise}
\end{cases}
\tag{1}
\end{align*}
Note: It's easy to prove the special case $k=d<n$. In order to show that the expression is zero otherwise we use techniques known as formal residual calculus for power series. They are based upon Cauchys residue theorem and were introduced by G.P. Egorychev (Integral Representation and the Computation of Combinatorial Sums) to compute binomial identies.
We start with the easy one
Case: $0\leq k=d<n$
If $k=d$ we obtain
\begin{align*}
\sum_{j=0}^{d}&(-1)^{d-j}\binom{d}{k}\binom{n-j-1}{n-d-1}\binom{n-d}{j-k}\tag{2}\\
&=\sum_{j=k}^{d}(-1)^{d-j}\binom{d}{k}\binom{n-j-1}{n-d-1}\binom{n-d}{j-k}\\
&=(-1)^{d-d}\binom{d}{d}\binom{n-d-1}{n-d-1}\binom{n-d}{0}\\
&=1
\end{align*}
This proves the first part of (1).
Recall that a binomial coefficient $\binom{n}{k}$ is zero if $k<0$ or $n<k$. We observe in (2) the the index $j$ of the rightmost binomial coefficient $\binom{n-d}{j-k}$has to be greater or equal $k$ otherwise the summand contributes zero. So, the lower bound of the index $j$ is $k$.
We also note that the binomial coefficient $\binom{n-d-1}{n-d-1}$ is equal to $1$ only in the case $n-d-1\geq 0$ which means $d$ less than $n$ and the easy part of (1) is shown.
We also see that $\binom{d}{k}$ is zero if $d$ is less than $k$. So, the only case which remains is
Case: $0\leq k<d$
We use the residue notation and write e.g.
\begin{align*}
\mathop{res}_z\frac{(1+z)^{n}}{z^{k+1}}=\frac{1}{2\pi i}\oint_{|z|=1}\frac{(1+z)^{n}}{z^{k+1}}\mathop{dz}\tag{3}
=[z^k](1+z)^n=\binom{n}{k}
\end{align*}
In fact we will use only two aspects of this theory:
Let $A(z)=\sum_{j=0}^{\infty}a_jz^j$ be a formal power series, then
- Write the binomial coeffients as residuals of corresponding formal power series
\begin{align*}
\mathop{res}_{z}\frac{A(z)}{z^{j+1}}=a_j\tag{4}
\end{align*}
- Apply the substitution rule for formal power series:
\begin{align*}
A(z)=\sum_{j=0}^{\infty}a_jz^{j}=\sum_{j=0}^{\infty}z^j\mathop{res}_{w}\frac{A(w)}{w^{j+1}}\tag{5}
\end{align*}
\begin{align*}
\binom{d}{k}\sum_{j=k}^{d}&(-1)^{d-j}\binom{n-j-1}{n-d-1}\binom{n-d}{j-k}\tag{6}\\
&=\binom{d}{k}\sum_{j=0}^{\infty}(-1)^{d-j}\mathop{res}_{z}\frac{(1+z)^{n-j-1}}{z^{n-d}}
\mathop{res}_{u}\frac{(1+u)^{n-d}}{u^{j-k+1}}\tag{7}\\
&=(-1)^d\binom{d}{k}\mathop{res}_{z}\frac{(1+z)^{n-1}}{z^{n-d}}
\sum_{j=0}^{\infty}\left(\frac{-1}{1+z}\right)^j \mathop{res}_u\frac{(1+u)^{n-d}u^k}{u^{j+1}}\tag{8}\\
&=(-1)^d\binom{d}{k}\mathop{res}_{z}\frac{(1+z)^{n-1}}{z^{n-d}}
\left(1-\frac{1}{1+z}\right)^{n-d}\left(\frac{-1}{1+z}\right)^k\tag{9}\\
&=(-1)^{d+k}\binom{d}{k}\mathop{res}_{z}(1+z)^{d-k-1}\\
&=(-1)^{d+k}\binom{d}{k}[z^{-1}](1+z)^{d-k-1}\tag{10}\\
&=0
\end{align*}
Since $(1+z)^{d-k-1}$ has not a non-zero power of $z^{-1}$, the expression (10) is zero. This proves the second part of (1)
Comment:
In (7) we extend the upper limit of the sum without changing the value since we are only adding zeroes and we rewrite the binomial coefficients using residues according to (3) and (4)
In (8) we do some rearrangements to prepare for the substitution rule
In (9) we apply the substitution rule according to (5)
\binom{d}{k}is no problem. The only time when the sum is nonzero is when d=k, and so the sum must be a multiple of 1; once again, no problem. $\endgroup$