UPDATED:
Since Apostol's table is imprecise for the latest values let's exhibit this partial table of $n!+\zeta^{(n)}(0)$ values obtained with the method proposed by Gottfried Helms in the comments :
$
\begin{array} {r|l}
n&\qquad n!+\zeta^{(n)}(0)\\
\hline
1 & +0.081061466795327258219670263594382360138602526362217\\
2 & -0.0063564559085848512101000267299604381989949101609199\\
3 & -0.0047111668622544477610608133663752854618076682959801\\
4 & +0.0028968119862920410127804722589943381088600650782966\\
5 & -0.00023290755845472453598583779581974789205717247050230\\
6 & -0.00093682513005092950428350854539855876385290926809868\\
7 & +0.00084982376500166915170602760235121839217676036899380\\
8 & -0.00023243173551155958285569006371686986154745560535153\\
9 & -0.00033058966361229644525612725015921912916311539120160\\
10 & +0.00054323411577970847223198894312031008561943002564803\\
11 & -0.00037549317290726365046703088410553955290852331712733\\
12 & -0.00001960353628101391976648402508433558658818213359962\\
13 & +0.00040724123256303314343212136681027307343924449505289\\
14 & -0.00057049201328177771564129138383813714231765446439354\\
15 & +0.00039392707898120442182766081893948743593101317331900\\
16 & +0.00008345880582550168172764880471555318446251614843452\\
17 & -0.00066094372962859689616940299813405772474841468462821\\
18 & +0.0010262272865408540021770141554688378775983106974390\\
19 & -0.00086557577677928299157607241403657110459312961654081\\
20 & +0.00001929367178370514010632997603577601048054770687535\\
21 & +0.0013569060521345494611491378326511761990288706578281\\
22 & -0.0026921564587532912840342571094899479367185487885538\\
23 & +0.0030513856212416271388454373861585656340439536386835\\
24 & -0.0014242918494185458532221867917952455892341070680451\\
25 & -0.0027077892128860067881974821917555423128848837698589\\
30&-0.026465704147079752693730404859959295339337073188577\\
35&-0.26359445473226969258965859491215128351504627358118\\
40&-2.9912738940588767630327451314666324157450427478360\\
45&-37.811691859847699571345792885440735948937675076443\\
50&-484.41085697391134019683488132115999695787532277743\\
55&-4532.7922577092171518919512255451187936120105777731\\
60&+62714.106769571852549815121861152393947489784478599\\
65&+7023172.7145242778883663789007092296487557987220273\\
70&+369710251.75434261376148718924306570270744599755002\\
75&+16153042555.891600628481729183019107036027090664570\\
80&+615738270543.41976362005501481867360304511761212199\\
85&+18734769337973.535747625469811963057045887984795841\\
90&+236370935383452.03942787310651808117015612065952142\\
95&-26002457205974856.121059702068315722218399244645218\\
100&-3067048412469082717.1320349345687277345600145601427\\
105&-208147105464557539810.93310552061394602332413623631\\
110&-9181100257482418076527.7843319896367738502453996735\\
115&-51947662171852808135142.656616304150605568437147323\\
120&+40156333121359621232445103.065796921480403337792144\\
125&+4885455264162691954362582051.1962929540984191959671\\
130&+326172379219132017786027255436.16366267172881142641\\
135&+5681896814647267766788984138309.9277764944754964868\\
140&-1823873410669202891713087061952487.2923381095183773\\
145&-287161238605183347710570327381611857.62150269361362\\
150&-21305861581790622498949173421790799625.108948639082\\
155&+41341935656925531212500416560539095352.234411848266\\
160&+247591097041903905305863994419088881629306.69527638\\
165&+35417487509305790307439844806554155410647762.2818942813077054202595\\
170&+1939388852429349721510180790653718054320127522.76657886070312620767\\
175&-219609544533102325798714608918968968215179933676.462881353291615995\\
180&-64398214417872662764963987879167602127249665707913.3748997726013798\\
185&-6471529441461413822723169640664516218513802097544790.17826333568547\\
190&+124737730975894951649278632325321300323483372940042824.738170084667\\
195&+146090125339857661850314283330560855583771401129477555170.288406962\\
200&+21761038288742061134507006188990514804372485314425207281255.1444267\\
205&+448206643590051608263691568113493984452862240936661159203824.575986\\
210&-436802309714509751568738654004052940208588317008859449997875439.739\\
215&-87517428053442479414927505503691736597016156193237933277989943662.6\\
220&-3847242990914462888287610077836765513262393392990395867588899745072.\\
225&+1.78354688800770254378644818320484851438306069887890285327456308 \rm{E}69\\
230&+4.39266696649098114932308252500592591975090024455299051343992957 \rm{E}71\\
235&+2.44222360080279317167994898279834010610042190503137084735641980 \rm{E}73\\
240&-1.01144099944259968270241440744065554134825102647611664424324558 \rm{E}76\\
245&-2.70449272645088612156914866282417480363449265831314743378453851 \rm{E}78\\
250&+3.05921285543816770383418771952247557481329946696456084332720656 \rm{E}79\\
\end{array}
$
I fear that this will grow without bounds even if much slower than $n!$ but an asymptotic formula could be conjectured from these values !
Concerning the limit :
$$\lim\sup_{n\rightarrow\infty}\left(\frac{\delta_{n}}{n}\right)^{\frac{1}{n}}$$
I can only show you the 'brainy' picture obtained for values of $n$ from $1$ to $250$ :

The largest value obtained is near $2.047$ but this doesn't seem to stop.
Note that this is nearly the same picture than for $\ \lim\sup_{n\rightarrow\infty}\left(\delta_{n}\right)^{\frac{1}{n}}\ $ (division by $n$ doesn't matter much).
If we observe that the real takeoff of $\delta_n$ waits until $n=25$ then a not too bad approximation of the previous curve is :
$$f(n)=\frac{\sqrt[3]{n-17}}3$$
represented here (for $n$ from $17$ to $250$) :

I tried to divide $\delta_n$ by different expressions in your limit and found :
$$\ \lim\sup_{n\rightarrow\infty}\left(\dfrac{\delta_{n}}{\sqrt[3]{n!}}\right)^{\frac{1}{n}}\ $$

with the interesting 'saturation' near $0.4646$.
//Scripts used (pari/gp) :
//Method proposed by Gottfried Helms (precomputed Stieltjes table)
zs(n)=(-1)^n*sum(k=0,#Stieltjes-n-1,Stieltjes[k+n+1]/k!)
//Direct evaluation of the nth derivative at z (ep= 1E-50 or less)
zp(z,n,ep)=sum(k=0,n,(-1)^k*binomial(n,k)*zeta(z+(n-2*k)*ep))/(2*ep)^n