Derivative of a Stochastic Integral with respect to Limit & with respect to Integrator I have recently come across an attempt to differentiate the following function with respect to $t$ and with respect to $W_t$:
$$F(W_t, t):=\int_{h=0}^{h=t}W_hdW_h$$
Is it possible (i.e. is it "well defined") to differentiate $F$ with respect to $t$ and with respect to $W_t$?
My train of thought would be as follows: by applying Ito's lemma to $W_t^2$, we know that $F_t$ evaluates to $\frac{1}{2}(W_t^2-t)$. Differentiating with respect to $W_t$ would then simply yield $W_t$, whilst differentiating with respect to $t$ would then yield $-\frac{1}{2}$:
Question 1: is the above correct? (I have doubts because $\frac{\partial W_t}{\partial t}$ is undefined. But on the other hand, we are effectively carrying out $\frac{\partial F}{\partial t}$, so following the logic of partial derivatives applied to a function of multiple variables, we should be able to treat all variables "not being differentiated with respect to" as constants).
Question 2: In general, is there a "rule" (that would be an "Ito" equivalent of the Fundamental Theorem of Calculus) for differentiating an Ito integral with respect to its upper limit?
Question 3: What if I now define the function $F(W_1(t),W_2(t),t)$ as follows ($W_1$ and $W_2$ are two independent Brownian motions on the same probability space):
$$F(W_1(t),W_2(t),t):=\int_{h=0}^{h=t}W_1(h)dW_2(h)$$
Would $\frac{\partial F}{\partial t}$, $\frac{\partial F}{\partial W_1}$ and $\frac{\partial F}{\partial W_2}$ be (well) defined?
 A: For question 1, you are correct that $F$ is not differentiable with respect to $t$.  It's not exactly clear what it would mean for $F$ to be differentiable with respect to $W_t$ because in general we don't think of $W_t$ changing without $t$ also changing.  So while $F$ is clearly differentiable with respect to its first argument, I'm not sure that's exactly what you want when you talk about $F(W_t,t)$ being differentiable with respect to $W_t$.
For the second question, which also addresses question 3, we have something very similar to the fundamental theorem of calculus: If $H$ is adapted, right-continuous, and bounded then $$\lim_{t \rightarrow 0} \frac{1}{W_t} \int_0^t H_s dW_s = H_0$$ in probability.
Proof: Fix $K, \varepsilon > 0$.  We compute
\begin{align*}
&\phantom{= }P\left( |\frac 1{W_t}\int_0^t H_s dW_s - H_0| > \varepsilon \right)\\
&= P\left( |\frac 1{W_t}\int_0^t (H_s - H_0)dW_s| > \varepsilon \right) \\
&=  P\left( |\frac 1{W_t}\int_0^t (H_s - H_0)dW_s| > \varepsilon, \frac{|W_t|}{\sqrt{t}} > K \right) + P\left( |\frac 1{W_t}\int_0^t (H_s - H_0)dW_s| > \varepsilon, \frac{|W_t|}{\sqrt{t}} \le K \right) \\
&\le P\left( |\frac 1{W_t}\int_0^t (H_s - H_0)dW_s| > \varepsilon, \frac{|W_t|}{\sqrt{t}} > K \right) + P\left(\frac{|W_t|}{\sqrt{t}} \le K \right).
\end{align*}
Now we focus on the first term and use Markov's inequality:
\begin{align*}
P\left( |\frac 1{W_t}\int_0^t (H_s - H_0)dW_s| > \varepsilon, \frac{|W_t|}{\sqrt{t}} > K \right) &\le P\left( |\frac {1}{K\sqrt{t}}\int_0^t (H_s - H_0)dW_s| > \varepsilon, \frac{|W_t|}{\sqrt{t}} > K \right) \\
&\le  P\left( |\frac {1}{K\sqrt{t}}\int_0^t (H_s - H_0)dW_s| > \varepsilon \right) \\
&\le \frac{1}{\varepsilon^2}\mathbb{E}\left[ \frac{1}{K^2 t} |\int_0^t (H_s - H_0)dW_s|^2\right] \\
&= \frac{1}{\varepsilon^2K^2 t} \mathbb{E}\left[\int_0^t |H_s-H_0|^2 ds \right] \\
&\le \frac{1}{\varepsilon^2K^2 t} \mathbb{E}\left[ t \sup_{0 \le s \le t} |H_s-H_0|^2 \right] \\
&= \frac{1}{\varepsilon^2 K^2}\mathbb{E}\left[\sup_{0 \le s \le t} |H_s-H_0|^2 \right].
\end{align*}
For the second term, by the scaling properties of Brownian motion we have $P\left(\frac{|W_t|}{\sqrt{t}} \le K \right) = P(|X| \le K)$ where $X$ is a standard normal random variable.  Combining everything, we have
\begin{align*}
\lim_{t \rightarrow 0} P\left( |\frac 1{W_t}\int_0^t H_s dW_s - H_0| > \varepsilon \right) &\le \lim_{t \rightarrow 0} \frac{1}{\varepsilon^2 K^2}\mathbb{E}\left[\sup_{0 \le s \le t} |H_s-H_0|^2 \right] + P(|X| \le K).
\end{align*}
Since $H$ is bounded, $\sup_{0 \le s \le t} |H_s-H_0|^2$ is also bounded, and since $H$ is right continuous we know that $\lim_{t \rightarrow 0} \sup_{0 \le s \le t} |H_s-H_0|^2 = 0$ almost surely, so by the dominated convergence theorem we have
\begin{align*}
\lim_{t \rightarrow 0} P\left( |\frac 1{W_t}\int_0^t H_s dW_s - H_0| > \varepsilon \right) &\le \lim_{t \rightarrow 0} \frac{1}{\varepsilon^2 K^2}\mathbb{E}\left[\sup_{0 \le s \le t} |H_s-H_0|^2 \right] + P(|X| \le K) \\
&= P(|X| \le K).
\end{align*}
Finally, sending $K \rightarrow 0$ gives $$\lim_{t \rightarrow 0} P\left( |\frac 1{W_t}\int_0^t H_s dW_s - H_0| > \varepsilon \right) = 0.$$
This result is actually still true if $H$ is continuous but unbounded by localizing $H$ first, so that would be the result you need for question 3.  Let me know if you want me to add on the details for that.
