How to prove that $\arg\max_{t}\langle \sum w_i a_{t_i}, a_t\rangle$ is close to $t_1$? Let $a_t$ be a Gaussian type function with center $t\in \mathbb R^d$, i.e.
$$a_t(x)=e^{-||x-t||^2}, \text{ for } x \in \mathbb R^d.$$
Let $b$ be a combination of these Gaussian functions
$$b = \sum_{i=1}^s w_i a_{t_i}$$
where $t_i \in \mathbb R^d$ for all $i=1,..., s$ and $w_1 > .... > w_s>0$ is a finite sequence of decreasing positive numbers.
Define $$\hat t = \arg\max_{t\in \mathbb R^d} \langle b, a_t\rangle.$$
Here the inner product $\langle b, a_t\rangle$ is defined as $\int_{x\in \mathbb R^d} b(x) a_t(x)dx$.
Intuitively, we can easily see that if the Gaussian functions are far away of each others, then $\hat t$ will be close to $t_1$, i.e. we have

Claim. If $\Delta= \min_{i\neq j; i, j\leq s}||t_i-t_j||\rightarrow +\infty$ then $\hat t \rightarrow t_1$.

I am wondering how to prove the claim rigorously. I have no idea since the function $\langle b, a_t\rangle$ is not even convex. Is there any idea to tackle this problem? Thanks!
 A: Define $D$ as the following constant:
$$ D= \int_{x \in \mathbb{R}^d} e^{-2||x||^2}dx$$
For the norm $||x||^2 = \sum_{i=1}^d x_i^2$ we have the following fact: For any $x, a, b \in \mathbb{R}^d$:
$$ ||x-a||^2 + ||x-b||^2 = 2||x-\frac{(a+b)}{2}||^2 + \frac{1}{2}||a-b||^2$$
Define $g:\mathbb{R}^d\rightarrow\mathbb{R}$ by
$$ g(t) = \int_{x \in \mathbb{R}^d} b(x) e^{-||x-t||^2}dx$$
Define
$$g^* = \sup_{t \in \mathbb{R}^d} g(t)$$
Then for any $t \in \mathbb{R}^d$ we get
\begin{align}
g(t) &= \sum_{i=1}^s w_i \int_{x \in \mathbb{R}^d} e^{-||x-t_i||^2} e^{-||x-t||^2} dx\\
&\overset{(a)}{=}\sum_{i=1}^s w_i \int_{x \in \mathbb{R}^d}e^{-\left[2||x - \frac{(t+t_i)}{2}||^2 + \frac{1}{2}||t-t_i||^2 \right]} dx\\
&= \sum_{i=1}^s w_i e^{-\frac{1}{2}||t-t_i||^2}\int_{x \in \mathbb{R}^d} e^{-2||x-\frac{(t+t_i)}{2}||^2}dx\\
&= D\sum_{i=1}^s w_i e^{-\frac{1}{2}||t-t_i||^2}
\end{align}
where step (a) uses our fact about norms.  In particular:
$$g^*\geq g(t_1) > D w_1$$

For each $r>0$ define $A_i(r)$ as the closed ball of radius $r$ about $t_i$:
$$ A_i(r) = \{x \in \mathbb{R}^d : ||x-t_i||\leq r\}$$
Fix $\epsilon>0$ and suppose $\epsilon$ is sufficiently small to ensure that $Dw_1>\epsilon$ and $Dw_1> Dw_2+\epsilon$. Fix $r>0$ big enough to ensure $Dsw_1e^{-r^2/2}\leq \epsilon$. Suppose the center points $t_1, ..., t_s$ are far enough apart so that the sets $A_1(r), A_2(r), ..., A_s(r)$ are disjoint.

*

*Case 1:  Suppose $t \notin \cup_{i=1}^d A_i(r)$.  Then $||t-t_i||\geq r$ for all $i$ and so
\begin{align}
g(t) &\leq D \sum_{i=1}^s w_i e^{-\frac{r^2}{2}} \\
&\leq D s w_1 e^{-\frac{r^2}{2}} \\
&\leq \epsilon \\
&< Dw_1 
\end{align}


*Case 2: Suppose $t \in A_k(r)$ for some $k \in \{2, ..., s\}$.  Then $||t-t_j||\geq r$ for all $j \neq k$ and so
$$ g(t) \leq Dw_k + D(s-1)w_1e^{-\frac{r^2}{2}}  \leq Dw_2 + \epsilon <Dw_1$$
It follows that if $t \notin A_1(r)$ then $g(t)<Dw_1<g^*$.  So the supremum value $g^*$ can only be approached over points in the compact set $A_1(r)$.  Since $g$ is continuous, it follows that the supremum can be achieved, and any maximizer point $t^*$ must satisfy $t^* \in A_1(r)$. Now for any point $t \in A_1(r)$ we have $||t-t_j||\geq r$ for all $j \in \{2, ..., s\}$ and so
\begin{align}
g(t) &\leq Dw_1 e^{-\frac{||t-t_1||^2}{2}} + Dsw_1e^{-r^2/2} \\
&\leq Dw_1 - Dw_1(1- e^{-\frac{||t-t_1||^2}{2}}) + \epsilon\\
&< g^*- Dw_1(1- e^{-\frac{||t-t_1||^2}{2}}) + \epsilon
\end{align}
In particular, if $t^*$ is a maximizer, then $t^* \in A_1(r)$ and so
$$ g^* = g(t^*) < g^* - Dw_1(1- e^{-\frac{||t^*-t_1||^2}{2}}) + \epsilon$$
In particular
$$ Dw_1 (1- e^{-\frac{||t^*-t_1||^2}{2}}) < \epsilon $$
Define $\delta = \frac{\epsilon}{Dw_1}$ and note by our assumptions that $0<\delta < 1$.  Define $z = \frac{||t^*-t_1||^2}{2}$. It follows that $e^{-z} >1-\delta$, so
\begin{align}
z &< -\log(1-\delta) \\
&= \log(\frac{1}{1-\delta})\\
&= \log(1 + \frac{\delta}{1-\delta})\\
&\leq \frac{\delta}{1-\delta}
\end{align}
In particular, if $t^*$ is a maximizer then
$$ \frac{||t^*-t_1||^2}{2} \leq \frac{\delta}{1-\delta} = \frac{\epsilon}{Dw_1 - \epsilon}$$
A: Michael proposed an amazing answer to the question. As you can see, his answer contains two parts. In this answer, I re-use his idea in the first part and reformulate the second part by 1) using tight inequalities and 2) using only parameter $r$ as a varying parameter. I also addressed the exponential convergence of $\hat t$ at the end of the proof.

Let $W = \sum_{i=1}^s w_i$ and $\Delta = \min_{i\neq j} ||t_i-t_j||>0$. Define $\color{blue}{r=\frac{\Delta}{2}}$.
Case 1. Let $t\in A_k(r)$ for some $k\in \{ 2,...,s\}$. The condition $r=\Delta/2$ implies that $||t-t_j||\geq r$ for all $j\neq k$ and $||t-t_k||\leq r$. Therefore
\begin{equation}
\begin{aligned}
    g(t)
    & \leq  Dw_ke^{-\frac{||t-t_k||^2}{2}} + D(W-w_k)e^{-r^2/2}\\
    & = Dw_k ( e^{-\frac{||t-t_k||^2}{2}} - e^{-r^2/2} ) + DW e^{-r^2/2}\\
    & \leq Dw_2 ( e^{-\frac{||t-t_k||^2}{2}} - e^{-r^2/2} ) + DW e^{-r^2/2}\\
    & \leq Dw_2 ( 1- e^{-r^2/2} ) + DWe^{-r^2/2}\\
    & = Dw_2 + D(W-w_2)e^{-r^2/2}\\
    & =: f(r).
\end{aligned}
\end{equation}
Case 2. For $t\in A=\mathbb R^d \setminus \bigcup_{i=1}^s A_i(r)$, we know that $||t-t_i||> r$ for all $i\in \{1,...,s\}$,
\begin{equation}
\begin{aligned}
    g(t)
    \leq  DWe^{-r^2/2} \leq f(r).
\end{aligned}
\end{equation}
From Case 1, Case 2 and the fact that $\mathbb R^d\setminus A_1(r) \subset A \cup A_2(r) \cup ...\cup A_s(r)$, we have
\begin{equation}
\begin{aligned}
    \sup_{t\in \mathbb R^d \setminus A_1(r)} g(t)
    & \leq  \sup_{t\in A \cup A_2(r) \cup ...\cup A_s(r)} g(t)\\
    & \leq f(r)\\
    & \color{blue}{<} f(r_1)\\
    & = Dw_1
\end{aligned}
\end{equation}
Here, we further assume that $\color{blue}{r>r_1}$ where $r_1$ is the positive constant such that $f(r_1)=Dw_1$. Note that $r_1$ is well defined since $w_1>w_2$ and $D>0$.
Together with the fact that $g(\hat t) \geq g(t_1) \geq Dw_1$, we conclude that $\sup_{t\in \mathbb R^d} g(t)$ can achieve its maximum value at a point, say $\hat t$, belonging to a compact set $A_1(r)$.
Since $\hat t \in A_1(r)$. We know that
\begin{equation}
\begin{aligned}
    g(\hat t)
    & \leq  Dw_1 e^{-\frac{||\hat t-t_1||^2}{2}} + D(W-w_1)e^{-r^2/2}\\
    & = Dw_1 - Dw_1 ( 1- e^{-\frac{||\hat t-t_1||^2}{2}} ) + D(W-w_1)e^{-r^2/2}\\
    & \leq g(\hat t) -  Dw_1 ( 1- e^{-\frac{||\hat t-t_1||^2}{2}} ) + D(W-w_1)e^{-r^2/2}.
\end{aligned}
\end{equation}
Remove $g(\hat t)$ in both sides, we have
\begin{equation}
\begin{aligned}
    Dw_1( 1- e^{-\frac{||\hat t-t_1||^2}{2}} ) \leq D(W-w_1)e^{-r^2/2}.
\end{aligned}
\end{equation}
This inequality is equivalent to
\begin{equation}
\begin{aligned}
     - \log \left( 1- \frac{W-w_1}{w_1} e^{-r^2/2} \right)\geq \frac{||\hat t-t_1||^2}{2}.
\end{aligned}
\end{equation}
To make sure that this inequality is well defined, we assume in addition that $\color{blue}{r\geq r_2}$ where $r_2$ is a positive constant such that  $\frac{W-w_1}{w_1} e^{-r_2^2/2}\in [0, 1)$.
So, if $r=\Delta/2\rightarrow +\infty$, then it satisfies both conditions $r>r_1$ and $r\geq r_2$ (actually, we can show that $r_1>r_2$). As a consequence, LHS converges to $0$ and thus $\hat t \rightarrow t_1$. Furthermore, since $-\log (1-x)\sim x$, we conclude that $\hat t$ exponentially converges to $t_1$, i.e.
$$||\hat t - t_1|| \sim C e^{-r^2/4},$$
where $C=\sqrt{ \frac{2(W-w_1)}{w_1}}$.
Hence, the Claim indeed holds true.
