Maximum value of $\lfloor a \rfloor^2+\lfloor b \rfloor^2+\lfloor c \rfloor^2$ under given conditions Consider $a,b,c \in \mathbb{R}-\{0\}$. A real valued function is defined as $$f(x,y,z)=2^y3^za^xb^yc^z+2^z3^xa^yb^zc^x+2^x3^ya^zb^xc^y$$ where $x,y,z \in \mathbb{Z}$ and $f(1,0,0)=4$ and $f(2,0,0)=6$. Find the maximum value of $\lfloor a \rfloor^2+\lfloor b \rfloor^2+\lfloor c \rfloor^2$.
Using given equations I got
$a+2b+3c=4$ and $a^2+4b^2+9c^2=6$ and also $2ab+6bc+3ac=5$. I have also obtained $a^2+b^2+c^2 \geq \displaystyle\frac{8}{7}$ but how to proceed further? Could someone help me with this?
 A: Sorry, just a partial answer:
If you substitute $x=a$, $y=2b$, $z=3c$, then your first two equations are $x+y+z=4$ and $x^2+y^2+z^2=6$.  These surfaces intersect in a small circle in the first octant, So that shows $a, b, c$ are all positive.  Your last inequality shows that at most one of them is greater than $1$ so the final answer is either $0$ or $1$.   You can work out that the center of the circle is at $(4/3,4/3,4/3)$ in $xyz$-space, and has radius $\sqrt{2/3}.$
I think that if you play with that (tedious) geometry a bit, you can prove that $a,b$, and $c$ are all less than one.  (I graphed the 3 surfaces in Maple, and this is certainly the case.)
A: I wrote about my whole thought process, including the stumbles in incorrect directions, in the hope that you can learn to come up with solutions yourself.
Let's continue from @Goddard's answer. But we will only use the geometry as an inspiration rather than a tool.
We want to prove that $a,b,c$ are all less than one. So we start working towards that. From the equations we get $y + z = 4 - x$, and $6-x^2 = y^2 + z^2 \ge (y+z)^2/2 = (4-x)^2/2$ by AM-GM. This means that  $4-8x+3x^2 \le 0$, solving which we obtain $\frac23\le x \le 2$. From this I started to doubt the initial claim.

Can you see why I used AM-GM that way? It is because I want to put some bound on $a$ (which is just $x$), and all we've got is the two equations relating $x,y,z$. So the essential thing to do is to fix some $x$, and see if there exists $y, z$ such that the two equations are satisfied. For example, if $x$ were 4, you can easily see that there could be no possible values for $y,z$, so we know that $x\ne 4$.
Now $x$ is just some fixed constant, so we'd better move it to where the other constants are. In other words, we move $x$ to the right hand side, $y+z=4-x$ and $y^2+z^2=6-x^2$. And my experience immediately tells me that there is an AM-GM inequality lurking here. Such experience can be summarized by "Squares overpower cross-terms; terms of higher exponents overpower those of lower ones". E.g. $x^2 + y^2 + z^2 \ge xy+yz+zx$ and $x^3 + y^3 \ge x^2y + y^2x$. Exercise: deduce these equalities about positive real numbers using AM-GM.

In particular, if we take $x = 2, y = z = 1$, we obtain $a = 2, b = 1/2, c = 1/3$. So $\lfloor a \rfloor^2+\lfloor b \rfloor^2+\lfloor c \rfloor^2 = 4$.
Well, this seems contradictory to the inequality $a^2 + b^2 + c^2 \le \frac87$ as claimed in the question. I'm not yet sure how such an inequality is deduced, but it is certainly a mistake (of either you or me).
Assuming that the mistake is yours (my apology if it is not the case, but I couldn't find one in my calculation, and you did not post how you arrived at such an inequality, so I can't check yours). We shall proceed to see that 4 is indeed the maximum. The most straightforward way is to prove an inequality of the form $a^2 + b^2 + c^2 \le k$, where $k < 5$.
Since we have $a^2 + 4b^2 + 9c^2 = 6$, we have (obviously, but it shall not be ignored!) $a^2 + b^2 + c^2 < 6$. So we are actually quite close. Since $\lfloor a \rfloor^2+\lfloor b \rfloor^2+\lfloor c \rfloor^2$ is an integer, the only gap is the possibility that $\lfloor a \rfloor^2+\lfloor b \rfloor^2+\lfloor c \rfloor^2=5$. A quick enumeration (i.e. listing out all possibilities) shows that this can happen only if one of the three squares is 4, with the others 1 and 0. But since $x \le 2$, and by symmetry of $x,y,z$, also $y \le 2$ and $z \le 2$; so $a \le 2, b \le 1, c \le 2/3$. So we must have $\lfloor a \rfloor^2 = 4$ (since nobody else could be 4), and $\lfloor b \rfloor^2 = 1$ (since $c$ is less than 1), in which case $a=2$ and $b=1$. But plugging that into the equation $a^2+4b^2+9c^2=6$ we see that $9c^2 = -2$, which is impossible. This finishes the proof.
A: $a+2b+3c=4$ ...(i)
$a^2+4b^2+9c^2=6$ ...(ii)
Given we have to find the maximum value of $\lfloor a \rfloor^2+\lfloor b \rfloor^2+\lfloor c \rfloor^2$ which has equal of $a^2, b^2, c^2$, we should maximize $a^2$ in (ii).
The minimum value of $a^2, b^2, c^2$ can be $0$ each. So individually, max value of
$a^2 = 6, b^2 = \dfrac{3}{2}, c^2 = \dfrac{6}{9}$ ...(iii)
So $\lfloor c \rfloor^2$ cannot be any value other than $0$.
Now, let's maximize $\lfloor a \rfloor^2$. From (iii), max value of $a = \sqrt6$ but any value greater than $2$ (and less than $3$) does not help. So, let's go with $a^2 = 4$.
Now, $4b^2+9c^2 = 2$. Even with $c^2$ as $0$, $b \lt 1$.
So, maximum value of $\lfloor a \rfloor^2+\lfloor b \rfloor^2+\lfloor c \rfloor^2 = 4$
