Maximal possible area of a rectangle inscribed in the given right triangle 

  
*A rectangle is to be inscribed in a right triangle having sides of length $6$ in, $8$ in, and $10$ in. Find the dimensions of the rectangle with greatest area assuming  the rectangle is positioned as in the accompanying figure.
  
*Solve the problem in Exercise 6 assuming the rectangle is positioned as in the accompanying figure.


This is an applied min/max problem. I'm confused as to what to do. I've seen someone else use similar triangles, but I'd really prefer a different method. 
 A: Let's do this by setting the right triangle on a coordinate system so that the 8 cm side is on the x- axis and imagine the triangle with the 8 cm side horizontal and 6 cm side vertical. Let h be the length of the side of the rectangle perpendicular to the hypotenuse and let 'z' be the length of the side of the rectangle in the hypotenuse. Also let 'x' be the length of the hypotenuse above the rectangle ie above 'z' and let y be the length of the hypotenuse below the rectangle ie below 'z'. The triangles formed at each end are similar to the original triangle and so we have h/x= 8/6 or x= (3/4)h. Also h/y= 6/8 and so y= (4/3)h. Since x+ y+ z=10, (3/4)h+ (4/3)h+ z= 10 giving z= 10- (25/12)h. The area of the rectangle is hz= 10h- (25/12)h^2 The derivative of it is 10- (25/6)h and for area to be maximum this derivative should be equal to 0 giving h= 60/25= 12/5= 2.4 cm. Then 'z'= 10- (25/12)(12/5)= 5 cm.Therefore, area = 5(2.4)=12 sq cm
A: Your teacher will hate the method in this answer, so don't use it
on the homework.
For problem 6, draw the $6,8,10$ right triangle $\triangle ACB$ as shown below:

Choosing any point $P$ along the hypotenuse $AB$ and dropping
perpendiculars from $P$ to $Q$ on $AC$ and $R$ on $BC$,
you have inscribed rectangle $PQCR$ in $\triangle ACB$ in the position desired. 
Now reflect triangle $\triangle PQA$ across the line $PQ$
to obtain triangle $\triangle PQS$, 
and reflect triangle $\triangle PRB$ across the line $PR$
to obtain triangle $\triangle PRT$.
Notices that if you place $P$ closer to $B$ than to $A$,
as shown in the figure, then the vertex $S$ lies outside the original
triangle $\triangle ACB$; triangles $\triangle PQS$ and $\triangle PRT$
completely cover rectangle $PQCR$ plus the triangle $\triangle SCT$,
and therefore
$$
Area(\triangle PQS) + Area(\triangle PRT) > Area(PQCR),
$$
but since 
$$Area(\triangle PQS) + Area(\triangle PRT) + Area(PQCR) 
 = Area(\triangle ACB),$$
this implies that $$Area(PQCR) < \frac12 Area(\triangle ACB).$$
If you place $P$ closer to $A$ than to $B$, then for similar reasons
you will again find that $Area(PQCR) < \frac12 Area(\triangle ACB).$
But if you place $P$ at the midpoint of the hypotenuse $AB$,
repeating the exercise of reflecting triangles will quickly show
that $Area(PQCR) = \frac12 Area(\triangle ACB).$
This is the maximum obtainable.
For problem 7, draw the $6,8,10$ right triangle $\triangle ACB$ as shown below:

That is, drop a perpendicular from the right angle $C$ to the hypotenuse $AB$, meeting the hypotenuse at $H$. Now if we draw a rectangle as show
in the figure for problem 7, with one side lying along the hypotenuse,
part of the rectangle will lie within triangle $\triangle AHC$ and
the rest will lie within triangle $\triangle BHC$.
If the rectangle has vertices at the midpoints of legs $AC$ and $BC$,
then each part of the rectangle is half the area of the triangle within
which it lies, and therefore
$$ Area(\text{rectangle}) = \frac12 Area(\triangle ACB).$$
But if either of those vertices lies at any other point of one of the legs,
the other vertex also will lie at some point that is not the midpoint
of its leg; each part of the rectangle will have less than half
the area of the triangle in which it lies (as shown in the solution
of part 6) and therefore
$$ Area(\text{rectangle}) < \frac12 Area(\triangle ACB).$$
So once again the maximum area of the rectangle is 
$\frac12 Area(\triangle ACB).$
This uses none of the standard tools of min/max problems, 
just simple geometry, so it won't help you learn how to solve
min/max problems more generally.
But it may convince you that the dimensions of the rectangle for
problem 6 are $3 \times 4$
and for problem 7 are $2.5 \times 4.8$
(which is what you get when you put the vertices at the midpoints
of the applicable sides of the triangle),
so when you get these answers in the way you're supposed to,
you'll believe them, and if you get different answers then you'll know
you used the min/max methods incorrectly.
A: In the first case the equation of the line is $y/8+x/6=1$ and $A=xy=8x(1-x/6)$ and $dA/dx=8(1-x/3)$, so I think you can conclude from here.
In the second case, indicate with $(x,0)$ and $(0,y)$ the points on the axes.
Then look at the $3$ triangles left out.
They are rectangle, have one angle in common with the "big one", so .. they are similar to the it. Find their area, deduce from that of the "big one" ..
