So given an objective function, lets say this is utility, and given a constrain, lets say budget, If I am asked to maximize utility I do lagrange like (objective function-utility)-L(constrain) and take partial derivatives , find lambda and so on... but imagine I am asked to minimize utility,how do I proceed then?
The Euler-Lagrange equation tells when the objective function is stationary (equivalent to a vanishing derivative). It does not tell whether it is at maximum, minimum, or neither. Other arguments are needed to determine what type the stationary point is.
To minimize utility, you would use the same Euler-Lagrange equation as to maximize the utility; both are stationary points.
If there is a boundary, the optimization needs to be done on the interior and the boundary, but the boundary optimization will have the added constraint that the solution is restricted to the boundary.