Say I have a function and I want to find the maximum and minimum on an interval, I would first differentiate the function and equate it to zero. Then use this value and the two ends of the interval to find the max and min. If, however the the derivative is undefined when it equals zero do I just scrap that as a value and use the bounds of the interval in the original function? does this mean that when its equal to zero its a critical point?
If, however the the derivative is undefined when it equals zero...
How can it equal zero and be undefined? In any case, points where the derivative is undefined are candidates for the max/min (in addition to the endpoints of the interval and points where the derivative is zero), so you should check the value of $f$ at those points too.
The candidates for maximum and minimum of a continuous function on an interval are the zeros of the derivative, the endpoints, and the points where the derivative does not exist.
Whether the term "critical point" includes a point where the function is not differentiable is a question of convention: most authors do not call these critical points, but some do.