Can sets have derivatives? I know how functions can be described, e.g. $y=x^2$ and in high school they teach you the general form, e.g. $x^2-y=0$ I believe so that later on they can abstract the notion of the curve into a set-theoretic notion, e.g. $S=\{ (x,y) : x^2-y=0\}$ and then the graph is just the same as coloring the elements of set $S$, but if the two are equivalent, how can you perform operations on the set in the same way you would on a graph?

How is the derivative of sets defined? How about integrals or functional equations?
 A: The derivative is still defined for functions. But now functions are sets, and you are only restricting yourself to special kind of sets (which are particular functions which have a derivative).
The point is that the notion of a derivative can be expressed in set theory. Namely, $f$ is differentiable at $x$ if the following limit exists:
$$\lim_{h\to 0}\frac{f(x+h)-f(x)}h$$
If it exists, then it has a value, so we can write the derivative as the set of ordered pairs $(x,u)$ such that $u$ is the value of the above limit.
Note that this does not mean that "the set has a derivative". It means that the function has a derivative. Even if you represent the function as a set.
The is similar. The point is that you can show there is a "faithful translation" from the language of calculus to the language of sets. So the entire thing can be expressed in the language of set theory, and therefore you can define the sets which define the integral of a function, and so on.
(This is especially true if you allow "normal mathematical operations" in the language defining the sets, like you have with $x^2-y=0$. But these too can be translated into set theory.)
A: The key observation is that you don't have just any set, you have a set with a lot of associated structure (the same is true for the function notation; you have to make a lot of assumptions to even be able to write down what "derivative" means, and then still most functions don't have a derivative; it's just that those functions are rarely interesting and thus you'll learn mostly about functions that do have a derivative).
A possible translation of the function derivative into set language could be as follows:
Define for any two pairs $p_1=(x_1,y_1)$ and $p_2=(x_2,y_2)$ with $x_1\ne x_2$ the associated slope $s(p_1,p_2) = (y_1-y_2)/(x_1-x_2)$.
Now consider a value $x$ in the domain of $f$. Then there exists an unique $p=(x,y)\in S$. We now say $S$ is differentiable at $x$ if for every sequence $(p_n)$ with $p_n=(x_n,y_n)\in S\setminus\{p\}$ and $\lim_{n\to\infty} x_n=x$, the series of slopes $s_n := s(p_n,p)$ is convergent to the same value $s$. The value $s$ is then called the derivative of $S$ at $x$.
The derivative function then can be written as the set
$$S' = \{(x,y): y \text{ is the derivative of } S \text{ at } x\}$$
Now having defined that, one might ask whether that definition could be extended beyond functions. For example, consider the set
$$S = \{(x,y): x^2+y^2=1\}$$
which is just the unit circle around the origin. Now this is clearly not a function, as there are generally two values of $y$ for a given value of $x$; however locally it looks very much like a function. Therefore it seems reasonable to actually have a derivative for that.
And indeed, it's quite easy to modify the definition to support this: Instead of calculating the derivative at a given $x$ (which no longer identifies an unique point) you just calculate the derivative at a given point. However we have to be careful to avoid the points at the same $x$, as there our slope is not defined. Therefore we get the following modified definition:
Be $p=(x,y)\in S$. We now say $S$ is differentiable in $p$ if for every sequence $(p_n)$ with $p_n = (x_n,y_n)$ that converges to $p$ and for which all $x_n\ne x$, the sequence $s_n=s(p_n,p)$ converges to the same value $s$. The value $s$ is then called the derivative of $S$ in $p$.
Now this explicit exclusion of $x_n=x$ seems inelegant; it is needed because for our definition of the slope, we get a division by zero if the points are actually lying on a vertical line. But there is in principle nothing wrong with vertical lines; it's just our representation of the slope that makes problems. So let's a closer look at the slope:
The slope is a quotient; such a quotient is determined by numerator and denominator, except that a common non-zero factor doesn't change the value, and a zero denominator is not allowed. So what if we relax that last restriction, and just say we have an object described by two numbers which may not both be zero, where a common factor doesn't matter?
This indeed gives a sound mathematical structure known as "real projective line". The members are not written as quotient, but usually just as pairs; however I think writing them as "$a:b$" also makes sense (as in "add milk and water in the relation $2:1$"). So if we describe our slopes not be real numbers, but by points in the real protective line, we can get rid of the problem (and as a bonus can also define the derivative at the points $(-1,0)$ and $(1,0)$ of the circle.
Thus we revise our definition of slope as follows:
The slope of a pair of points $p_1=(x_1,y_1)$ and $p_2 = (x_2,y_s)$ is the point on the projective line given by $(y_1-y_2):(x_1-x_2)$.
With this revised definition, we can simplify our modified derivative to:
Be $p\in S$. We now say $S$ is differentiable in $p$ if for every sequence $(p_n)$ that converges to $p$, the sequence $s_n=s(p_n,p)$ converges to the same value $s$. The value $s$ is then called the derivative of $S$ in $p$.
Note that there's no explicit mention of $x$ left in this derivative.
Now how would you define the derivative function? Well, obviously it would be
$$S' = \{(p,s): s \text{ is the derivative of } S \text{ in } p\}$$
Note that it is no longer a function from $\mathbb R$ to $\mathbb R$, but from $S$ to $\mathbb R\mathrm P^1$ (the latter being the standard notation for the real projective line).
A: In the most basic sense a derivative is thought of as a rate of change in one variable in terms of another.  Thus one variable is a function of the other.
Ex.  If $x^2 + y^2 = c$ we can for find at any point the rate of change of x in terms of y by making x a localized function of y at that point.  
The obvious question is what if it isn't localizably functionable.  Say the set is all $x,Y$ such that $x^2 + y^2 \le 4$.  What could a derivative at point (0, 3/2) mean for example?  Well, it doesn't really mean anything unless there are further conditions. (Which there can be.) 
A: In analysis there is a notion of a "derived set", which is the set of all limit points of another set. The prime notation for derivatives is often used for derived sets, but the connection between derived sets and derivatives of functions is superficial. The only meaningful relationship between the two is that both involve deriving a new object from another object of the same kind in such a way that the new object provides some interesting information about the original object.
