I was able to solve the problem with a computer algorithm but am still not certain how it can be done mathematically. It was pointed out by @Greg Muller that the knapsack problem is applicable but I just don't have the mathematical prowess to apply it. If someone could show how that can be done please do.
Will share my logic here, hopefully it is useful to someone out there.
The first step is to determine the spell with the most damage per cast time.
This spell becomes the "baseline" spell since it will guarantee the highest damage per second. Meaning, you should always cast this spell if the following 2 conditions are met:
1) The baseline spell is available (not on cooldown).
2) You are not currently casting a spell.
So it then becomes a matter of filling in other spells while the baseline spell is on cooldown. Between (cast time) and (cooldown - cast time). However, some overlapping can occur (rule 2 above is false).
It then becomes a matter of recursing through all non-baseline spells to find all sequences of spells which do not violate the 2 rules.
For spells which DO overlap you must penalize them for potential damage the baseline spell could have done (up to its maximum damage).
Take for example, 2 spells
1: 300 damage, 3s cast time, 10s cooldown
2: 290 damage, 3s cast time, 3s cooldown
The most damage comes from the sequence 1 - 2 - 2 - 2. Which causes an overlap of 2 seconds into a potential #1 cast. However, this is still beneficial since if you dont cast the third spell (ie. 1 - 2 - 2) you will do 880 damage with 1 second to spare. If you cast the extra #2 spell you will do 1170 - 2 second of #1 which is 200. So 970 damage is your relative damage.
This algorithm is significantly faster than other algorithms which look for sequences that match a target goal: ie. time limit or damage.