Part of what you seem to be talking about is either computability theory (also called recursion theory) or complexity theory, depending on whether or not the problems you're interested in are solvable by a computer. If you're primarily interested in solvable problems, complexity theory considers which problems can be used to efficiently solve other problems. Computability theory, by contrast, considers which non-solvable problems we could solve if we had access to the solution to certain other non-solvable problems. In both, there is a notion of combining multiple problems or reducing problems.
On the other hand, there's no good mathematical notion of a "heuristic", so far as I know, simply because "heuristic" is not really a well-defined notion - it just means "general strategy which has generally been right in practice". Numerical analysis considers methods of approximating a solution to a mathematical problem, but I don't think that's what you're looking for.
Questions about the "structure" of the problem space or solution space depends on what you mean by "structure" - both computability and complexity theory have a notion of the structure of the space of problems (neither makes a distinction between the problem and its solution, though).
Finding a heuristic for a problem is computer science's job, though again a numerical analyst might be interested.
Your last question, what kind of problems are most prevalent in our universe, isn't really a theory question at all - it's something that would be determined by experiment. For that, you're looking for something like physics, but don't get your hopes up.