How to extract an individual's (normalized) contribution from a group? To make this concrete assume you have two teams playing against each other. Each team can only have n players on the field, and may be substituted for a different player at different times  during the game (say hockey or basketball).
You have a list of all the game's important events that can occur.
It's easy to calculate the average of a statistic for each team, or each player for one game.
But imagine you had a league of teams. Teams won't play against all other teams and players won't play against all other players.
What mathematical techniques exist in order to extract statistics about a player which are normalized (as if the player had played with average players against all average players)? In other words, how do you extract statistics for each player so that the statistics can be truly compared with each other?
You could imagine a player (#1) who plays only against crappy teams who looks very good because their stat averages are higher then the league average. There might be another player (#2) who plays against very good teams and therefor his stats are lower. But against normalized competition player #2 would have better stats then player #1.
How do you get at those normalized values when the players play against (and with) incomplete subsets of the other players?
Pointers to ideas or methods and/or actual suggestions appreciated.
 A: The simple answer is: it depends on the sport. And on what you're trying to measure.
Statisticians have come up with some concepts for various sports, and despite widespread chatter about them, they fail to provide any real predictive power. Furthermore, many of them rely on assumptions that are practically begging the question, so I find them to be dubious at best. Even further exacerbating the situation is that sporting events take place over a finite non-zero window of time, yet the common statistical methods all but ignore time series effects on multiple scales (e.g. in-game vs. over the course of a season vs. over the course of a career). So what we're left with is "advanced counting and dividing." These methods are rarely cross-validated and retro-actively analyzed. Time marches on, and poor past predictions are easy to bury.
Some metrics in the NHL, for example, consist of "Shots on Goal," "Fenwick," and "Corsi." Simply put, Shots on Goal are any pucks directed on net that either score or induce a goalie save. Shots that hit the post or miss the net are not considered Shots on Goal. Fenwick further considers shots that miss the net. Corsi considers shots that are blocked by defenders.
Yes, this is what passes for "advanced statistics" in the NHL community.
To try to feel smart, lots of hockey bloggers like to measure players' individual SoG/Corsi/Fenwick by looking at who's on the ice. They can then measure, for instance, Corsi-for by looking at the difference of offensive Corsi (Corsi attempts by a player's team with the player on the ice) and defensive Corsi (Corsi attempts by the opponent when the player is on the ice). These are normalized to a percentage scale, so that a CF% of 50% means that the player is on the ice for as many Corsi-for as Corsi-against.
Corsi-rel is then basically taking this number and measuring it against the rest of the player's team.
Sometimes, to make arguments against who's better, people will derive other quality of competition metrics based around the same concepts. For example, QualComp in the NHL is measured as a +/- rating (the difference of goals for by goals against) normalized by player time-on-ice. I'm eliding a bunch of details because I find this all to be very droll. But it brings up an interesting point: confounding.
The overarching issue is that most of the regression models used are using metrics that are based in part on the same exact data. A player's +/- rating is wholly embedded in their Corsi-for/Corsi-against rating, for instance. Without controlling for these factors, one essentially begs the question.
Ultimately, sports are a spatio-temporal process, and without better positional tracking (which the NBA is developing, the NFL will begin using this season, and the NHL has been researching), there is little way to statistically rigorously determine an individual's personal contribution.
