Different Ways To Calculate Conditional Probabilities

Question

I thought of the following math problem:

Suppose there is a basketball coach that wants to test the following hypothesis: The coach believes that once a player successfully scores a few baskets - the player is then more likely to score more baskets.

Suppose the basketball coach then carries out an experiment - the coach asks different players to shoot baskets and records the results. As an example, the data might look something like this (I simulated this using the R programming language):

    Player                                                               Baskets
1 Player 1                      Miss Miss Miss Miss Hit  Hit  Hit  Miss Miss Hit
2 Player 2                                               Hit  Miss Miss Hit  Hit
3 Player 3            Miss Miss Hit  Hit  Hit  Miss Hit  Miss Hit  Miss Hit  Hit
4 Player 4 Hit  Hit  Hit  Miss Miss Hit  Miss Miss Miss Hit  Hit  Miss Hit  Miss
5 Player 5                               Hit  Hit  Miss Hit  Miss Miss Hit  Miss

Based on this format of data - I thought of the 3 following methods to answer the coach's question:

• Method 1: For each individual player, count the number of times (in general, over all shots) that "Miss" follows "Miss", "Miss" follows "Hit", "Hit" follows "Hit" and "Hit" follows "Hit". Repeat this for all players, and then you can construct a 4-State Markov Chain with Conditional Probabilities

• Method 2: For each individual player, ignore everything except the last two shots. Then, count the number of at "Miss" follows "Miss", "Miss" follows "Hit", "Hit" follows "Hit" and "Hit" follows "Hit". Repeat this for all players, and then you can construct a 4-State Markov Chain with Conditional Probabilities only taking into consideration the last shot.

• Method 3: For each individual player, assign a value of "1" when a basket is "Hit" and a value of "0" when a basket is missed". Then calculate the average for each player (e.g. Hit, Hit, Miss, Hit, Hit = 1+1+0+1+1 / 5 = 0.8) but ignore the last basket.

The data should now be in the following format:

    Player Average_of_All_Baskets_Excluding_Last Last_Basket
1 Player 1                                 0.330         Hit
2 Player 2                                 0.500         Hit
3 Player 3                                 0.545         Hit
4 Player 4                                 0.500        Miss
5 Player 5                                 0.570        Miss

Based on this approach, a Regression Model (e.g. Logistic Regression) can be fit that models the probability of making the next basket, based on the scoring average of the player up until that point. We can also add other variables such as the "result of the second last basket" or the "average of the second last and the third last baskets" that can try to capture and benefit the model with more recent information. As a result, this model will try to estimate the conditional probability of making the next basket based on the history of the player. However, I do not know if using "lagged values of the response variable" will violate the assumptions of the Regression Model.

Thus, my question is - are all 3 methods valid approaches to estimating the conditional probabilities and testing the hypothesis of the coach? Are some of these methodologies more "valid" than others (e.g. perhaps some contain inherent biases and flaws)?

Thanks!

Answer 1

There is a rather simple way of testing this. For every player, calculate two proportions/probabilities from the data:

$(1)$ the probability of making a shot given they made the shot immediately before

$(2)$ the probability of making a shot given they missed the shot immediately before

You can accomplish this by sorting the data according to shot order, stepping through the results one at a time, and keep track of two totals: the number of shots made whenever a shot was made just before, and the number of shots made whenever a shot was missed just before.

Then, for every player, take the difference between these two probabilities you calculated. And finally, take the sample mean $\bar x$ over all these differences, and determine if there is a statistically significant difference between $\bar x$ and $0$. You can do this with a one-sample t-test. If the difference between $\bar x$ and $0$ is statistically significant, then this indicates the result of a given shot influences the result of the very next shot (i.e. the probability of making a shot changes depending on the result of previous shot). Otherwise, the results of the shots would appear to be independent of one another.

Just make sure you have a representative sample of the population of players you want to generalize to (assuming you're trying to conclude something about the "hot hand" phenomenon in general). With a large enough sample size, you could certainly test this phenomenon with individual players as well. In that case you would forgoe taking a sample mean and simply test the difference between their two probabilities to see if varies significantly from $0$.

I suggest this because sometimes less is more, and if you can validly test a hypothesis using a t-test instead of Markov chains and regression models, then go with t-test.

Answer 2

These are all at least somewhat valid depending on exactly what hypothesis you want to test. Method 2 is probably worse than method 1 for most purposes though.

Method 1 will be a good model if the result of the previous shot affects the probability of success on the next shot, but results of shots from further back don't matter as much.

Method 2 is the same as method 1 except it only examines the last 2 shots. That only seems useful to me if your hypothesis is "Success on the previous shot improves P(success) on the current shot, but only for the last shot in the session". That seems like a weird hypothesis to me (probably makes more sense to hypothesize that the relationship would hold for all shots in the session) so I would prefer method 1 over method 2.

Method 3 is a pretty different approach. This time the hypothesis is "It's not all about the #1 most recent shot; if you're having a good session overall then you'll have a better chance of making the next shot." That's a different totally reasonable hypothesis so it seems worth testing.

If I were investigating this situation then I would build both models 1 and 3, and test which one ends up fitting the data more accurately. That would then be evidence in favor of "having a good session overall is the important thing" vs "making the previous shot is the only thing that matters".

Of course there are other factors too: If you have tons of data, you might even try to fit separate models for each individual player to test whether they all respond the same way to "hot streaks", but if your dataset is smaller then it'll be best to stick to very simple models.