small populations representativeness A question relating to home purchase loan denial rates, which I am examining as part of a work project. It's also a question that has come up in other aspects of my work. It is essentially about how much weight to put into figures drawn from small populations (not samples.) I know that when you are dealing with samples of a population you can talk about measures of variance (standard deviation, etc.) and margins of error. I have also seen, in researching this question, the coefficient of variance used in ratio data. But those only apply, as I understand it, when you are using a sample to estimate a population parameter.
What I want to talk about is something like this: Suppose that I were the second person to apply to purchase a home in my town, and that the previous applicant had been successful. I could look at his success and say, "Well, so far there has been a 100 percent success rate, so my chances are good!", but that would probably be a little naïve.
Is there a specific quantitative measure of how much weight you should put into small population figures for "probabilistic" purposes like this?
 A: I can think of two approaches:
1) If you think that your small town is similar to other towns for which you have data, you could consider many towns and have more observations to work with. Essentially, this would avoid your problem.
2) Otherwise, we are back to the "weight" question, which seems to imply that you are weighting the observations against something else, like your best guess without having seen any observations. In other words, you are talking like a Bayesian. So you could formalize your prior belief and then update it using the observations. See wikipedia on Bayesian inference
Also, if you want to make predictions about your chances, it would be helpful to have some predictors/covariates/independent variables, because other people may have different credit scores, or have a lower down payment, etc. If you had observable variables about the towns, the houses, and the people applying, you could use them in a probit or logit regression to make predictions for other people.
Also, you'll want to be clear about for whom you are estimating chances of success, because depending on what you want, selection bias could cause problems. People who know their chances are poor (for whatever reason) are probably much less likely to apply. So the people that you see apply are those whose chances are higher. A person chosen at random from the town would have a lower probability of success.
