The standard algorithm to estimate power laws and assess the goodness of fit (or rather the plausibility of the fit) is the one following Clauset, Shalizi, and Newman, http://dx.doi.org/10.1137/070710111 . It basically works like so: The problem is that the exponent and the minimum value have to be estimated simultaneously. It therefore generate a list of all possible minimum values (which is finite as the sample is finite), then the exponent is estimated for all candidate minimum values, then the Kolmogorov-Smirnov statistic for each of the estimates (the maximum distance between the empirical CDF and the theoretical CDF with the estimated values) is computed and the estimate (pair of minimum value and exponent) with the best KS-statistic (smallest maximum distance) is chosen. Subsequently the goodness/plausibility for this fit is is estimated by generating a large number of artificial samples of the same size (drawn from a power law distribution with these parameters), repeating the process for each of them, and comparing the KS-statistic values. If the empirical sample results in a better KS-value for a certain share (Clauset, Shalizi, and Newman propose 10%) the distribution is deemed plausible.
I noticed that it should be more likely for a fit to be deemed plausible if only the very tail is fitted (i.e. a very high minimum value is selected). That way, the number of observations compared to the empirical distribution is smaller. It might therefore be the case that if a lower minimum value were selected, the plausibility of the fit is rejected while for a higher minimum value it is deemed plausible. This would be the case if the KS-statistic for the lower minimum value were better than for the larger minimum value. (And at the same time the KS-statistic of larger artificial distributions may be much better, thereby rejecting the plausibility of the fit.) This would lead to less close fits (if the minimum value is high enough) being more readily accepted than fits that are actually more accurate ones which would not be the intended result.
Is there some way around this problem? Did anyone have this problem before? Or am I mistaken with the initial observation at some point (i.e. the rejection of more accurate fits can not happen)?
Perhaps it would be nice to have a mechanism that selects the fit with the best KS-statistic that is plausible with some confidence value (say, 10%) chosen by the one who applies the test. (I understand that simply applying this to the mechanism would require running the entire plausibility-assessment procedure, generating large numbers of artificial distributions and fitting each of them etc., not just once, for the estimated value but as many times as there are possible minimum values (i.e. as many as there are observations in the set). This would for realistically large samples not be feasible from a point of view of what computation capacity would be required.)