I have recently gathered a large amount of data, representing the time elapsed (in seconds) between two consecutive comments on the same post of a social network website.

I want to find a probability distribution giving a close approximation of these results. I was initially thinking that a geometric distribution would be the best, but the distribution of this data looks as follows (only pictured up to 60, but it goes much farther):

enter image description here

I have looked into several other distributions (beta, gamma, poisson, exponential...), but I can't seem to make them fit these results, even approximately.

What kind of distribution should I be looking into to model this situation?

EDIT: I can't really access MatLab, but I do have RStudio. Using fitdistr from the MASS package, a Weibull fit gives me parameters of 0.41501 and 69.57486, which seems quite bad here (orange = fit): enter image description here

The gamma fit fails entirely during the gradient computing phase.

R summary:

  Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
  1.0       6.0      13.0     488.2      57.0 1716000.0

  stddev    Q90%    Q99%
  6364.06   435     10113.24
  • 2
    $\begingroup$ Weibull is a general family of exponential distributions. You could try that - maybe even use MLE to get a fit! $\endgroup$ – Sean Roberson Aug 23 '17 at 19:57
  • 3
    $\begingroup$ That looks pretty gamma-like to me, did you try taking an MLE fit? It might also be good to plot the tail on a semilogy scale (since it can be hard to distinguish an exponential tail from a power law tail on a standard scale). $\endgroup$ – Ian Aug 23 '17 at 20:08
  • 1
    $\begingroup$ Actually, can you export the data file that generated this? I'd be interested to play with it a little bit. $\endgroup$ – Ian Aug 23 '17 at 20:45
  • 1
    $\begingroup$ (1) It would be easier to speculate cogently if you would give the sample mean, median, SD, and perhaps 75th, 90th, and 99th quantiles. (2) If you have access to Minitab, you might try the menu path STAT > Quality > Individual distribution ID. It attempts to fit a variety of right-skewed distributions (Weibull, lognormal, gamma, exp'l, etc.) and shows GOF P-values and prob plots that help judge fit. But take care: Weibull or gamma will appear to fit exp'l data best best because exp'l is subfam of both Weib & gamma; extra params allow overfitting. // R has ID library, I've not tried it yet. $\endgroup$ – BruceET Aug 23 '17 at 20:52
  • 1
    $\begingroup$ @Ian Here is the raw data: pastebin.com/NEsPJSkP. I also updated my post with relevant information, including some of the values you asked for. $\endgroup$ – pie3636 Aug 23 '17 at 21:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.