I'm currently developing a helper function as part of an object processing function, and we programmers sometimes have to resort to math, if we're lucky :)
The main function is supposed to fetch and process a potentially large number of objects. To minimize overhead the objects are fetched in bulks. If there are few enough objects to fit into both computer memory and the allowed timeframe, they can all be fetched and processed in one go (bulk). If there are more than what these limits allow, the main function will do one bulk at a time.
The helper function is called "getSuggestedBulkSize" and its purpose is to suggest the maximum number of objects that can go into one bulk without violating the limitations of time and memory. To do this, it needs some statistical data, which is why the main function initially fetches and processes a small bulk of objects. It then stores both the time taken and the memory consumption of that bulk. Since both time and memory consumption may vary widely from bulk to bulk, the small initial sample may be a poor representative for future bulks.
My helper function bases its suggested bulk size on the averages of time and memory, but from the initial bulk and a few bulks on, these averages vary too much to be trustworthy. So far I've looked at their standard deviations to assess their trustworthiness, but it has been shown that the SD of averages from the first few bulks can be quite small, and suddenly grow wider before they "stabilize".
So my question is this; should I look to this stabilizing behaviour when trying to assess whether the averages are representative enough for the coming bulks or are there better perspectives to choose from? If looking at the stability of the averages is preferrable, how would I do that?
Please forgive my naïve approach to this - it's been a while since I last fiddled with statistics :)