# Tagged Questions

33 views

### How to appropriately normalize financial data?

I am evaluating a normalization of financial data. Two colleagues feel certain of a certain approach, and I am befuddled. The approach shapes the results how they want (i.e., brings possibly extreme ...
15 views

### Fit Conditional Probability Given Continuous Response

Given a sample of data, all the techniques aimed to find conditional probabilities of the response variable require that this is dichotomous or at least categorical (see multinomial logistic ...
39 views

### Analyzing data to determine seasonal indices?

The quarterly sales levels for the past four years of a leading tyre manufacturer are displayed on the table table How to analyze the data to determine the specific seasonal indices,and what ...
83 views

### 2D data fitting

I have some numbers as a function of 2 variables: $(x, y) \mapsto z$. I would like to know which function $z=z(x,y)$ best fits my data. Unfortunately, I don't have any hint, I mean, there's no ...
38 views

### Is there a scientific way to assign weights to historical data?

This is a common question I usually face while processing historical data. I have year on year data of an event for the past N years.I would like to assign weights to the data of these N years so that ...
15 views

### Clustering Problem

If I wanted to build a Dendogram but didn't have the means (example: computational) to generate it due to the very high number of nodes in my data, what would be my options then? Is starting out with ...
14 views

### Fitting discrete distribution (negative binomial) for early data values

I'm having some difficulties with fitting a discrete distribution function (I'm specifically using the negative binomial distribution). Here's my setting: I have a source of incoming items, each with ...
45 views

### Scaling data into $[-1,1]$

I have a data in the matrix for: \begin{bmatrix} 1 & 2 & 3 & 9 & 6\\ 8 & 2 & 7 & 4 & 6 \\ 1 & 2 & 8 & 7 & 4 \end{bmatrix} Each row corresponds ...
36 views

### Best way to generalize a distribution of data into smaller subsets?

Suppose I had a data set as follows: 168.95, 176.83, 178.13, 179.61, 179.44, 172.83, 173.37, 174.06, 174.94, 175.43, 175.73, 175.5, 173.78, 174.06, 172.71, 174.3, 178.38, 178.43, 177.18, 175.34, ...
43 views

### The number of nuts in a package of “Premium Cashews” is normally distributed with a mean of 433 and a standard deviation of 6 nuts.

The number of nuts in a package of "Premium Cashews" is normally distributed with a mean of 433 and a standard deviation of 6 nuts. Packages with fewer than 420 nuts or more than 445 nuts will be ...
18 views

### Continuity correction: Change P(2 ≤ x < 9) to continuous?

Convert discrete probability into continuous probability using continuous correction: attempt: Discrete: P(2 ≤ x < 9) therefore continuous should be Continuous: P(1.5 < X < 8.5) Is this ...
11 views

### Computing a threshold value for paired data

I have a dataset of vector pairs; for each pair ($x_1$, $x_2$) I have two values $V$($x_1$,$x_2$) and $U$($x_1$,$x_2$) (two metrics). I also know a threshold value $U^*$ such that any pair ($x,y$) ...
77 views

### How do I 'reverse engineer' the standard deviation?

My problem is fairly concrete and direct. My company loves to do major business decisions based on many reports available on the media. These reports relates how our products are fairing in ...
13 views

### How do I calculate the function of an (expected) logarithmic distribution, if I know the area under the curve & one data point?

I'm trying to get the function that lets me determine the value of y, where x is an arbitrary number. In this case, I'm looking at sales charts, and I'm making the assumption that the function is ...
10 views

### Likelihood function of truncated data

I am having a little trouble understanding the concept and derivation of the likelihood of truncated data. For example, if I want to find the likelihood function based on a sample from a ...
38 views

### Significance Tests & Confidence Intervals for Non-normal data

I have 2 nonnormally distributed samples of different sizes (N1~=N2). To evaluate whether there is a significant difference between these samples, I used the Mann_whitney U test (ranksum in MATLAB) ...
25 views

### Non-close-form Regression Research

As I try to process some physic experiment data that I don't have the closed form formula with unknown parameters, I have to use some regression models like polynomials or normal distributions . The ...
81 views

### Hypothesis Testing, simple against composite.

i've got some trouble understanding this exercise from Amemiya's Introduction to statistics: Given the density $f(x) = \frac{1}{\theta}$ from $0 < x < \theta$, and 0 elsewhere, we are to test ...
81 views

### Comparing annualised volatility from monthly and annual data

I fear there is a very simple answer to this question and its killing me that I can't see it. I am interested in calculating historical volatility: I have monthly index values starting in Jan 2005 ...
13 views

### Isolating clusters in data

This is a problem that I keep encountering in one form or another every so often. Given a large N dimension space with a sufficiently large (M) sample of points. [1] If I were to be asked to find a ...
84 views

### Mathematical model building with dependent and independent variables

I have been working with data and building models on data. I have developed models in regression using cubic and power series. It works fine for variables with one dependent and one independent ...
41 views

### Experimental Data Analysis

Suppose I have performed 2 different experiments to measure the number of cell in a mice. In the first experiment, we measure the cell in odd day (day 5,7,9,11,13) and in the second, we measure the ...
17 views

### mutual information for power low distributed data?

I have a dataset with power-low distribution, I would like to measure some kind of correlation/mutual information between the data to its class,(for feature selection task), Can I use ...
170 views

### Can you compare data sets with percentages with the Chi Square test?

I’m confused about the Chi Square test.. Apparently you have to use it when comparing frequencies only, but what if both data sets are percentages? e.g. comparing the percentage of smokers in specific ...
24 views

### data interpretation

let us consider following picture and question: INCOME: 1. Money from fund raising programmes, 2. Grant from the government, 3. Contributions from individuals, 4. Contributions from corporations, ...
79 views

### The correct probability distribution / way to analyze daily changes

I am working on a report which is being sent through to end users that should flag to them any "large changes" in the day-to-day values for the past 30 days for something we would assume the ...
138 views

### Mutual Information for clustering

I'm working on a document clustering application and decided to use Normalized Mutual Information as one of the measures of effectivenes. But I don't really understand how to implement this in that ...
64 views

### How to normalize these numbers for better visualization?

My dataset is like this: a1 4565380 a2 676477 a3 359939 ... b1 222431 b2 12222 ... g1 139 ... h1 134 i1 10 j1 11 and goes on.. The problem is when I ...
116 views

### Highest points in heart cycle graph

I'm making an application that reads the heart cycle from a device, and I've aimed to get this image: Now, I need to get the highest points that appear in every cycle in order to calculate the ...
20 views

### Where can I find a longitudinal study with a binary response?

I've spent the last couple of months trying to find real data (not simulated) of a longitudinal study, but I can't seem to find one. Any topic and time frame is fine, but the dependent variable must ...
28 views

### What is feature selection?

I have been participating in the kaggle digit recognizer competition and after some initial success someone suggested that I look into feature selection. I've done a bit of research but I am having a ...
197 views

### Exponent p-value generated in Excel

Excel gave me a p-value of 1.44909E-09 Notice is does not say .09 but 09 This is confusing me, I am trying to analyze my data but am stuck at this point. If it were E-9 it could be ...
42 views

123 views

### Test of goodness of fit - normal distribution of file sizes on HDD

I'm trying to prove, that sizes of files stored on HDD comes from normal distribution. I already made a java code that gave me list of all files on disk with filesizes in Bytes: ...
365 views

### Data Type Classification in Statistics

It is my understanding that in statistics one has 4 basic data types: nominal, ordinal, ratio, and interval. I see cases where people refer to "count data" (which is a random variable whose range is ...
64 views

### Determining Sample Size - Design Problem

I have a hypothetical problem were say I have a sample size of about 5000 people. I then will collect data on them say, how man calories of food they eat per day for a month. Now I can't obviously run ...
100 views

### Data Manipulation and Compression

I have this data: ...
61 views

### Finding outliers in a linear series

I have a series of observations made against time. Given perfect observations, the trend should be reasonably linear. As I expect there to be measurement errors, I wish to exclude them from further ...
49 views

### Determine a curve's dominant direction over time

I have a sequence of numbers in a certain order, for example {60,61,62,1,63,,64,102,65,66,200} If this is drawn as a graph, this will be a curve going upwards, but with some aggressive jumps upwards ...
670 views

### What is the distribution of a data set

I understand what the probability distribution is. I also have a personal understanding/interpretation of the concept of distribution of a dataset. Whenever I see this expression I imagine a graph ...
47 views

### arctic sea ice data by year

I want to demonstrate the usefulness of regression lines to my class, and I was hoping to do so with some data about the Arctic sea ice. What I'd like is the maximal area covered by the ice per year, ...
231 views

### Given a set of data (distance vs time) obtain a closer approximate for velocity

This is actually a physics experiment. I've collected a set of data Distance and time it takes for a cart to move down an incline plane. From here, I've calculated the average time (...