my question involves a panel dataset. I have a number of firms over a period from 2001-2017 for various variables like total assets, net income, liabilities,….

Example of firm dataset which has some missing values:

Firm – date – total assets – net income – liabilities

Apple – 2001 – xyz – xyz – xyz

Apple – 2017 – xyz – missing – xyz

Amazon – 2001 – xyz – xyz – missing

Amazon – 2017 – missing – xyz – xyz

Now I want to calculate some descriptive statistics of the firms in the dataset. For example, I want to calculate the average total assets across all firms (to answer e.g. How big is the average firm? How much liabilities has the average firm?). As the dataset has some missing values, I should not just take the average across total assets (column total assets). Should I rather first calculate the average across each firm (despite some missing values) and then the average across all firm averages (giving all firms the same weight)?

Or are there any better descriptive statistic measures for this kind of dataset with some missing values?


Here is a method of 'imputation' that is sometimes used for other purposes, but might work for you if there are few missing values.

Suppose the 2013 figure for Amazon in missing. Find the Amazon row mean, the 2013 column mean, and the grand mean of the whole array. Then impute Amazon 2013 as Amazon mean + 2013 mean - grand mean.

This usually works OK if missing values are few and widely scattered. Also, with only a few missing values you would probably not have to assign different weights to firms with a missing value. But remember that imputation does not create information, it just helps to smooth over a few gaps.

There are better and more complicated imputation methods, but this is low-tech and it might work for you.

One caveat: For other parts of your analysis (and for archival purposes) you will want the original data without the imputed values. So keep a copy of the original data, and make sure the 'doctored' version is marked as such.

Mini-example: Consider the following matrix:

      [,1] [,2] [,3] [,4] [,5]
 [1,]    1    2    3    4    5
 [2,]    6   (7)   8    9   10
 [3,]    1    2    2    3    3
 [4,]   16   17   18   19   20

Suppose the 7 in position (2,2) is missing. Then roughly the 2nd row mean is 8 the second column mean is 7 and the grand mean is 7.8, so the imputed value would be 7.2. (Of course, this is cheating because I kept the 'missing value' when finding the row, column, and grand means.)

The correct means without using the missing '7' would be slightly different, and so the imputed value would also be slightly different. You can try it the 'honest' way by removing the 7 before taking the averages and see what you get.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.