If you add up the total and divide among the number of people you get the amount each person should have paid. If you add up the bills each person has paid and subtract from the amount they should have paid you have the amount each person owes. These amounts should total to 0.
In terms of optimising, I understand that you want to split this up into person-to-person debts rather than person-to-group and group-to-person debts in such a way as to minimise the number of person-to-person debts. This looks rather like a bin-packing problem, so it is probably NP-complete to optimise.
(From a social point of view, the best solution is probably to have one trusted person collect from those who owe to the group and pay to those who are owed).