Simen's answer won't work for yellow and blue mixing.
There are two types of mixing of colors: mixing light and mixing pigments. When you are mixing light, then adding something will make it brighter. When you are mixing pigments, you are taking light out... making it darker.
Hence the confusion. On a computer monitor, yellow (255R, 255G, 0B) plus blue (0, 0, 255) is equal to white. If you do some sort of average, you get gray. With pigments, you get green, of course.
Simulating what the pigments do is very difficult. You need to convert to reflectance and form a product, red with red, green with green, and blue with blue. Even that won't work as you expect, but it will highlight the fact that a mixture of pure yellow pigment and pure blue pigment will give you black, and not green.
To make it work out, you need to start with the correct pigments. Cyan (0, 255, 255) mixed with yellow (255, 0, 0) gives you green when you multiply.