<p>Looking for one that is fast enough and still graceful with memory. The image is a 24bpp  System.Drawing.Bitmap.</p>

If you need an exact number, then you are going to have to loop over all of the pixels.  Probably storing the color and a count in a hash is the best way to go because of the sparseness of the colors.
Using the Color.ToArgb() in the hash instead of the color object would probably be a good idea too.
Also, if speed is a major concern, you don't want to use a function like GetPixel(x, y) -- instead try to process chunks at a time (row a time).  If you can, get a pointer to the beginning of the image memory and do it unsafe.