Do you use anything special to clean off the lapping compound? I usually wipe with a paper towel then go over them with a Q-tip dipped in rubbing alcohol. Seems to take a while to get them cleaned up.
I use a paper towel to get the majority off, then wrap a shop rag around the bottom of the ring and hit it with Brakleen. Also, prior to lapping, I'll roll up a small wad of paper towel, and stuff it in the screw recess to keep out the lapping compound.