I don't use all copper bullets but I'm certainly not prejudiced against them.
I do think most of the concern regarding lead bullets is a red herring as far as humans go. As stated in other posts above, the primary danger to children comes from exposure to lead oxides, especially lead paint dust or chips.
Metallic lead is not near as big a deal provided some common sense is used. Accidentally swallowing a bird shot doesn't pose near as much risk as handling an old piece of lead painted board and then not washing your hands before eating.
I often take deer to be processed as well but I do go ahead and cut away the blood shot meat and try to remove bullet fragments while field dressing.
Since your family eats game often, just make sure your game processor is on the same page as you. Talk with him or her outside of game season.