I always carry a set of ear plugs to use if I have time before the shot, which many times I do. I am deaf in my right ear, nerve damage when I was born, so I don't have much hearing to loose.
If an opportunity to take a shot arises without time, I don't even give it a second thought, I just shoot. I never wear protection when bird hunting either.
