Yes, not always but I do some of the time. (It is legal in the states I hunt.) I tend to leave the silencer off when hunting thicker brush, just because of the length, but otherwise really like hunting with a quieter rifle.
Animals still hear the noise and react, but often don't seem to know what it is or where it came from. Several times now I've had other deer walk out into view just a couple minutes after shooting; they seem to wonder what's going on.
