Folks tend to make this harder than it really is. I've been killing whitetail deer for over 40 years, it ain't that hard! Many of those years we were allowed 3-4 tags. Centerfire calibers from .24 to .338. Add in round balls from a muzzle loader. Killed a few with TSXs, Sierras, Speers, Noslers, Hornady Interlock and Interbond, even a 75 gr. 25 caliber cast bullet at 2k fps.
Put a bullet constructed for deer, not varmints, in the right place and get out your knife.