Those big bullets work well without shooting up a bunch of meat. I was hunting with a buddy who used a 30-06 Federal 220 factory load to shoot a mule deer fawn at right around 250 yards. Bullet expanded adequately and deer was killed efficiently without hardly any bloodshot meat.
I personally used a 30-06 Lapua Mega 200 gr. RN to shoot a couple of roe deer in Germany. They are about half the size of Whitetails. Also used that bullet on a couple of young wild boar. Perfect results.
The only thing a faster lighter bullet would have done is shoot up more meat, it would not have killed deader, and could not have killed quicker. The recoil difference is irrelevant to an experienced shooter.