Guess I got lucky, so far with three different bullets and 3 different powders mine has shot everything well. So far the sabots I like best are the Harvestor green ribbed sabots. The ones that came on the 300 gr, Nosler Ballistic Tips are good also. You are right about not needing really powerful loads for most deer hunting.
With my first Knight UL, 80gr of BH with 300gr .452 XTPs punched through a number of deer for my sons, including one that went 172lbs dressed. Bullet weight matters too, I guess. Have since gone to 100gr, but for no particular reason, except maybe that it might shoot a bit flatter. POI up to 100 seems pretty much the same.