Not knocking what you're doing but I guess my hesitation would be with controlling diseases over time.
EHD is the killer down here. Our strategy was a hardy herd over freaky, non native 300" bucks. The more native, the more hardy. So far the herd has been extremely hardy. Knock on wood, because your concern is real.
Case in point will be the fifth and final buck. When you see him its obvious he is the farthest from native. He almost died this year, EHD most likely. Nothing but bones. Every rib, hip bone, and spine shown. It was a sad sight. I said he had no chance of living. To our surprise, he appears to be making a recovery, acting normal again and putting weight back on. He's still puny, but 30 lbs minimum heavier than he was 45 days ago. Our objective is clean, typical bucks, but dang it, this fellow now has me pulling for him.