I currently have a .358 Norma Mag and .375 H&H Mag (and I have had a .338 Win Mag).
If you handload go with the .358 Norma for versatility! I shoot bullets ranging from 110gr to 310gr. Furthermore, there is a huge variety of .35 cal bullet molds if you cast, and realy few for the .338 or .375 by comparison.
The .358 is the best combination of flat shooting, large bullet diameter and mass vs recoil -- if you can handle the recoil of course!
The .358 is a dandy (if very gruesome!) varmint rifle. A 125gr Hornady HollowPoint designed for .357 Magnum handgun velocities like 1200fps will really have a noticeable effect on a varmint when it connects at 3000fps! You can push them even faster, but some bullets destruct in mid air. Forget all this if you value the pelt though... !
If you don't handload, go for the .375 or .338. Rifles in .375 tend to be overly heavy, so in general I'd give the nod to the .338 for a non-handloader who has to carry his own rifle.