I suspect when guys say they can't/won't "handle" a .500 S7W, they don't mean carrying it around in a holster, but actually holding and shooting it.

My choice is the .480, also. Never shot or really cared to shoot a .454. I do have, and use, a .45 Colt Blackhawk, but don't try to load it to .454 levels. In my .44's and the .45 I tend to shoot 300+ gr hard cast at +/- 1000 fps, and they really get the job done with less recoil, which is good for my wrists and my revolvers. I look at the .480 as more of a good thing. 400r or heavier hard cast at the same velocity makes a bigger hole and should be hitting "harder". I would probably like to have a .480 Bisley, but my Super Redhawk does a fine job.