The interesting thing is the Lott really wasn't designed to push 500's @ 2300 fps. It was designed to easily drive 500's 2100 fps. With the powders available at the time of the 458 wins introduction, 500 @ 2100 wasn't easily achieved with the smaller case.
As to an extra 50 fps being noticeable when pushing 500 gr pills, absolutely! You can even notice differences in recoil between different powders. I tried VV N550, and while I could only get 2200 w/ 500's, it was an unpleasant experience.