For Me never. In a light weight rifle as big as I would go is 3x9 and I’m in the process of taking my 3x9 off to put a 2.5x8 on.
I have never had a problem with my rifles holding zero and I mean like years.