You're absolutely right, all scopes can fail. I've personally seen 19 BRANDS (not individual scopes) fail to hold zero, or otherwise screw up, such as groups opening up to several inches at 100 yards, or refusing to adjust in any predictable way. The question is not whether they can fail, but how often they might.
If somebody doesn't shoot all that much, and only shoots relatively light-recoiling rifles at close ranges, then most scopes will hold up OK. But not everybody shoots .30-30's or .308's only at woods ranges, on one or two deer a year.
And scope failure is not always "just a little." Have seen scopes of several brands that seemed reliable suddenly shift POI several inches (or more) at 100 yards. One of the earliest examples in my experience was a well-known brand that shifted enough to gut-shoot a deer at 200 yards, which even in your world might be within acceptable range.
If you haven't seen the same thing, then I might suggest that your experience isn't nearly as extensive as you apparently believe.
“Montana seems to me to be what a small boy would think Texas is like from hearing Texans.” John Steinbeck
