Perhaps the same reason that catchers block the plate in baseball even though it is clearly against the rules and it is never called
If anything, in my experience the batters have crowded the plate and the catcher more often than vice versa. I caught for about 11 years and never once saw a batter called for crowding either. However, I did have idiot umpires tell me I couldn't move back anymore all because I didn't want smashed in the head with the bat.