Of course not. Just shoot whatever rifle you want, use decent bullets, try your best to make good shots, and enjoy yourself. I love to hunt but I also love rifles. In a given year I’ll shoot “big game”, mostly pigs, with rifles from 17 to 45 caliber. They all work when used with some level of precision and all will produce a rodeo when poor shots are made. I just don’t see the need for the trepidation and/or disdain for the use of 22 calibers on deer sized game.