I haven't shot silhouette for many years but you are right, it is a lot of fun! I used a Remington 700 BDL 7mm08 with a 12x Leupold scope with a dot reticle and target knobs and shot in Hunter Rifle class. I started shooting at a range near New Braunfels when I lived in Austin. Recoil becomes an issue as you will fire a minimum of 40 rounds in the match plus sighters before the match. I shot 140 grain Sierras on chickens, pigs, and turkeys. For the rams at 500 meters, I used the 168 bthp Sierra. Rams are hard to take down, especially with a low hit. I lost some trying to use 140s on them but never with the 168s.
You can shoot the match with your hunting rifle at first. First time I shot was with a Ruger M77 270 and a 4X scope, just using hold over. I think I hit 8 out of 40! That many full power loads out of a 270 will get your attention. Give it a try, it's difficult but fun. It will make a better shooter out of you too.
