Back when I was stationed in MT, a now good friend of mine that I was stationed with, had a Ruger 77 MKII 7mm Rem Mag, that shot similar to what you described. He gave it to me to fix for him, and while I don’t think he had ever cleaned the rifle, but the main issue was the torquing of the action screws. Once I got it all figured out, it shot a respectable 3 shot group.