Its a bedding and pressure on the barrel problem... I'd also check the mag box and make sure it's not binding or torqueing the action. The mag box should not be tight between the floor plate and the bottom of the action...
I had a mag box binding up on me. I was about to chuck the rifle into the tree line. Went home took a belt sander to the mag box, went back to the range problem of wandering/shifting point of impact solved.
