It sounds like you're not using a rear bag and having a little trouble taking yourself out as one of the variables. May have been mentioned, but a rear bag will help a lot in isolating the rifle.