Something I have seen before(but not as bad as op is saying ) espcially with lite barrels exccessive "up" barrel pressure on the end of the stock can change POI a lot , only way i know to check this is "credit card bed " the action so as the barrel is floated or very lite contact and see how much the POI changes. Don't belive me take a known rifle that is floated and take it out of the stock and rap a bunch of black tape around barrel (to make lots of up pressure) where it hits the end of the stock put back in stock and see how high it shoots.