HB
I have had rifles do the same thing. One time I had found a load which on consecutive 3 shot groups shot about 3/8". Went home and loaded 50 rounds and a few days later went to fine tune POA shot 1 1/2" - 3" groups. Same bullet, same COAL, individually weighed each charge, once fired brass etc,etc. I never duplicated that original series. The rifle did shoot another load with a different bullet/powder very well and consistently. This is why it is better to shoot 5 - 10 shot groups to make sure that the rifle shoots consistently and has not randomly produced a "good" group.
You state that the B&C stock was bedded for another rifle, could it possibly have the action under tension/ torsion causing a change in POI with each shot?