I just skimmed through the replies and didn't notice anyone mention that your bullets may be unstable. If so, they get progressively worse as they go down range. At 100 yds, they hit nearly point on, and given a descent group but as they continue downrange, they become increasingly "wobbly" (technical term), which destroys accuracy.
It also really increases drag so the bullet begins to slow much faster and thus, the bullet has a longer time to drop. Thus the much lower average POI. That seems to be the case in the photo you have shown but it's hard to be sure.
I would suggest a substantially shorter bullet - but check your rifle's crown first. Might be other causal mechanisms too, but those are two easy ones to try.
The too symptoms taken together add up to a ballistic problem. I would further guess that if you shot at clean cardboard (or better clean foam board) you would find the 200 yds holes are larger, more oblong than the 100 yds holes are. That would be the third symptom to look for.
