I have an 03-A3 I rebarreled to 6.5-06 and am very happy with it.
As has been said I’d strongly consider “keeping it in the (06) family” not for the pressure reasons as yours is a modern rifle but for smooth operating.
Mine has the milled trigger guard tapered, a Timney Spoertsman trigger, Wisner “Model 70-style” shroud safety, and custom-built S&K SKulptered bases/rings.
I love custom Springfields.